BrainLearn

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Dann Corbit
Posts: 11626
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: BrainLearn

Post by Dann Corbit » Fri Jul 03, 2020 5:16 am

With the transparent memory access that AMD is working on (and also having the CPU read and write HBM memory transparently) I think that rather than replacements, we will see hybrids. Perhaps the only hybridization will be a shared hash, but even that would be awfully nice. Imagine if every place where LC0 was stronger was written to the hash and Sf read it, and every place where LC0 was weak was written to the hash and LC0 read it and there was no need to copy to and from video memory because we can just *use it directly* what sort of chess that would produce. I think it would be revolutionary. I also think that an AND/OR proof search using GPUs searching for checkmates would be incredible. I guess in ten years we will see chess hardware/software combinations that would slap the current software and hardware silly.

Wonderful simile, by the way.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Leo
Posts: 969
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: BrainLearn

Post by Leo » Fri Jul 03, 2020 4:31 pm

amchess wrote:
Wed Jul 01, 2020 1:16 pm
Zerbinati wrote:
Wed Jul 01, 2020 1:00 pm
amchess wrote:
Wed Jul 01, 2020 12:46 pm
Zerbinati wrote:
Wed Jul 01, 2020 12:16 pm
@amchess I rewrite it in more detail because maybe you haven't seen it

For information however:
in the infinity gui neither of the 2 engines works
an empty file is created in the installation folder of the gui and not the engine
The uci commands to activate the type of learning of Shashchess are not displayed in the options
Eman successfully creates the BIN file
both on Playchess and on Infinity
You are both programmers
I hope you will solve it
Eman does'n't have Q-learning.
Even in linux, for example, Arena doesn't display an uci option ("Petrosian").
Conversely, in windows, it works perfectly.
So, this is a gui bug, not the engine and this is never the first time.
For example, chessbase updates continually their guis also to solve this kind of problems.
Anyway, we can show you, after a match play in Fritz gui, experience files are created.
Sorry, but we can't do more than this.
Finally, as we wrote in the readme file, Q-learning is for selfplay: why to use it in playchess or infinitychess?
Andrea I refer to the standard mode
this problem was also present in version 11
I changed the code and deleted the new version drop down menu
have the possibility to compile a version with the standard default learning activated
in fritz gui it works well
while in the infinity GUI it creates an empty BIN file
also by installing the engine in the GUI installation folder
If InfinityChess gui can't even show the uci option, it's clear it's a gui problem, not the engine.
So, sorry, but we can't help you.
What is Q-learning? I never heard of it before.
Advanced Micro Devices fan.

amchess
Posts: 148
Joined: Tue Dec 05, 2017 1:42 pm

Re: BrainLearn

Post by amchess » Fri Jul 03, 2020 6:43 pm

Leo wrote:
Fri Jul 03, 2020 4:31 pm
[What is Q-learning? I never heard of it before.
https://groups.google.com/forum/#!topic ... IzmiSCB8lg

Leo
Posts: 969
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: BrainLearn

Post by Leo » Fri Jul 03, 2020 6:59 pm

amchess wrote:
Fri Jul 03, 2020 6:43 pm
Leo wrote:
Fri Jul 03, 2020 4:31 pm
[What is Q-learning? I never heard of it before.
https://groups.google.com/forum/#!topic ... IzmiSCB8lg
Thanks. Ill read it.
Advanced Micro Devices fan.

peter
Posts: 2160
Joined: Sat Feb 16, 2008 6:38 am
Full name: Peter Martan

Re: BrainLearn

Post by peter » Fri Jul 03, 2020 7:14 pm

amchess wrote:
Fri Jul 03, 2020 6:43 pm
Leo wrote:
Fri Jul 03, 2020 4:31 pm
[What is Q-learning? I never heard of it before.
https://groups.google.com/forum/#!topic ... IzmiSCB8lg
Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?
Last edited by peter on Fri Jul 03, 2020 7:19 pm, edited 2 times in total.
Peter.

matejst
Posts: 216
Joined: Mon May 14, 2007 6:20 pm
Full name: Boban Stanojević

Re: BrainLearn

Post by matejst » Fri Jul 03, 2020 7:15 pm

Great comments, Uly and Dann!

My hope too is that soon we will learn to combine both approaches, and have a better understanding of the knowledge brought by NNs. A NN that could be incrementally trained -- just like the experience.bin of BrainLearn -- would be an excellent idea. Every engine could become unique that way, play some positions better, etc. It could be very interesting.

amchess
Posts: 148
Joined: Tue Dec 05, 2017 1:42 pm

Re: BrainLearn

Post by amchess » Fri Jul 03, 2020 7:28 pm

peter wrote:
Fri Jul 03, 2020 7:14 pm
amchess wrote:
Fri Jul 03, 2020 6:43 pm
Leo wrote:
Fri Jul 03, 2020 4:31 pm
[What is Q-learning? I never heard of it before.
https://groups.google.com/forum/#!topic ... IzmiSCB8lg
Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?
The best is to use Q-learning for self play.
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.
There are some principles similar to NN-network in Q-learning, but the programming paradigm is different.
Q-learning bin files don't have depth information compared to the standard, but a more refined score, based on a gameLine vector.
NNUE-learning uses an neural network.

peter
Posts: 2160
Joined: Sat Feb 16, 2008 6:38 am
Full name: Peter Martan

Re: BrainLearn

Post by peter » Fri Jul 03, 2020 7:35 pm

amchess wrote:
Fri Jul 03, 2020 7:28 pm
peter wrote:
Fri Jul 03, 2020 7:14 pm
amchess wrote:
Fri Jul 03, 2020 6:43 pm
Leo wrote:
Fri Jul 03, 2020 4:31 pm
[What is Q-learning? I never heard of it before.
https://groups.google.com/forum/#!topic ... IzmiSCB8lg
Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?
The best is to use Q-learning for self play.
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.
You mean define two times same engine (e.g.BrainLearn 12) with different names in the GUI and each one adressing an experience.bin of it's own?
So copy the .exe into two directories with storing experience in each one separately to merge those two then?
Or adress same experience.bin by same .exe with only two different names of .eng- files for the GUI to organize engine- engine- tournament between two times same .exe?
Peter.

amchess
Posts: 148
Joined: Tue Dec 05, 2017 1:42 pm

Re: BrainLearn

Post by amchess » Fri Jul 03, 2020 7:41 pm

peter wrote:
Fri Jul 03, 2020 7:35 pm
amchess wrote:
Fri Jul 03, 2020 7:28 pm
peter wrote:
Fri Jul 03, 2020 7:14 pm
amchess wrote:
Fri Jul 03, 2020 6:43 pm
Leo wrote:
Fri Jul 03, 2020 4:31 pm
[What is Q-learning? I never heard of it before.
https://groups.google.com/forum/#!topic ... IzmiSCB8lg
Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?
The best is to use Q-learning for self play.
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.
You mean define two times same engine (e.g.BrainLearn 12) with different names in the GUI and each one adressing an experience.bin of it's own?
So copy the .exe into two directories with storing experience in each one separately to merge those two then?
Or adress same experience.bin by same .exe with only two different names of .eng- files for the GUI to organize engine- engine- tournament between two times same .exe?
I mean two times the same engine with different names in the GUI and each one adressing an experience.bin of it's own for self play.

peter
Posts: 2160
Joined: Sat Feb 16, 2008 6:38 am
Full name: Peter Martan

Re: BrainLearn

Post by peter » Fri Jul 03, 2020 7:51 pm

amchess wrote:
Fri Jul 03, 2020 7:41 pm
peter wrote:
Fri Jul 03, 2020 7:35 pm
amchess wrote:
Fri Jul 03, 2020 7:28 pm
peter wrote:
Fri Jul 03, 2020 7:14 pm
amchess wrote:
Fri Jul 03, 2020 6:43 pm
Leo wrote:
Fri Jul 03, 2020 4:31 pm
[What is Q-learning? I never heard of it before.
https://groups.google.com/forum/#!topic ... IzmiSCB8lg
Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?
The best is to use Q-learning for self play.
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.
You mean define two times same engine (e.g.BrainLearn 12) with different names in the GUI and each one adressing an experience.bin of it's own?
So copy the .exe into two directories with storing experience in each one separately to merge those two then?
Or adress same experience.bin by same .exe with only two different names of .eng- files for the GUI to organize engine- engine- tournament between two times same .exe?
I mean two times the same engine with different names in the GUI and each one adressing an experience.bin of it's own for self play.
Ok, thanks again, will give it a try.
Peter.

Post Reply