Page 13 of 16
Re: Trying out the Leela Hybrid engine (NN with AB)
Posted: Thu Mar 07, 2019 3:24 pm
by sincerelybrent
Jaglo wrote: ↑Thu Mar 07, 2019 10:26 am
Hi KillerDucky. Your project is very interesting. But...
The biggest weakness of LC0 is endgame and tactics. Leela handles other elements well, and in the positional game she outperforms the classic engines.
I have tested eigenmann (100 positions) and AH_tactics (250 positions). Stockfish 10 plays them almost perfectly (97/100 and 246/250).
The results of LC0 (network 32988) are 68% in both cases. ...
I have chosen those positions that LC0 can't solve (and Stockfish does). The eigenmann subset consists of 32 items. The AH_tectics subset is 79 positions. In this subset I tested Leelafish with different settings. The best results I received when:
auxengine-depth=20/25
auxengine-boost=100.00
...
What other depth and boost combinations have you tried? And where can I download the AH tactic tests? Thanks,
Re: Trying out the Leela Hybrid engine (NN with AB)
Posted: Thu Mar 07, 2019 7:20 pm
by Paloma
@ sincerelybrent
AH_Endgames-250 and AH_Tactics-250 Results and epd's
http://www.talkchess.com/forum3/viewtop ... =2&t=41023
use the links in the first post
Re: Trying out the Leela Hybrid engine (NN with AB)
Posted: Thu Mar 07, 2019 8:28 pm
by Jaglo
What other depth and boost combinations have you tried? And where can I download the AH tactic tests? Thanks,
auxengine-threshold=10/100/1000
auxengine-depth=15/20/25
auxengine-boost=50/100
auxengine-follow-pv-depth=4 (1? 10? - I do not remember)
I didn't check all combinations, only after the change I did a test and if it gave an improvement I moved it with other parameters, trying to achieve further improvement. A deeper depth is needed for Stockfish to find a solution at all (approx. 20-30). Setting the boost to 100 was also important. I haven't noticed the influence of other parameters.
The Eigemann website no longer exists. I once downloaded an epd and converted it to chessbase, and now I don't know how to convert it back to epd. AH_Endgames_250 should be similar.
Re: Trying out the Leela Hybrid engine (NN with AB)
Posted: Fri Mar 08, 2019 12:58 am
by sincerelybrent
Re: Trying out the Leela Hybrid engine (NN with AB)
Posted: Thu Mar 14, 2019 1:25 am
by KillerDucky
v0.21.0-fish-v0.8 released!
https://github.com/killerducky/lc0/wiki/LeelaFish
https://github.com/killerducky/lc0/releases
Important note: Stockfish 10 official release has a bug that Leelafish hits more often than normal. To workaround this: 1) Use a recent dev version (after 2018-12-16) or 2) Set Threads=1 in AuxEngineOptions.
Change notes:
* Merge Lc0 bugfix, required to run T40 nets:
* Improved folding of batch norm into weights and biases - fixes negative gamma bug. (LeelaChessZero#779)
* Now Leelafish is based off official release v0.21.0 of Lc0 (previously v0.21.0-dev).
Re: Trying out the Leela Hybrid engine (NN with AB)
Posted: Thu Mar 14, 2019 3:49 am
by MikeB
KillerDucky wrote: ↑Thu Mar 14, 2019 1:25 am
v0.21.0-fish-v0.8 released!
https://github.com/killerducky/lc0/wiki/LeelaFish
https://github.com/killerducky/lc0/releases
Important note: Stockfish 10 official release has a bug that Leelafish hits more often than normal. To workaround this: 1) Use a recent dev version (after 2018-12-16) or 2) Set Threads=1 in AuxEngineOptions.
Change notes:
* Merge Lc0 bugfix, required to run T40 nets:
* Improved folding of batch norm into weights and biases - fixes negative gamma bug. (LeelaChessZero#779)
* Now Leelafish is based off official release v0.21.0 of Lc0 (previously v0.21.0-dev).
+1 good work - appreciate it!
Re: Trying out the Leela Hybrid engine (NN with AB)
Posted: Thu Mar 14, 2019 7:03 pm
by kalikoba2019
Nice one, and , if the auxiliary engine accepts to read and play .bin books, would it be possible to let Leelafish play its auxiliary engine book if it exists ?
Re: Trying out the Leela Hybrid engine (NN with AB)
Posted: Thu Mar 14, 2019 9:44 pm
by Jaglo
Leelafish v0.8
net: 32988
auxengine=Stockfish 10 (Hash=1024;Threads=1)
auxengine-depth=25
auxengine-boost=100.00
Syzygy 345
eigenmann: 84%
Perhaps the result for the endings would have been better if the depth of analysis had not been constant, but depended on the time.
AH_tactics: 94%
Not perfect, but much better than pure LC0.
positional play (tests gcp+typp): 39/63
LC0 achieves the same result.
Re: Trying out the Leela Hybrid engine (NN with AB)
Posted: Fri Mar 15, 2019 12:11 am
by KillerDucky
kalikoba2019 wrote: ↑Thu Mar 14, 2019 7:03 pm
Nice one, and , if the auxiliary engine accepts to read and play .bin books, would it be possible to let Leelafish play its auxiliary engine book if it exists ?
I haven't used .bin books. I guess some engines (but not Stockfish?) have UCI options that allow it to use .bin? You can pass any arbitrary UCI options using AuxEngineOptions:
Code: Select all
AuxEngineOptions Hash1024;Threads=1;PolyglotOptionName=c:\mybook.bin
Re: Trying out the Leela Hybrid engine (NN with AB)
Posted: Fri Mar 15, 2019 8:20 am
by Ozymandias
KillerDucky wrote: ↑Fri Mar 15, 2019 12:11 am
kalikoba2019 wrote: ↑Thu Mar 14, 2019 7:03 pm
Nice one, and , if the auxiliary engine accepts to read and play .bin books, would it be possible to let Leelafish play its auxiliary engine book if it exists ?
I haven't used .bin books. I guess some engines (but not Stockfish?) have UCI options that allow it to use .bin? You can pass any arbitrary UCI options using AuxEngineOptions:
Code: Select all
AuxEngineOptions Hash1024;Threads=1;PolyglotOptionName=c:\mybook.bin
Leela uses the AuxEngine's PV to boost the Policy of those moves. Otherwise Leela's search runs the same as before, including all position evaluations.
If you pass that uci option to the auxiliary engine, won't the PV be just one move? The best move in the position according to the book? In that case, it sounds like Leelafish would just play unrestricted, just as regular Leela would. The only difference would be that it would take into account the book's best move, but the decision would rest ultimately in its own judgement. Quite far away from what a book is supposed to do.