Trying out the Leela Hybrid engine (NN with AB)

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
sincerelybrent
Posts: 47
Joined: Fri Feb 15, 2019 10:41 pm
Full name: Brent M

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by sincerelybrent » Thu Mar 07, 2019 2:24 pm

Jaglo wrote:
Thu Mar 07, 2019 9:26 am
Hi KillerDucky. Your project is very interesting. But...

The biggest weakness of LC0 is endgame and tactics. Leela handles other elements well, and in the positional game she outperforms the classic engines.

I have tested eigenmann (100 positions) and AH_tactics (250 positions). Stockfish 10 plays them almost perfectly (97/100 and 246/250).

The results of LC0 (network 32988) are 68% in both cases. ...

I have chosen those positions that LC0 can't solve (and Stockfish does). The eigenmann subset consists of 32 items. The AH_tectics subset is 79 positions. In this subset I tested Leelafish with different settings. The best results I received when:

auxengine-depth=20/25
auxengine-boost=100.00
...
What other depth and boost combinations have you tried? And where can I download the AH tactic tests? Thanks,

Paloma
Posts: 746
Joined: Thu Dec 25, 2008 8:07 pm

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Paloma » Thu Mar 07, 2019 6:20 pm

@ sincerelybrent

AH_Endgames-250 and AH_Tactics-250 Results and epd's

http://www.talkchess.com/forum3/viewtop ... =2&t=41023

use the links in the first post

Jaglo
Posts: 12
Joined: Sat Mar 02, 2019 6:45 pm
Full name: Jacek Glowinski

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Jaglo » Thu Mar 07, 2019 7:28 pm

What other depth and boost combinations have you tried? And where can I download the AH tactic tests? Thanks,
auxengine-threshold=10/100/1000
auxengine-depth=15/20/25
auxengine-boost=50/100
auxengine-follow-pv-depth=4 (1? 10? - I do not remember)

I didn't check all combinations, only after the change I did a test and if it gave an improvement I moved it with other parameters, trying to achieve further improvement. A deeper depth is needed for Stockfish to find a solution at all (approx. 20-30). Setting the boost to 100 was also important. I haven't noticed the influence of other parameters.

The Eigemann website no longer exists. I once downloaded an epd and converted it to chessbase, and now I don't know how to convert it back to epd. AH_Endgames_250 should be similar.

User avatar
sincerelybrent
Posts: 47
Joined: Fri Feb 15, 2019 10:41 pm
Full name: Brent M

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by sincerelybrent » Thu Mar 07, 2019 11:58 pm

Paloma wrote:
Thu Mar 07, 2019 6:20 pm
@ sincerelybrent

AH_Endgames-250 and AH_Tactics-250 Results and epd's

http://www.talkchess.com/forum3/viewtop ... =2&t=41023

use the links in the first post
Thank you very much

KillerDucky
Posts: 12
Joined: Fri Mar 23, 2018 1:31 am

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by KillerDucky » Thu Mar 14, 2019 12:25 am

v0.21.0-fish-v0.8 released!

https://github.com/killerducky/lc0/wiki/LeelaFish
https://github.com/killerducky/lc0/releases

Important note: Stockfish 10 official release has a bug that Leelafish hits more often than normal. To workaround this: 1) Use a recent dev version (after 2018-12-16) or 2) Set Threads=1 in AuxEngineOptions.

Change notes:

* Merge Lc0 bugfix, required to run T40 nets:
* Improved folding of batch norm into weights and biases - fixes negative gamma bug. (LeelaChessZero#779)
* Now Leelafish is based off official release v0.21.0 of Lc0 (previously v0.21.0-dev).

MikeB
Posts: 3024
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by MikeB » Thu Mar 14, 2019 2:49 am

KillerDucky wrote:
Thu Mar 14, 2019 12:25 am
v0.21.0-fish-v0.8 released!

https://github.com/killerducky/lc0/wiki/LeelaFish
https://github.com/killerducky/lc0/releases

Important note: Stockfish 10 official release has a bug that Leelafish hits more often than normal. To workaround this: 1) Use a recent dev version (after 2018-12-16) or 2) Set Threads=1 in AuxEngineOptions.

Change notes:

* Merge Lc0 bugfix, required to run T40 nets:
* Improved folding of batch norm into weights and biases - fixes negative gamma bug. (LeelaChessZero#779)
* Now Leelafish is based off official release v0.21.0 of Lc0 (previously v0.21.0-dev).
+1 good work - appreciate it!

kalikoba2019
Posts: 8
Joined: Sun Mar 03, 2019 11:46 am
Full name: Kalikoba

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by kalikoba2019 » Thu Mar 14, 2019 6:03 pm

Nice one, and , if the auxiliary engine accepts to read and play .bin books, would it be possible to let Leelafish play its auxiliary engine book if it exists ?

Jaglo
Posts: 12
Joined: Sat Mar 02, 2019 6:45 pm
Full name: Jacek Glowinski

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Jaglo » Thu Mar 14, 2019 8:44 pm

Leelafish v0.8
net: 32988
auxengine=Stockfish 10 (Hash=1024;Threads=1)
auxengine-depth=25
auxengine-boost=100.00
Syzygy 345

eigenmann: 84%
Perhaps the result for the endings would have been better if the depth of analysis had not been constant, but depended on the time.

AH_tactics: 94%
Not perfect, but much better than pure LC0.

positional play (tests gcp+typp): 39/63
LC0 achieves the same result.

KillerDucky
Posts: 12
Joined: Fri Mar 23, 2018 1:31 am

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by KillerDucky » Thu Mar 14, 2019 11:11 pm

kalikoba2019 wrote:
Thu Mar 14, 2019 6:03 pm
Nice one, and , if the auxiliary engine accepts to read and play .bin books, would it be possible to let Leelafish play its auxiliary engine book if it exists ?
I haven't used .bin books. I guess some engines (but not Stockfish?) have UCI options that allow it to use .bin? You can pass any arbitrary UCI options using AuxEngineOptions:

Code: Select all

AuxEngineOptions Hash1024;Threads=1;PolyglotOptionName=c:\mybook.bin

User avatar
Ozymandias
Posts: 1067
Joined: Sun Oct 25, 2009 12:30 am

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Ozymandias » Fri Mar 15, 2019 7:20 am

KillerDucky wrote:
Thu Mar 14, 2019 11:11 pm
kalikoba2019 wrote:
Thu Mar 14, 2019 6:03 pm
Nice one, and , if the auxiliary engine accepts to read and play .bin books, would it be possible to let Leelafish play its auxiliary engine book if it exists ?
I haven't used .bin books. I guess some engines (but not Stockfish?) have UCI options that allow it to use .bin? You can pass any arbitrary UCI options using AuxEngineOptions:

Code: Select all

AuxEngineOptions Hash1024;Threads=1;PolyglotOptionName=c:\mybook.bin
Leela uses the AuxEngine's PV to boost the Policy of those moves. Otherwise Leela's search runs the same as before, including all position evaluations.
If you pass that uci option to the auxiliary engine, won't the PV be just one move? The best move in the position according to the book? In that case, it sounds like Leelafish would just play unrestricted, just as regular Leela would. The only difference would be that it would take into account the book's best move, but the decision would rest ultimately in its own judgement. Quite far away from what a book is supposed to do.

Post Reply