Trying out the Leela Hybrid engine (NN with AB)

[sincerelybrent]

Hi KillerDucky. Your project is very interesting. But...

The biggest weakness of LC0 is endgame and tactics. Leela handles other elements well, and in the positional game she outperforms the classic engines.

I have tested eigenmann (100 positions) and AH_tactics (250 positions). Stockfish 10 plays them almost perfectly (97/100 and 246/250).

The results of LC0 (network 32988) are 68% in both cases. ...

I have chosen those positions that LC0 can't solve (and Stockfish does). The eigenmann subset consists of 32 items. The AH_tectics subset is 79 positions. In this subset I tested Leelafish with different settings. The best results I received when:

auxengine-depth=20/25
auxengine-boost=100.00
...

What other depth and boost combinations have you tried? And where can I download the AH tactic tests? Thanks,

[Paloma]

@ sincerelybrent

AH_Endgames-250 and AH_Tactics-250 Results and epd's

http://www.talkchess.com/forum3/viewtop ... =2&t=41023

use the links in the first post

[Jaglo]

What other depth and boost combinations have you tried? And where can I download the AH tactic tests? Thanks,

auxengine-threshold=10/100/1000
auxengine-depth=15/20/25
auxengine-boost=50/100
auxengine-follow-pv-depth=4 (1? 10? - I do not remember)

I didn't check all combinations, only after the change I did a test and if it gave an improvement I moved it with other parameters, trying to achieve further improvement. A deeper depth is needed for Stockfish to find a solution at all (approx. 20-30). Setting the boost to 100 was also important. I haven't noticed the influence of other parameters.

The Eigemann website no longer exists. I once downloaded an epd and converted it to chessbase, and now I don't know how to convert it back to epd. AH_Endgames_250 should be similar.

[sincerelybrent]

@ sincerelybrent

AH_Endgames-250 and AH_Tactics-250 Results and epd's

http://www.talkchess.com/forum3/viewtop ... =2&t=41023

use the links in the first post

Thank you very much

[KillerDucky]

v0.21.0-fish-v0.8 released!

https://github.com/killerducky/lc0/wiki/LeelaFish
https://github.com/killerducky/lc0/releases

Important note: Stockfish 10 official release has a bug that Leelafish hits more often than normal. To workaround this: 1) Use a recent dev version (after 2018-12-16) or 2) Set Threads=1 in AuxEngineOptions.

Change notes:

* Merge Lc0 bugfix, required to run T40 nets:
* Improved folding of batch norm into weights and biases - fixes negative gamma bug. (LeelaChessZero#779)
* Now Leelafish is based off official release v0.21.0 of Lc0 (previously v0.21.0-dev).

[MikeB]

v0.21.0-fish-v0.8 released!

https://github.com/killerducky/lc0/wiki/LeelaFish
https://github.com/killerducky/lc0/releases

Important note: Stockfish 10 official release has a bug that Leelafish hits more often than normal. To workaround this: 1) Use a recent dev version (after 2018-12-16) or 2) Set Threads=1 in AuxEngineOptions.

Change notes:

* Merge Lc0 bugfix, required to run T40 nets:
* Improved folding of batch norm into weights and biases - fixes negative gamma bug. (LeelaChessZero#779)
* Now Leelafish is based off official release v0.21.0 of Lc0 (previously v0.21.0-dev).

+1 good work - appreciate it!

[kalikoba2019]

Nice one, and , if the auxiliary engine accepts to read and play .bin books, would it be possible to let Leelafish play its auxiliary engine book if it exists ?

[Jaglo]

Leelafish v0.8
net: 32988
auxengine=Stockfish 10 (Hash=1024;Threads=1)
auxengine-depth=25
auxengine-boost=100.00
Syzygy 345

eigenmann: 84%
Perhaps the result for the endings would have been better if the depth of analysis had not been constant, but depended on the time.

AH_tactics: 94%
Not perfect, but much better than pure LC0.

positional play (tests gcp+typp): 39/63
LC0 achieves the same result.

[KillerDucky]

Nice one, and , if the auxiliary engine accepts to read and play .bin books, would it be possible to let Leelafish play its auxiliary engine book if it exists ?

I haven't used .bin books. I guess some engines (but not Stockfish?) have UCI options that allow it to use .bin? You can pass any arbitrary UCI options using AuxEngineOptions:

Code: Select all

AuxEngineOptions Hash1024;Threads=1;PolyglotOptionName=c:\mybook.bin

[Ozymandias]

Nice one, and , if the auxiliary engine accepts to read and play .bin books, would it be possible to let Leelafish play its auxiliary engine book if it exists ?

I haven't used .bin books. I guess some engines (but not Stockfish?) have UCI options that allow it to use .bin? You can pass any arbitrary UCI options using AuxEngineOptions:

Code: Select all

AuxEngineOptions Hash1024;Threads=1;PolyglotOptionName=c:\mybook.bin

Leela uses the AuxEngine's PV to boost the Policy of those moves. Otherwise Leela's search runs the same as before, including all position evaluations.

If you pass that uci option to the auxiliary engine, won't the PV be just one move? The best move in the position according to the book? In that case, it sounds like Leelafish would just play unrestricted, just as regular Leela would. The only difference would be that it would take into account the book's best move, but the decision would rest ultimately in its own judgement. Quite far away from what a book is supposed to do.