Trying out the Leela Hybrid engine (NN with AB)

[KillerDucky]

@pohl4711: Thanks for the logfile, I was able to spot the problem with it! Sorry I didn't test this myself. It happens with v0.4 if you do not use Syzygy. Fixed in v0.5!

https://github.com/killerducky/lc0/rele ... -fish-v0.5

* Fix crash in v0.4 when Syzygy is not used.
* Stop the AuxEngine when lc0 is done searching.
* This prevents us from cheating by thinking into opponent's time for the last AuxEngine search.
* The cheating was only minor, on my machine ~20 AuxEngine evals are done per move in ~70ms each. So AuxEngine would think an extra 35ms per move on average.
* Fix off by one error in AuxEngineFollowPvDepth.
* Setting this to 1 is supposed to only update the first move, but it would update 2 moves.
* Also update default from 3 to 4 so the behavior is the same as previous releases.
* Add some more logging stats

Code: Select all

auxengine_queue_ size 252 Average duration 72ms Number of evals 23 Number of updates 64

[sincerelybrent]

...It happens with v0.4 if you do not use Syzygy. Fixed in v0.5!

That's why it worked for me. Nice catch, KillerDucky, and thanks for the update.

[MikeB]

@pohl4711: Thanks for the logfile, I was able to spot the problem with it! Sorry I didn't test this myself. It happens with v0.4 if you do not use Syzygy. Fixed in v0.5!

https://github.com/killerducky/lc0/rele ... -fish-v0.5

* Fix crash in v0.4 when Syzygy is not used.
* Stop the AuxEngine when lc0 is done searching.
* This prevents us from cheating by thinking into opponent's time for the last AuxEngine search.
* The cheating was only minor, on my machine ~20 AuxEngine evals are done per move in ~70ms each. So AuxEngine would think an extra 35ms per move on average.
* Fix off by one error in AuxEngineFollowPvDepth.
* Setting this to 1 is supposed to only update the first move, but it would update 2 moves.
* Also update default from 3 to 4 so the behavior is the same as previous releases.
* Add some more logging stats

Code: Select all

auxengine_queue_ size 252 Average duration 72ms Number of evals 23 Number of updates 64

...just a very big +1 for doing this KillerDucky - it's a real contribution to the computer chess community! Can you tell us more on how you use it , what settings you use and why? Still trying to get my head wrapped around this - I want to use it mostly for analysis - so any suggestions or comments you might have would be useful. Thanks in advance.

[pohl4711]

@pohl4711: Thanks for the logfile, I was able to spot the problem with it! Sorry I didn't test this myself. It happens with v0.4 if you do not use Syzygy. Fixed in v0.5!

https://github.com/killerducky/lc0/rele ... -fish-v0.5

* Fix crash in v0.4 when Syzygy is not used.
* Stop the AuxEngine when lc0 is done searching.
* This prevents us from cheating by thinking into opponent's time for the last AuxEngine search.
* The cheating was only minor, on my machine ~20 AuxEngine evals are done per move in ~70ms each. So AuxEngine would think an extra 35ms per move on average.
* Fix off by one error in AuxEngineFollowPvDepth.
* Setting this to 1 is supposed to only update the first move, but it would update 2 moves.
* Also update default from 3 to 4 so the behavior is the same as previous releases.
* Add some more logging stats

Code: Select all

auxengine_queue_ size 252 Average duration 72ms Number of evals 23 Number of updates 64

I still strongly recommend to use go movestime for the aux-engine, not a fixed depth, this should lead to higher depths in endgame and more Elo-gain at this weak point of Leela !

[Jesse Gersenson]

If I have a machine with 72 threads, how many would I assign to the auxengine?

[pohl4711]

An early intermediate result of LeelaFish in my long thinking-time tournament (86 games only) looks very promising...
Stockfish 10 vs. LeelaFish v0.3 N:32930 (the TCEC Superfinal 14 Net) and AuxEngine: Houdini 6 (Depth = 16, Threshold = 100)

https://www.sp-cc.de/long-thinkingtime.htm

[sincerelybrent]

0.5 is looking good, KillerDucky, but game 12 threw an exception for me. Here is the log:
http://www.westportchessclub.org/public ... ception.7z

Otherwise, with a LeelaRatio of 1.0, LeelaFish 0.5 w/ Houdini 6 (1 thread, 1024 hash, tactical mode=1) is beating Stockfish 10 64-bit POPCNT by 4 games in 50 total.

[Javier Ros]

Many thanks to KillerDucky for this great idea!
It works perfectly for me.

I wonder if it is possible to specify Houdini Tactical Mode adding in Arena:

Hash=1024;Threads=1;Tactical Mode=1

in AuxEngineOptions

[dkappe]

Trying out 0.5. I’ve noticed that with lots of aux engines (cheng4, sting 14 to name a few) it gives me a segmentation fault. Winter is an engine that works perfectly. I suspect that the info string parsing is a bit brittle.

[KillerDucky]

I have a fix for one crash. Here is a test version:
https://ci.appveyor.com/project/killerd ... s/22675046
(Click on cuda/opencl/blas, then click on Aritifacts, then download lc0.exe).

@sincerelybrent I don't think this will fix your case, it looks different. I added a little more logging that might help narrow it down.

@pohl4711 yes I think using movetime instead of depth is better. I will add that after fixing some of these crashes.