Trying out the Leela Hybrid engine (NN with AB)

[Eduard]

Great thanks! I only test with CPU. Have used McCain X1 as AuxEngine. No more tactical mistakes to see from Leela.

[lovetb]

Thank you very much. I will test it.

Have you thought about using Auxiliary engine 100% for the end game or after a certain number of moves (configurable) ?

[pohl4711]

Currently it runs SF using "go depth N" where N is a parameter. Would it be better to maybe use "go movetime N"? Where SF could decide to spend more or less time depending on the position?

Go movetime let SF NOT decide to spend more or less time depending on the position... go movetime means, SF moves, when the time specified with that parameter is over...

That would be definitly better a better solution, because a fix movetime will lead to much higher search-depths in the endgame, than a fix depth-parameter!


(And please fix the "caused an exception"-Crash when ponder is on in an engine-match or tournament (Fritz16)(CUDA-version)).


Regards - Stefan (SPCC)

[Ozymandias]

Go movetime let SF NOT decide to spend more or less time depending on the position... go movetime means, SF moves, when the time specified with that parameter is over...

That would be definitly better a better solution, because a fix movetime will lead to much higher search-depths in the endgame, than a fix depth-parameter!

SF doesn't decide how much time to spend for a given position, neither with movetime nor with fixed depth. Time management doesn't become a part of the equation in either case.

As for the second part, your statement's rather obvious, what's not so obvious is that it would be a good thing to invest more time in the endgame. Laskos did a test some time ago, about how engines spend time on different phases of the game, I wish I could find it, but IIRC it showed that the phase that gets less of a return for time spent (ELO wise), is precisely the endgame.

[KillerDucky]

pohl4711, please set LogFile to leelafish.log and send me it on my github issues. I can't debug crashes without logfiles.

The most common problem wrong AuxEngineOptions, please make sure you have the right format (no quotes, I saw several people doing that). If it's wrong you should see a line with "auxengine.cc:" plus an error message from the AuxEngine.

It seems there is no UCI standard for reporting errors back to the user? Maybe I could output a uci info string with an error message? Anyone have suggestions for error reporting?

[pohl4711]

pohl4711, please set LogFile to leelafish.log and send me it on my github issues. I can't debug crashes without logfiles.

The most common problem wrong AuxEngineOptions, please make sure you have the right format (no quotes, I saw several people doing that). If it's wrong you should see a line with "auxengine.cc:" plus an error message from the AuxEngine.

It seems there is no UCI standard for reporting errors back to the user? Maybe I could output a uci info string with an error message? Anyone have suggestions for error reporting?

At the moment, I have no free PC, sorry. I had Komodo 12.3 as AuxEngine. It worked without any problems with ponder off in an engine-match in Fritz16, so the AuxengineOptions were definitly correct (Threads=1, Hash=1024). When ponder is set to on (in Fritz 16), Leelafish crashes, when it is pondering and the opponent engine plays its move and Leelafish should switch from pondering to thinking - caused an exception.
Should be easy to reproduce in Fritz...

[Eduard]

If I take Komodo 9.02 as AuxEngine then Leelafish (CPU version) crashes. What could be the reason? There are the same settings as with stockfish.

[yanquis1972]

even w/ GTX 1080 i'm seeing +50 elo at 1+1, with a sample size caveat. T40 outperformed 32930 over 50 games (+35 elo), -70 elo to SFdev (4 cores @ 4.4ghz). same T40 network was -135 to SFDEV & -30 to 32930 (only 25 games each).

using 3 threads for SFdev as aux engine, 1024MB hash, everything else default.


looking at the output its like an invisible hand, i don't really understand how the 2 interface, but it definitely works.

[jjoshua2]

using 3 threads for SFdev as aux engine, 1024MB hash, everything else default.

Insomuch as the defaults are balanced for 1 thread, using more threads will reach the same depth faster, and thus have empty queues too often. So with more threads it should be necessary either to increase the default depth or decrease the number of nodes that launches a new search to the queue. You can see queue size in the logging if you put a log file name in that uci option, but it's quite difficult to optimize this way.

[MikeB]

using 3 threads for SFdev as aux engine, 1024MB hash, everything else default.

Insomuch as the defaults are balanced for 1 thread, using more threads will reach the same depth faster, and thus have empty queues too often. So with more threads it should be necessary either to increase the default depth or decrease the number of nodes that launches a new search to the queue. You can see queue size in the logging if you put a log file name in that uci option, but it's quite difficult to optimize this way.

Yes, you need to raise SD or ST when you increase threads. It is difficult to optimize by hand as far a I can tell, but on the other hand this is a magnificent first step!