Trying out the Leela Hybrid engine (NN with AB)

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Eduard
Posts: 199
Joined: Fri Oct 26, 2018 10:58 pm
Location: Germany
Full name: Eduard Nemeth
Contact:

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Eduard » Thu Feb 21, 2019 3:03 am

Great thanks! I only test with CPU. Have used McCain X1 as AuxEngine. No more tactical mistakes to see from Leela. :P

lovetb
Posts: 16
Joined: Thu Dec 13, 2018 5:57 am
Full name: MR PRASHANTH KRISHNACHARI

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by lovetb » Thu Feb 21, 2019 11:01 am

Thank you very much. I will test it.

Have you thought about using Auxiliary engine 100% for the end game or after a certain number of moves (configurable) ?

pohl4711
Posts: 1042
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by pohl4711 » Thu Feb 21, 2019 11:16 am

KillerDucky wrote:
Tue Feb 19, 2019 8:59 pm

Currently it runs SF using "go depth N" where N is a parameter. Would it be better to maybe use "go movetime N"? Where SF could decide to spend more or less time depending on the position?
Go movetime let SF NOT decide to spend more or less time depending on the position... go movetime means, SF moves, when the time specified with that parameter is over...

That would be definitly better a better solution, because a fix movetime will lead to much higher search-depths in the endgame, than a fix depth-parameter!


(And please fix the "caused an exception"-Crash when ponder is on in an engine-match or tournament (Fritz16)(CUDA-version)).


Regards - Stefan (SPCC)

User avatar
Ozymandias
Posts: 1097
Joined: Sun Oct 25, 2009 12:30 am

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Ozymandias » Thu Feb 21, 2019 1:04 pm

pohl4711 wrote:
Thu Feb 21, 2019 11:16 am
Go movetime let SF NOT decide to spend more or less time depending on the position... go movetime means, SF moves, when the time specified with that parameter is over...

That would be definitly better a better solution, because a fix movetime will lead to much higher search-depths in the endgame, than a fix depth-parameter!
SF doesn't decide how much time to spend for a given position, neither with movetime nor with fixed depth. Time management doesn't become a part of the equation in either case.

As for the second part, your statement's rather obvious, what's not so obvious is that it would be a good thing to invest more time in the endgame. Laskos did a test some time ago, about how engines spend time on different phases of the game, I wish I could find it, but IIRC it showed that the phase that gets less of a return for time spent (ELO wise), is precisely the endgame.

KillerDucky
Posts: 12
Joined: Fri Mar 23, 2018 1:31 am

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by KillerDucky » Thu Feb 21, 2019 4:29 pm

pohl4711, please set LogFile to leelafish.log and send me it on my github issues. I can't debug crashes without logfiles.

The most common problem wrong AuxEngineOptions, please make sure you have the right format (no quotes, I saw several people doing that). If it's wrong you should see a line with "auxengine.cc:" plus an error message from the AuxEngine.

It seems there is no UCI standard for reporting errors back to the user? Maybe I could output a uci info string with an error message? Anyone have suggestions for error reporting?

pohl4711
Posts: 1042
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by pohl4711 » Thu Feb 21, 2019 6:10 pm

KillerDucky wrote:
Thu Feb 21, 2019 4:29 pm
pohl4711, please set LogFile to leelafish.log and send me it on my github issues. I can't debug crashes without logfiles.

The most common problem wrong AuxEngineOptions, please make sure you have the right format (no quotes, I saw several people doing that). If it's wrong you should see a line with "auxengine.cc:" plus an error message from the AuxEngine.

It seems there is no UCI standard for reporting errors back to the user? Maybe I could output a uci info string with an error message? Anyone have suggestions for error reporting?
At the moment, I have no free PC, sorry. I had Komodo 12.3 as AuxEngine. It worked without any problems with ponder off in an engine-match in Fritz16, so the AuxengineOptions were definitly correct (Threads=1, Hash=1024). When ponder is set to on (in Fritz 16), Leelafish crashes, when it is pondering and the opponent engine plays its move and Leelafish should switch from pondering to thinking - caused an exception.
Should be easy to reproduce in Fritz...

User avatar
Eduard
Posts: 199
Joined: Fri Oct 26, 2018 10:58 pm
Location: Germany
Full name: Eduard Nemeth
Contact:

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Eduard » Thu Feb 21, 2019 8:34 pm

If I take Komodo 9.02 as AuxEngine then Leelafish (CPU version) crashes. What could be the reason? There are the same settings as with stockfish. :x

yanquis1972
Posts: 1762
Joined: Tue Jun 02, 2009 10:14 pm

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by yanquis1972 » Thu Feb 21, 2019 10:26 pm

even w/ GTX 1080 i'm seeing +50 elo at 1+1, with a sample size caveat. T40 outperformed 32930 over 50 games (+35 elo), -70 elo to SFdev (4 cores @ 4.4ghz). same T40 network was -135 to SFDEV & -30 to 32930 (only 25 games each).

using 3 threads for SFdev as aux engine, 1024MB hash, everything else default.


looking at the output its like an invisible hand, i don't really understand how the 2 interface, but it definitely works.

jjoshua2
Posts: 20
Joined: Sat Mar 10, 2018 5:16 am

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by jjoshua2 » Fri Feb 22, 2019 12:12 am

yanquis1972 wrote:
Thu Feb 21, 2019 10:26 pm
using 3 threads for SFdev as aux engine, 1024MB hash, everything else default.
Insomuch as the defaults are balanced for 1 thread, using more threads will reach the same depth faster, and thus have empty queues too often. So with more threads it should be necessary either to increase the default depth or decrease the number of nodes that launches a new search to the queue. You can see queue size in the logging if you put a log file name in that uci option, but it's quite difficult to optimize this way.

MikeB
Posts: 3288
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by MikeB » Fri Feb 22, 2019 3:52 am

jjoshua2 wrote:
Fri Feb 22, 2019 12:12 am
yanquis1972 wrote:
Thu Feb 21, 2019 10:26 pm
using 3 threads for SFdev as aux engine, 1024MB hash, everything else default.
Insomuch as the defaults are balanced for 1 thread, using more threads will reach the same depth faster, and thus have empty queues too often. So with more threads it should be necessary either to increase the default depth or decrease the number of nodes that launches a new search to the queue. You can see queue size in the logging if you put a log file name in that uci option, but it's quite difficult to optimize this way.
Yes, you need to raise SD or ST when you increase threads. It is difficult to optimize by hand as far a I can tell, but on the other hand this is a magnificent first step!

Post Reply