Trying out the Leela Hybrid engine (NN with AB)

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
pohl4711
Posts: 1042
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by pohl4711 » Sun Feb 24, 2019 6:26 am

KillerDucky wrote:
Sun Feb 24, 2019 12:23 am
v0.21.0-fish-v0.4 released! Just one feature addition and one minor bug fix.

* Add support for AuxEngine to use Syzygy endgame tablebases. Automagic if you configure Lc0 to use Syzygy and AuxEngine supports them.
* Fix a bug with boosting Policy to larger than 100%. Also set maximum AuxEngineBoost to 100%. Internally Lc0 uses a compressed format for Policy so values larger than 100% do not work.
v0.4 Crashes after 3-4 seconds of thinking in Fritz (with Stockfish, Houdini or Komodo as AuxEngine).

Steppenwolf
Posts: 37
Joined: Thu Jan 31, 2019 3:54 pm
Full name: Sven Steppenwolf

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Steppenwolf » Sun Feb 24, 2019 8:57 am

I can confirm this (for the blas flavour) under Fritz 16 with SF as additional engine, it also crashes.

User avatar
Eduard
Posts: 199
Joined: Fri Oct 26, 2018 10:58 pm
Location: Germany
Full name: Eduard Nemeth
Contact:

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Eduard » Sun Feb 24, 2019 9:20 am

On my Laptop i3 2x2,4 GHz CPU-Version under Fritz 15 with Shashchess works.

User avatar
sincerelybrent
Posts: 47
Joined: Fri Feb 15, 2019 10:41 pm
Full name: Brent M

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by sincerelybrent » Sun Feb 24, 2019 1:47 pm

Mine works fine after updating my drivers from NVIDIA. I also downloaded their latest CUDA.

LeelaFish 0.4
GeForce RTX 2070
cudnn-fp16
Fritz 16
Stockfish 10 x64 popcnt

yanquis1972
Posts: 1762
Joined: Tue Jun 02, 2009 10:14 pm

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by yanquis1972 » Sun Feb 24, 2019 7:13 pm

Laskos wrote:
Fri Feb 22, 2019 5:53 pm
I think it's important to keep in mind that we assume Auxiliary engine must just help Lc0 tactically, and ideally it's much weaker than both Lc0 and its opponent. Many might be be tempted to use Auxiliary engine the level of SF, while their Lc0 is weaker than SF in some conditions. Even on my powerful GPU set-up this might occur. So, I took as Auxiliary engine Arasan 19.01 on 1 thread, an AB engine some 800-900 Elo points weaker than both Lc0 t30 and SF on 4 cores. It still helps a lot, solving many more positions on trimmed WAC tactical test suite and not harming almost at all the positional abilities of Lc0 (on my positional test-suite). But I changed some settings to get such a result with a weak Auxiliary engine:

AuxEngineBoost=20.00
AuxEngineDepth=10
AuxEngineThreshold=10
AuxEngineFollowPvDepth=3

In games, it increases Lc0 t30 strength against SF10 (4 cores) by some 35-40 Elo points, and the weak Arasan 19.01 on 1 thread helps almost as much as SF10 as an auxiliary engine (SF10 adds some 40-50 Elo points). The games were at bullet TC.

With SF10 as an auxiliary engine, many settings will work, as it is close in strength or stronger than Lc0, but with a weak auxiliary AB engine, finding correct settings is trickier. It has to know when it can intervene, and this is usually the case of gross tactical mistakes of Lc0. Still, endgames are bad.
haven't tried w/ 32930 but i get remarkable results w/ these settings & T40:

Engine Score Lc
1: Lc0 T40 37.0/100 ··················································
2: Leelafish SFDev+T40KL 2 threads 32.0/50 =111=1=1==========1=========1111=1=====1===1=====1
3: Leelafish SFDev+T40KL 31.0/50 1===11=1==11=1=============1=111=1==============01

100 games played / Tournament finished
Name of the tournament: T40 vs LFKai 1 v 2 threads
Site/ Country: PC, United States
Level: Blitz 1/1
Hardware: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz with 15.9 GB Memory

using sfdev on 1/2 cores. imo the author's done a remarkable job in such short time; it plays like leela without the idiosyncracies. i suspect matefinder or perhaps sting would produce similar results. yet to try those, or a match vs SFDEV on multiple cores tho.

currently id41109+sfdev is +1 =3 -0 on t30 in a 4+4 match.

User avatar
Laskos
Posts: 9324
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Laskos » Mon Feb 25, 2019 12:38 am

yanquis1972 wrote:
Sun Feb 24, 2019 7:13 pm
Laskos wrote:
Fri Feb 22, 2019 5:53 pm
I think it's important to keep in mind that we assume Auxiliary engine must just help Lc0 tactically, and ideally it's much weaker than both Lc0 and its opponent. Many might be be tempted to use Auxiliary engine the level of SF, while their Lc0 is weaker than SF in some conditions. Even on my powerful GPU set-up this might occur. So, I took as Auxiliary engine Arasan 19.01 on 1 thread, an AB engine some 800-900 Elo points weaker than both Lc0 t30 and SF on 4 cores. It still helps a lot, solving many more positions on trimmed WAC tactical test suite and not harming almost at all the positional abilities of Lc0 (on my positional test-suite). But I changed some settings to get such a result with a weak Auxiliary engine:

AuxEngineBoost=20.00
AuxEngineDepth=10
AuxEngineThreshold=10
AuxEngineFollowPvDepth=3

In games, it increases Lc0 t30 strength against SF10 (4 cores) by some 35-40 Elo points, and the weak Arasan 19.01 on 1 thread helps almost as much as SF10 as an auxiliary engine (SF10 adds some 40-50 Elo points). The games were at bullet TC.

With SF10 as an auxiliary engine, many settings will work, as it is close in strength or stronger than Lc0, but with a weak auxiliary AB engine, finding correct settings is trickier. It has to know when it can intervene, and this is usually the case of gross tactical mistakes of Lc0. Still, endgames are bad.
haven't tried w/ 32930 but i get remarkable results w/ these settings & T40:

Engine Score Lc
1: Lc0 T40 37.0/100 ··················································
2: Leelafish SFDev+T40KL 2 threads 32.0/50 =111=1=1==========1=========1111=1=====1===1=====1
3: Leelafish SFDev+T40KL 31.0/50 1===11=1==11=1=============1=111=1==============01

100 games played / Tournament finished
Name of the tournament: T40 vs LFKai 1 v 2 threads
Site/ Country: PC, United States
Level: Blitz 1/1
Hardware: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz with 15.9 GB Memory

using sfdev on 1/2 cores. imo the author's done a remarkable job in such short time; it plays like leela without the idiosyncracies. i suspect matefinder or perhaps sting would produce similar results. yet to try those, or a match vs SFDEV on multiple cores tho.

currently id41109+sfdev is +1 =3 -0 on t30 in a 4+4 match.
Wow, good results indeed with T40. I am also getting apparently good results with T30 at 30'' + 0.5'' using Aux Engine Houdini 6.03 Tactical on 2 threads (similar settings of LcFish to those I posted) in a long round-robin, simulating something like CCCC or TCEC in some late rounds:

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   1 lc0_Fish_32930                200      40     200   76.0%   39.0%
   2 lc0_32930                     137      35     200   68.8%   47.5%
   3 SF_dev                        131      39     200   68.0%   39.0%
   4 Kom_123                       -45      39     200   43.5%   36.0%
   5 Ethereal_1125                -158      42     200   28.7%   31.5%
   6 Andscacs_095                 -301      49     200   15.0%   25.0%
Finished match
I haven't looked at actual games, but observe that the draw rate of LcFish is in line with regular engines, unlike regular Lc0 with much higher draw rate, which often fails to convert wins and ends them in draws. All the regular engines were on 4 i7 4790 cores.

I am curious too about T40 nets, will check them someday. They seem to scale better than T30, so maybe a bit longer TC than I used with T30 is necessary.

User avatar
sincerelybrent
Posts: 47
Joined: Fri Feb 15, 2019 10:41 pm
Full name: Brent M

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by sincerelybrent » Mon Feb 25, 2019 3:39 am


Wow, good results using ... Aux Engine Houdini 6.03 Tactical on 2 threads ...
How are you able to force Houdini tactical mode? I mean, you have to feed LeelaFish commands in the command line. Right? Or does setting Houdini to tactical mode, via the gui, work too? Thanks for your help

User avatar
M ANSARI
Posts: 3399
Joined: Thu Mar 16, 2006 6:10 pm

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by M ANSARI » Mon Feb 25, 2019 7:17 am

Decided to give this engine a try ... but it seems to crash on the Fritz GUI when I start the engine in analysis? I tried the default setting first but as soon as the engine starts I get a DOS prompt and the engine stalls ... sometimes it gives an exception fault. I think there should be a read me file where it explains a lot of the stuff there as most of it is hard to decipher what it changes. An accurate description of each parameter would allow people to experiment with different parameters and maybe stumble on something better than default.

With regards to the engine path ... should it be between " " ?

I attach the Logfile
Attachments
Leelafish_.4_Log.zip
(4.19 KiB) Downloaded 34 times

Hugo
Posts: 775
Joined: Tue Dec 01, 2009 10:10 am

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by Hugo » Mon Feb 25, 2019 10:34 am

hi there

I need to get it working by using the lc0.config file
--aux-engine-path=C:\Users\Skylake\Documents\InfinityChess\Data\Engines\BrainFish_190222/BrainFish_190222_x64_modern.exe
--aux-engine-options=Hash=1024;Threads=2
this is my configile, path of engine is correct, but lc0 gives error message
"unknown command line flag
--aux-engine-path=C:\Users\Skylake\Documents\InfinityChess\Data\Engines\BrainFish_190222/BrainFish_190222_x64_modern.exe"

in Fritz gui it works witzh this parameters:
[OPTIONS]
Threads=8
Slowmover=0.800000
Backend=1
AuxEnginePath=C:\Users\Skylake\Documents\InfinityChess\Data\Engines\BrainFish_190222/BrainFish_190222_x64_modern.exe
AuxEngineOptions=Hash=1024;Threads=2
any Idea whats wron in my lc0.config file?

regards, C.K.

User avatar
velmarin
Posts: 1598
Joined: Mon Feb 21, 2011 8:48 am

Re: Trying out the Leela Hybrid engine (NN with AB)

Post by velmarin » Mon Feb 25, 2019 10:42 am

AuxEnginePath=C:\Users\Skylake\Documents\InfinityChess\Data\Engines\BrainFish_190222/ :evil: (ERROR)BrainFish_190222_x64_modern.exe
Correct: AuxEnginePath=C:\Users\Skylake\Documents\InfinityChess\Data\Engines\BrainFish_190222\BrainFish_190222_x64_modern.exe

Better to put the auxiliary engine in the leelafish directory, directly.
  and in the configuration box only the name of the engine and it works perfect.

Post Reply