Stockfish NN release (NNUE)

[Raphexon]

The later one has the shallow depth pruning disabled so the fixed depth is stronger but in normal play it will cost a lot of elo - about 170 to 200.

That's outright nonsense!

This patch only disables the pruning for the very first 4 iterations of the search,
which avoids outright blunders and strange scores.
It doesn't cost any elo even at very fast games.

Code: Select all

And still competitive at ultrafast 1"+0.01" time control:

    Finished game 500 (SF-Master vs SF-LowPlies4a): 1/2-1/2 {Draw by 3-fold repetition}
    Score of SF-LowPlies4a vs SF-Master: 196 - 173 - 131 [0.523] 500
    Elo difference: 16.0 +/- 26.2, LOS: 88.4 %, DrawRatio: 26.2 %
    Finished match

Then I have misunderstood the patch. I am sorry. Why is it marked worth 170 Elo (and even 200 elo in SF dev) if it doesn't lose elo?

Because it doesn't completely nullify the old pruning at shallow depths.
It only disables it for the first 4 plys.

If it would disable pruning for the first 9 plys or so, then that would lose a lot of ELO.
But disabling for the 4 ply only introduces a fairly minor search slowdown.

[Master Om]

For analysing and playing I recommend this release: https://github.com/nodchip/Stockfish/re ... 2020-06-04
Use the blas compile for maximum nps.
This net was trained with the sped-up binary so it needs at least this release. The later one has the shallow depth pruning disabled so the fixed depth is stronger but in normal play it will cost a lot of elo - about 170 to 200. It might also be possible that it scales better on multiple cores. For me this net plays very good chess. It drew Leela already and even held against SF dev as white in a KGA. Let me know if this doesn't help.

Which net ?

[Chessqueen]

For analysing and playing I recommend this release: https://github.com/nodchip/Stockfish/re ... 2020-06-04
Use the blas compile for maximum nps.
This net was trained with the sped-up binary so it needs at least this release. The later one has the shallow depth pruning disabled so the fixed depth is stronger but in normal play it will cost a lot of elo - about 170 to 200. It might also be possible that it scales better on multiple cores. For me this net plays very good chess. It drew Leela already and even held against SF dev as white in a KGA. Let me know if this doesn't help.

Which net ?

Sorry to post this here, but Stockfish is in trouble vs GM Foreest

[kranium]

Code: Select all

   # PLAYER                           : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish.nnue.halfkp_256x2-3    : 2526.5   24.8     57.5     100   57.5%
   2 Stockfish_9_x64_bmi2             : 2473.5   24.8     42.5     100   42.5%
   
1) Stockfish.nnue.halfkp_256x2-3  2526 :    100 (+25,=65,-10),  57.5 %
   vs.                                 :  games (  +,  =,  -),   (%) :   Diff,  SD, CFS (%)
   Stockfish_9_x64_bmi2                :    100 ( 25, 65, 10),  57.5 :    +53,  23,   99.0

2) Stockfish_9_x64_bmi2           2474 :    100 (+10,=65,-25),  42.5 %
   vs.                                 :  games (  +,  =,  -),   (%) :   Diff,  SD, CFS (%)
   Stockfish.nnue.halfkp_256x2-3       :    100 ( 10, 65, 25),  42.5 :    -53,  23,    1.0

Code: Select all

   # PLAYER                               : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish_10                         : 2503.4   26.2     53.0     104   51.0%
   2 Stockfish.nnue.halfkp_256x2-32-32    : 2496.6   26.2     51.0     104   49.0%
   
1) Stockfish_10                       2503 :    104 (+20,=66,-18),  51.0 %
   vs.                                     :  games (  +,  =,  -),   (%) :   Diff,  SD, CFS (%)
   Stockfish.nnue.halfkp_256x2-32-32       :    104 ( 20, 66, 18),  51.0 :     +7,  24,   61.1

2) Stockfish.nnue.halfkp_256x2-32-32  2497 :    104 (+18,=66,-20),  49.0 %
   vs.                                     :  games (  +,  =,  -),   (%) :   Diff,  SD, CFS (%)
   Stockfish_10                            :    104 ( 18, 66, 20),  49.0 :     -7,  24,   38.9    

Thx ChickenLogic-

I had some time to run some games, it plays awesome chess and appears to be somewhere around SF 10 strength...only 30-40 elo short of SF11.
Those games are 1+1 with 8 threads and 2048 Mb Hash each, on an intel i9-990k.

Looks like it has enormous potential...let's hope the SF team puts some resources into trying it.
The extra code needed is quite small, the system simply uses existing SF with the newly created NN evaluation.

Here is the source, halfkp network, and binaries I used in above testing...
https://www.dropbox.com/s/a18teq9o71rc9 ... 04.7z?dl=0

just unzip and add stockfish.nnue.halfkp_256x2-32-32.exe (or any binary actually) as an engine in your GUI.

[cucumber]

Is anyone able to get this to work on Linux? I get a segfault calling into wait_for_search_finished from ThreadPool::set almost immediately after running ./stockfish. The wait_for_search_finished is different from what I see in SF 11, so I tried replacing it with SF 11's. No dice. I have no idea why it's segfaulting and I'm having trouble debugging it.

edit: Regular Stockfish, compiled with the same compiler within minutes of compiling SF NNUE, works just fine.

[Raphexon]

Is anyone able to get this to work on Linux? I get a segfault calling into wait_for_search_finished from ThreadPool::set almost immediately after running ./stockfish. The wait_for_search_finished is different from what I see in SF 11, so I tried replacing it with SF 11's. No dice. I have no idea why it's segfaulting and I'm having trouble debugging it.

edit: Regular Stockfish, compiled with the same compiler within minutes of compiling SF NNUE, works just fine.

Does this help?

https://github.com/nodchip/Stockfish/co ... t-39726766

[cucumber]

Is anyone able to get this to work on Linux? I get a segfault calling into wait_for_search_finished from ThreadPool::set almost immediately after running ./stockfish. The wait_for_search_finished is different from what I see in SF 11, so I tried replacing it with SF 11's. No dice. I have no idea why it's segfaulting and I'm having trouble debugging it.

edit: Regular Stockfish, compiled with the same compiler within minutes of compiling SF NNUE, works just fine.

Does this help?

https://github.com/nodchip/Stockfish/co ... t-39726766

Hm, thanks for the recommendation. I've just tried it, but the issue persists.

[Damir]

This is only supposed to be a test run to see how far I can come so I can share good parameters for training. I will do D=6 after I'm content with my results. But I fear depth greater than D=6 really requires multiple people. The idea is to train an initial net and then do fen generation with the trained Fish. So in the end it might not matter too much what depth we start with.

I see.

I just started the 1st training run with the bigger net (halfkp) ...
the net will have a size of about 20 MB! That's quite a difference compared to 400 kB for the smaller net.

Hi Jörg

How do you see if the net is beeing used ?

[cucumber]

I set up a new windows virtual machine just to try to build and tweak this. I still segfault on launch when compiling on Windows with gcc 8. How are people building this? It doesn't produce a functioning executable on either of the OSes that I've tried.

[Joerg Oster]

This is only supposed to be a test run to see how far I can come so I can share good parameters for training. I will do D=6 after I'm content with my results. But I fear depth greater than D=6 really requires multiple people. The idea is to train an initial net and then do fen generation with the trained Fish. So in the end it might not matter too much what depth we start with.

I see.

I just started the 1st training run with the bigger net (halfkp) ...
the net will have a size of about 20 MB! That's quite a difference compared to 400 kB for the smaller net.

Hi Jörg

How do you see if the net is beeing used ?

SF runs noticeably slower with the net loaded.