Stockfish NN release (NNUE)

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Raphexon
Posts: 302
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: Stockfish NN release (NNUE)

Post by Raphexon » Sun Jun 07, 2020 1:25 pm

ChickenLogic wrote:
Sun Jun 07, 2020 1:19 pm
Joerg Oster wrote:
Sun Jun 07, 2020 1:00 pm
ChickenLogic wrote:
Sun Jun 07, 2020 10:18 am
The later one has the shallow depth pruning disabled so the fixed depth is stronger but in normal play it will cost a lot of elo - about 170 to 200.
That's outright nonsense!

This patch only disables the pruning for the very first 4 iterations of the search,
which avoids outright blunders and strange scores.
It doesn't cost any elo even at very fast games.

Code: Select all

And still competitive at ultrafast 1"+0.01" time control:

    Finished game 500 (SF-Master vs SF-LowPlies4a): 1/2-1/2 {Draw by 3-fold repetition}
    Score of SF-LowPlies4a vs SF-Master: 196 - 173 - 131 [0.523] 500
    Elo difference: 16.0 +/- 26.2, LOS: 88.4 %, DrawRatio: 26.2 %
    Finished match
Then I have misunderstood the patch. I am sorry. Why is it marked worth 170 Elo (and even 200 elo in SF dev) if it doesn't lose elo?
Because it doesn't completely nullify the old pruning at shallow depths.
It only disables it for the first 4 plys.

If it would disable pruning for the first 9 plys or so, then that would lose a lot of ELO.
But disabling for the 4 ply only introduces a fairly minor search slowdown.

User avatar
Master Om
Posts: 397
Joined: Wed Nov 24, 2010 9:57 am
Location: INDIA

Re: Stockfish NN release (NNUE)

Post by Master Om » Sun Jun 07, 2020 1:41 pm

ChickenLogic wrote:
Sun Jun 07, 2020 10:18 am
For analysing and playing I recommend this release: https://github.com/nodchip/Stockfish/re ... 2020-06-04
Use the blas compile for maximum nps.
This net was trained with the sped-up binary so it needs at least this release. The later one has the shallow depth pruning disabled so the fixed depth is stronger but in normal play it will cost a lot of elo - about 170 to 200. It might also be possible that it scales better on multiple cores. For me this net plays very good chess. It drew Leela already and even held against SF dev as white in a KGA. Let me know if this doesn't help.
Which net ?
Always Expect the Unexpected

Chessqueen
Posts: 1063
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Stockfish NN release (NNUE)

Post by Chessqueen » Sun Jun 07, 2020 2:59 pm

Master Om wrote:
Sun Jun 07, 2020 1:41 pm
ChickenLogic wrote:
Sun Jun 07, 2020 10:18 am
For analysing and playing I recommend this release: https://github.com/nodchip/Stockfish/re ... 2020-06-04
Use the blas compile for maximum nps.
This net was trained with the sped-up binary so it needs at least this release. The later one has the shallow depth pruning disabled so the fixed depth is stronger but in normal play it will cost a lot of elo - about 170 to 200. It might also be possible that it scales better on multiple cores. For me this net plays very good chess. It drew Leela already and even held against SF dev as white in a KGA. Let me know if this doesn't help.
Which net ?
Sorry to post this here, but Stockfish is in trouble vs GM Foreest https://www.twitch.tv/chesscomevents

User avatar
kranium
Posts: 1909
Joined: Thu May 29, 2008 8:43 am

Re: Stockfish NN release (NNUE)

Post by kranium » Sun Jun 07, 2020 3:24 pm

kranium wrote:
Sun Jun 07, 2020 10:53 am

Code: Select all

   # PLAYER                           : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish.nnue.halfkp_256x2-3    : 2526.5   24.8     57.5     100   57.5%
   2 Stockfish_9_x64_bmi2             : 2473.5   24.8     42.5     100   42.5%
   
1) Stockfish.nnue.halfkp_256x2-3  2526 :    100 (+25,=65,-10),  57.5 %
   vs.                                 :  games (  +,  =,  -),   (%) :   Diff,  SD, CFS (%)
   Stockfish_9_x64_bmi2                :    100 ( 25, 65, 10),  57.5 :    +53,  23,   99.0

2) Stockfish_9_x64_bmi2           2474 :    100 (+10,=65,-25),  42.5 %
   vs.                                 :  games (  +,  =,  -),   (%) :   Diff,  SD, CFS (%)
   Stockfish.nnue.halfkp_256x2-3       :    100 ( 10, 65, 25),  42.5 :    -53,  23,    1.0

Code: Select all

   # PLAYER                               : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish_10                         : 2503.4   26.2     53.0     104   51.0%
   2 Stockfish.nnue.halfkp_256x2-32-32    : 2496.6   26.2     51.0     104   49.0%
   
1) Stockfish_10                       2503 :    104 (+20,=66,-18),  51.0 %
   vs.                                     :  games (  +,  =,  -),   (%) :   Diff,  SD, CFS (%)
   Stockfish.nnue.halfkp_256x2-32-32       :    104 ( 20, 66, 18),  51.0 :     +7,  24,   61.1

2) Stockfish.nnue.halfkp_256x2-32-32  2497 :    104 (+18,=66,-20),  49.0 %
   vs.                                     :  games (  +,  =,  -),   (%) :   Diff,  SD, CFS (%)
   Stockfish_10                            :    104 ( 18, 66, 20),  49.0 :     -7,  24,   38.9    
Thx ChickenLogic-

I had some time to run some games, it plays awesome chess and appears to be somewhere around SF 10 strength...only 30-40 elo short of SF11.
Those games are 1+1 with 8 threads and 2048 Mb Hash each, on an intel i9-990k.

Looks like it has enormous potential...let's hope the SF team puts some resources into trying it.
The extra code needed is quite small, the system simply uses existing SF with the newly created NN evaluation.
Here is the source, halfkp network, and binaries I used in above testing...
https://www.dropbox.com/s/a18teq9o71rc9 ... 04.7z?dl=0

just unzip and add stockfish.nnue.halfkp_256x2-32-32.exe (or any binary actually) as an engine in your GUI.

cucumber
Posts: 128
Joined: Sun Oct 14, 2018 6:21 pm
Full name: JSmith

Re: Stockfish NN release (NNUE)

Post by cucumber » Sun Jun 07, 2020 5:09 pm

Is anyone able to get this to work on Linux? I get a segfault calling into wait_for_search_finished from ThreadPool::set almost immediately after running ./stockfish. The wait_for_search_finished is different from what I see in SF 11, so I tried replacing it with SF 11's. No dice. I have no idea why it's segfaulting and I'm having trouble debugging it.

edit: Regular Stockfish, compiled with the same compiler within minutes of compiling SF NNUE, works just fine.

Raphexon
Posts: 302
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: Stockfish NN release (NNUE)

Post by Raphexon » Sun Jun 07, 2020 5:27 pm

cucumber wrote:
Sun Jun 07, 2020 5:09 pm
Is anyone able to get this to work on Linux? I get a segfault calling into wait_for_search_finished from ThreadPool::set almost immediately after running ./stockfish. The wait_for_search_finished is different from what I see in SF 11, so I tried replacing it with SF 11's. No dice. I have no idea why it's segfaulting and I'm having trouble debugging it.

edit: Regular Stockfish, compiled with the same compiler within minutes of compiling SF NNUE, works just fine.
Does this help?

https://github.com/nodchip/Stockfish/co ... t-39726766

cucumber
Posts: 128
Joined: Sun Oct 14, 2018 6:21 pm
Full name: JSmith

Re: Stockfish NN release (NNUE)

Post by cucumber » Sun Jun 07, 2020 5:38 pm

Raphexon wrote:
Sun Jun 07, 2020 5:27 pm
cucumber wrote:
Sun Jun 07, 2020 5:09 pm
Is anyone able to get this to work on Linux? I get a segfault calling into wait_for_search_finished from ThreadPool::set almost immediately after running ./stockfish. The wait_for_search_finished is different from what I see in SF 11, so I tried replacing it with SF 11's. No dice. I have no idea why it's segfaulting and I'm having trouble debugging it.

edit: Regular Stockfish, compiled with the same compiler within minutes of compiling SF NNUE, works just fine.
Does this help?

https://github.com/nodchip/Stockfish/co ... t-39726766
Hm, thanks for the recommendation. I've just tried it, but the issue persists.

Damir
Posts: 2402
Joined: Mon Feb 11, 2008 2:53 pm
Location: Denmark
Full name: Damir Desevac

Re: Stockfish NN release (NNUE)

Post by Damir » Sun Jun 07, 2020 6:32 pm

Joerg Oster wrote:
Tue Jun 02, 2020 8:10 pm
ChickenLogic wrote:
Tue Jun 02, 2020 7:53 pm
This is only supposed to be a test run to see how far I can come so I can share good parameters for training. I will do D=6 after I'm content with my results. But I fear depth greater than D=6 really requires multiple people. The idea is to train an initial net and then do fen generation with the trained Fish. So in the end it might not matter too much what depth we start with.
I see.

I just started the 1st training run with the bigger net (halfkp) ...
the net will have a size of about 20 MB! That's quite a difference compared to 400 kB for the smaller net.
Hi Jörg :)

How do you see if the net is beeing used ? :o

cucumber
Posts: 128
Joined: Sun Oct 14, 2018 6:21 pm
Full name: JSmith

Re: Stockfish NN release (NNUE)

Post by cucumber » Sun Jun 07, 2020 8:08 pm

I set up a new windows virtual machine just to try to build and tweak this. I still segfault on launch when compiling on Windows with gcc 8. How are people building this? It doesn't produce a functioning executable on either of the OSes that I've tried.

Joerg Oster
Posts: 783
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Stockfish NN release (NNUE)

Post by Joerg Oster » Sun Jun 07, 2020 8:19 pm

Damir wrote:
Sun Jun 07, 2020 6:32 pm
Joerg Oster wrote:
Tue Jun 02, 2020 8:10 pm
ChickenLogic wrote:
Tue Jun 02, 2020 7:53 pm
This is only supposed to be a test run to see how far I can come so I can share good parameters for training. I will do D=6 after I'm content with my results. But I fear depth greater than D=6 really requires multiple people. The idea is to train an initial net and then do fen generation with the trained Fish. So in the end it might not matter too much what depth we start with.
I see.

I just started the 1st training run with the bigger net (halfkp) ...
the net will have a size of about 20 MB! That's quite a difference compared to 400 kB for the smaller net.
Hi Jörg :)

How do you see if the net is beeing used ? :o
SF runs noticeably slower with the net loaded.
Jörg Oster

Post Reply