Can the sardine! NNUE clobbers SF.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Raphexon
Posts: 476
Joined: Sun Mar 17, 2019 12:00 pm
Full name: Henk Drost

Can the sardine! NNUE clobbers SF.

Post by Raphexon »

https://www.mediafire.com/file/qsxolqqf ... 2.pgn/file

SFNNUE 27-6 net. Latest binaries
SFdev from 10 june
Both 1 thread and 128MB hash. Rest default.
Time control: 120s + 1.2s
Hardware: i5-10400
Books: HERT 500
Chess Interface: cutechess GUI
Results:
Score of NNUE vs stockfish10june2020: 178 - 92 - 730 [0.543]
Elo difference: 29.95 +/- 11.11



[pgn][Round "62"]
[White "NNUE"]
[Black "stockfish10june2020"]
[Result "1-0"]
[ECO "C18"]
[Opening "French"]
[PlyCount "153"]
[TimeControl "120+1.2"]
[Variation "Winawer, Advance Variation"]

1. e4 {book} e6 {book} 2. d4 {book} d5 {book} 3. Nc3 {book} Bb4 {book}
4. e5 {book} c5 {book} 5. a3 {book} Bxc3+ {book} 6. bxc3 {book} Qa5 {book}
7. Bd2 {+0.43/24 3.7s} Qa4 {-0.08/25 8.6s} 8. Qb1 {+0.48/24 2.9s}
a6 {-0.09/26 7.2s} 9. Nf3 {+0.73/23 3.0s} Nd7 {-0.27/26 7.3s}
10. Be2 {+0.60/23 7.8s} Ne7 {-0.33/22 1.1s} 11. h4 {+0.87/21 1.8s}
h6 {-0.33/21 3.1s} 12. h5 {+0.86/21 2.5s} Qc6 {-0.25/25 5.2s}
13. Rh4 {+0.95/21 2.1s} Nf5 {0.00/23 2.1s} 14. Rg4 {+1.25/21 2.8s}
O-O {0.00/23 2.4s} 15. Qc1 {+1.40/22 2.6s} Kh7 {-1.25/27 16s}
16. Rf4 {+1.45/22 3.9s} b6 {-1.01/23 4.1s} 17. g4 {+2.11/24 4.5s}
Ne7 {-0.92/22 2.3s} 18. Bd3+ {+2.53/23 3.8s} Kh8 {-1.18/27 9.5s}
19. Rf6 {+4.10/25 5.1s} Nxf6 {-3.36/26 4.0s} 20. exf6 {+3.91/24 2.4s}
e5 {-3.21/26 2.0s} 21. fxe7 {+3.99/25 2.0s} Re8 {-3.17/25 3.6s}
22. Nxe5 {+4.16/24 2.0s} Rxe7 {-3.58/32 21s} 23. Be2 {+4.08/24 2.8s}
Qe6 {-3.59/24 1.2s} 24. Bf4 {+4.18/24 3.6s} Kg8 {-2.95/22 1.7s}
25. Qe3 {+4.15/23 2.5s} Re8 {-3.02/25 7.2s} 26. Kd2 {+4.49/24 6.1s}
f6 {-3.12/24 6.6s} 27. Ng6 {+4.34/23 2.6s} Qd7 {-3.36/24 2.4s}
28. Qd3 {+4.67/21 2.0s} c4 {-3.44/26 8.3s} 29. Qg3 {+4.76/24 4.0s}
Ra7 {-3.58/27 5.1s} 30. Rg1 {+4.76/25 3.5s} Qa4 {-2.70/21 1.4s}
31. Bf3 {+5.38/24 2.7s} Qc6 {-3.20/28 5.6s} 32. Re1 {+5.48/23 2.6s}
Rxe1 {-3.23/24 0.94s} 33. Kxe1 {+5.55/23 2.4s} Kf7 {-3.57/23 2.3s}
34. Kd2 {+6.23/23 2.8s} Ke8 {-4.60/24 3.2s} 35. Bg2 {+6.10/21 3.0s}
Qd7 {-4.15/25 2.4s} 36. Be3 {+6.58/22 2.4s} Qc6 {-4.37/24 0.78s}
37. f4 {+6.55/24 3.2s} Qd6 {-4.46/22 1.0s} 38. Qf3 {+6.67/23 2.5s}
Be6 {-4.70/25 4.3s} 39. g5 {+6.73/25 2.7s} f5 {-4.65/19 0.38s}
40. Qg3 {+6.80/26 4.7s} Bc8 {-4.83/23 2.0s} 41. Bf3 {+6.77/24 2.5s}
Rc7 {-5.00/23 1.8s} 42. Qg2 {+7.85/26 2.3s} Bb7 {-4.85/22 0.33s}
43. gxh6 {+8.97/26 2.0s} gxh6 {-4.16/21 0.42s} 44. Nh4 {+9.10/25 2.0s}
Rf7 {-5.46/25 1.2s} 45. Qg8+ {+9.56/25 3.3s} Rf8 {-5.27/27 0.91s}
46. Qh7 {+9.53/25 2.2s} Rf7 {-6.59/24 1.6s} 47. Qh8+ {+9.77/24 2.2s}
Rf8 {-6.75/23 0.56s} 48. Qe5+ {+10.05/25 3.8s} Qxe5 {-6.78/24 1.3s}
49. dxe5 {+10.23/25 2.9s} Kd7 {-7.69/26 4.4s} 50. Bxb6 {+10.38/24 2.1s}
Kc6 {-7.98/25 1.7s} 51. Ba5 {+10.83/23 2.4s} Rf7 {-8.03/20 0.14s}
52. Ke3 {+11.33/21 1.9s} Bc8 {-8.20/23 1.9s} 53. Kd4 {+11.77/24 3.1s}
Be6 {-8.35/20 0.26s} 54. Ng6 {+11.83/23 2.8s} Rg7 {-9.16/23 2.5s}
55. Bb4 {+11.96/24 2.2s} a5 {-9.44/21 0.46s} 56. Bxa5 {+12.54/24 2.9s}
Bg8 {-8.94/20 1.8s} 57. Bb4 {+12.52/24 2.6s} Be6 {-10.04/23 1.3s}
58. Bc5 {+12.94/24 2.4s} Rb7 {-9.92/18 0.60s} 59. Nf8 {+13.34/24 2.3s}
Bg8 {-10.63/22 0.93s} 60. Bb4 {+13.42/23 2.0s} Bf7 {-10.88/26 1.1s}
61. e6 {+14.28/25 1.8s} Be8 {-16.19/25 1.1s} 62. Bxd5+ {+15.15/24 1.9s}
Kc7 {-59.17/26 0.82s} 63. Bxb7 {+15.54/25 2.0s} Kxb7 {-154.14/37 2.6s}
64. Ng6 {+15.72/23 2.3s} Ka7 {-33.90/21 1.5s} 65. Ke5 {+16.83/24 2.1s}
Kb6 {-9.35/18 0.18s} 66. Kxf5 {+M31/38 1.8s} Kb7 {-60.90/22 1.8s}
67. Ke5 {+M21/57 1.7s} Kb6 {-60.90/23 0.42s} 68. f5 {+M19/58 1.8s}
Ba4 {-M20/27 0.83s} 69. f6 {+M17/59 1.6s} Bxc2 {-M16/33 0.46s}
70. e7 {+M15/61 1.6s} Ba4 {-M14/51 0.33s} 71. f7 {+M13/68 1.7s}
Bc6 {-M12/59 0.41s} 72. e8=Q {+M11/71 1.5s} Bxe8 {-M10/76 0.43s}
73. fxe8=Q {+M9/126 1.5s} Ka6 {-M8/178 0.49s} 74. Qc6+ {+M7/245 0.93s}
Ka7 {-M6/10 0s} 75. Qb5 {+M5/245 0.012s} Ka8 {-M4/23 0s}
76. Qa6+ {+M3/245 0.005s} Kb8 {-M2/15 0s} 77. Bd6# {+M1/245 0.004s, White mates}[/pgn]
Raphexon
Posts: 476
Joined: Sun Mar 17, 2019 12:00 pm
Full name: Henk Drost

Re: Can the sardine! NNUE clobbers SF.

Post by Raphexon »

Binary + net can be downloaded from here:

https://workupload.com/file/ggEUrvNVgmH
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Can the sardine! NNUE clobbers SF.

Post by zullil »

Raphexon wrote: Thu Jul 16, 2020 11:12 am Binary + net can be downloaded from here:

https://workupload.com/file/ggEUrvNVgmH
I have a Linux system. Are there build instructions for this engine anywhere?
Raphexon
Posts: 476
Joined: Sun Mar 17, 2019 12:00 pm
Full name: Henk Drost

Re: Can the sardine! NNUE clobbers SF.

Post by Raphexon »

zullil wrote: Thu Jul 16, 2020 11:14 am
Raphexon wrote: Thu Jul 16, 2020 11:12 am Binary + net can be downloaded from here:

https://workupload.com/file/ggEUrvNVgmH
I have a Linux system. Are there build instructions for this engine anywhere?
Nodchip's repo contains all the needed code to compile the binary.
Net is still can still be had from the previous link.
https://github.com/nodchip/Stockfish/bl ... c/Makefile

Profile nnue is the one you want.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Can the sardine! NNUE clobbers SF.

Post by zullil »

Raphexon wrote: Thu Jul 16, 2020 11:26 am
zullil wrote: Thu Jul 16, 2020 11:14 am
Raphexon wrote: Thu Jul 16, 2020 11:12 am Binary + net can be downloaded from here:

https://workupload.com/file/ggEUrvNVgmH
I have a Linux system. Are there build instructions for this engine anywhere?
Nodchip's repo contains all the needed code to compile the binary.
Net is still can still be had from the previous link.
https://github.com/nodchip/Stockfish/bl ... c/Makefile

Profile nnue is the one you want.
Thanks. It would be very helpful if info about nnue were added to the Makefile's help target. :D (Or to the Readme.)
lkaufman
Posts: 5981
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: Can the sardine! NNUE clobbers SF.

Post by lkaufman »

Raphexon wrote: Thu Jul 16, 2020 11:06 am https://www.mediafire.com/file/qsxolqqf ... 2.pgn/file

SFNNUE 27-6 net. Latest binaries
SFdev from 10 june
Both 1 thread and 128MB hash. Rest default.
Time control: 120s + 1.2s
Hardware: i5-10400
Books: HERT 500
Chess Interface: cutechess GUI
Results:
Score of NNUE vs stockfish10june2020: 178 - 92 - 730 [0.543]
Elo difference: 29.95 +/- 11.11



[pgn][Round "62"]
[White "NNUE"]
[Black "stockfish10june2020"]
[Result "1-0"]
[ECO "C18"]
[Opening "French"]
[PlyCount "153"]
[TimeControl "120+1.2"]
[Variation "Winawer, Advance Variation"]

1. e4 {book} e6 {book} 2. d4 {book} d5 {book} 3. Nc3 {book} Bb4 {book}
4. e5 {book} c5 {book} 5. a3 {book} Bxc3+ {book} 6. bxc3 {book} Qa5 {book}
7. Bd2 {+0.43/24 3.7s} Qa4 {-0.08/25 8.6s} 8. Qb1 {+0.48/24 2.9s}
a6 {-0.09/26 7.2s} 9. Nf3 {+0.73/23 3.0s} Nd7 {-0.27/26 7.3s}
10. Be2 {+0.60/23 7.8s} Ne7 {-0.33/22 1.1s} 11. h4 {+0.87/21 1.8s}
h6 {-0.33/21 3.1s} 12. h5 {+0.86/21 2.5s} Qc6 {-0.25/25 5.2s}
13. Rh4 {+0.95/21 2.1s} Nf5 {0.00/23 2.1s} 14. Rg4 {+1.25/21 2.8s}
O-O {0.00/23 2.4s} 15. Qc1 {+1.40/22 2.6s} Kh7 {-1.25/27 16s}
16. Rf4 {+1.45/22 3.9s} b6 {-1.01/23 4.1s} 17. g4 {+2.11/24 4.5s}
Ne7 {-0.92/22 2.3s} 18. Bd3+ {+2.53/23 3.8s} Kh8 {-1.18/27 9.5s}
19. Rf6 {+4.10/25 5.1s} Nxf6 {-3.36/26 4.0s} 20. exf6 {+3.91/24 2.4s}
e5 {-3.21/26 2.0s} 21. fxe7 {+3.99/25 2.0s} Re8 {-3.17/25 3.6s}
22. Nxe5 {+4.16/24 2.0s} Rxe7 {-3.58/32 21s} 23. Be2 {+4.08/24 2.8s}
Qe6 {-3.59/24 1.2s} 24. Bf4 {+4.18/24 3.6s} Kg8 {-2.95/22 1.7s}
25. Qe3 {+4.15/23 2.5s} Re8 {-3.02/25 7.2s} 26. Kd2 {+4.49/24 6.1s}
f6 {-3.12/24 6.6s} 27. Ng6 {+4.34/23 2.6s} Qd7 {-3.36/24 2.4s}
28. Qd3 {+4.67/21 2.0s} c4 {-3.44/26 8.3s} 29. Qg3 {+4.76/24 4.0s}
Ra7 {-3.58/27 5.1s} 30. Rg1 {+4.76/25 3.5s} Qa4 {-2.70/21 1.4s}
31. Bf3 {+5.38/24 2.7s} Qc6 {-3.20/28 5.6s} 32. Re1 {+5.48/23 2.6s}
Rxe1 {-3.23/24 0.94s} 33. Kxe1 {+5.55/23 2.4s} Kf7 {-3.57/23 2.3s}
34. Kd2 {+6.23/23 2.8s} Ke8 {-4.60/24 3.2s} 35. Bg2 {+6.10/21 3.0s}
Qd7 {-4.15/25 2.4s} 36. Be3 {+6.58/22 2.4s} Qc6 {-4.37/24 0.78s}
37. f4 {+6.55/24 3.2s} Qd6 {-4.46/22 1.0s} 38. Qf3 {+6.67/23 2.5s}
Be6 {-4.70/25 4.3s} 39. g5 {+6.73/25 2.7s} f5 {-4.65/19 0.38s}
40. Qg3 {+6.80/26 4.7s} Bc8 {-4.83/23 2.0s} 41. Bf3 {+6.77/24 2.5s}
Rc7 {-5.00/23 1.8s} 42. Qg2 {+7.85/26 2.3s} Bb7 {-4.85/22 0.33s}
43. gxh6 {+8.97/26 2.0s} gxh6 {-4.16/21 0.42s} 44. Nh4 {+9.10/25 2.0s}
Rf7 {-5.46/25 1.2s} 45. Qg8+ {+9.56/25 3.3s} Rf8 {-5.27/27 0.91s}
46. Qh7 {+9.53/25 2.2s} Rf7 {-6.59/24 1.6s} 47. Qh8+ {+9.77/24 2.2s}
Rf8 {-6.75/23 0.56s} 48. Qe5+ {+10.05/25 3.8s} Qxe5 {-6.78/24 1.3s}
49. dxe5 {+10.23/25 2.9s} Kd7 {-7.69/26 4.4s} 50. Bxb6 {+10.38/24 2.1s}
Kc6 {-7.98/25 1.7s} 51. Ba5 {+10.83/23 2.4s} Rf7 {-8.03/20 0.14s}
52. Ke3 {+11.33/21 1.9s} Bc8 {-8.20/23 1.9s} 53. Kd4 {+11.77/24 3.1s}
Be6 {-8.35/20 0.26s} 54. Ng6 {+11.83/23 2.8s} Rg7 {-9.16/23 2.5s}
55. Bb4 {+11.96/24 2.2s} a5 {-9.44/21 0.46s} 56. Bxa5 {+12.54/24 2.9s}
Bg8 {-8.94/20 1.8s} 57. Bb4 {+12.52/24 2.6s} Be6 {-10.04/23 1.3s}
58. Bc5 {+12.94/24 2.4s} Rb7 {-9.92/18 0.60s} 59. Nf8 {+13.34/24 2.3s}
Bg8 {-10.63/22 0.93s} 60. Bb4 {+13.42/23 2.0s} Bf7 {-10.88/26 1.1s}
61. e6 {+14.28/25 1.8s} Be8 {-16.19/25 1.1s} 62. Bxd5+ {+15.15/24 1.9s}
Kc7 {-59.17/26 0.82s} 63. Bxb7 {+15.54/25 2.0s} Kxb7 {-154.14/37 2.6s}
64. Ng6 {+15.72/23 2.3s} Ka7 {-33.90/21 1.5s} 65. Ke5 {+16.83/24 2.1s}
Kb6 {-9.35/18 0.18s} 66. Kxf5 {+M31/38 1.8s} Kb7 {-60.90/22 1.8s}
67. Ke5 {+M21/57 1.7s} Kb6 {-60.90/23 0.42s} 68. f5 {+M19/58 1.8s}
Ba4 {-M20/27 0.83s} 69. f6 {+M17/59 1.6s} Bxc2 {-M16/33 0.46s}
70. e7 {+M15/61 1.6s} Ba4 {-M14/51 0.33s} 71. f7 {+M13/68 1.7s}
Bc6 {-M12/59 0.41s} 72. e8=Q {+M11/71 1.5s} Bxe8 {-M10/76 0.43s}
73. fxe8=Q {+M9/126 1.5s} Ka6 {-M8/178 0.49s} 74. Qc6+ {+M7/245 0.93s}
Ka7 {-M6/10 0s} 75. Qb5 {+M5/245 0.012s} Ka8 {-M4/23 0s}
76. Qa6+ {+M3/245 0.005s} Kb8 {-M2/15 0s} 77. Bd6# {+M1/245 0.004s, White mates}[/pgn]
I ran a similar test, at 3' + 3", 1 one thread, July 6 SF, 5 GHz i7, Hert 250 lowdraw book, +31 elo after 292 games, confirming your result.
Komodo rules!
Raphexon
Posts: 476
Joined: Sun Mar 17, 2019 12:00 pm
Full name: Henk Drost

Re: Can the sardine! NNUE clobbers SF.

Post by Raphexon »

I am probably going to need some help submitting NNUE as patch on test.stockfishchess.

Or maybe somebody else wants to do it.
Main difficulty for me is getting nn.bin from the github repo to the right place so I can just issue a standard UCI command for the test.

https://github.com/gekkehenker/Stockfish
User avatar
cdani
Posts: 2204
Joined: Sat Jan 18, 2014 10:24 am
Location: Andorra

Re: Can the sardine! NNUE clobbers SF.

Post by cdani »

Congratulations! Is really very strong, stronger than regular Stockfish. I'm doing a little test at 5 minutes + 2 seconds and is wining also, like the other tests published here.
User avatar
Rebel
Posts: 7032
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: Can the sardine! NNUE clobbers SF.

Post by Rebel »

Played a quick 100 games match, will play more later.

Code: Select all

Score of sf11 vs henk: 20 - 36 - 44  [0.420] 100
...      sf11 playing White: 14 - 11 - 25  [0.530] 50
...      sf11 playing Black: 6 - 25 - 19  [0.310] 50
...      White vs Black: 39 - 17 - 44  [0.610] 100
Elo difference: -56.1 +/- 51.4, LOS: 1.6 %, DrawRatio: 44.0 %
Finished match
Excellent result and without becoming a patron :wink:

Even more impressive considering the NPS

Code: Select all

SF11 bench NPS 1.2 million
Henk bench NPS 578.559
Depth overview

Code: Select all

Engine                 Depth       Time   Games     Moves  Average Forfeit  Book Depth     MIDG   EARLY    ENDG    LATE
sf11                   22.48    1:03:03     100      7282    0.52     0      826  8.26    18.74 | 18.92 | 21.82 | 30.99
henk                   21.37    1:02:53     100      7279    0.52     0      795  7.95    17.29 | 17.20 | 20.06 | 31.48
One thing, the sim-test crashes, is it possible to fix this? The similarity with the Lc0 related nets would be interesting.
90% of coding is debugging, the other 10% is writing bugs.
User avatar
Rebel
Posts: 7032
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: Can the sardine! NNUE clobbers SF.

Post by Rebel »

2000 40m/20s games

Code: Select all

Score of henk vs sf11: 621 - 433 - 946  [0.547] 2000
...      henk playing White: 381 - 150 - 469  [0.616] 1000
...      henk playing Black: 240 - 283 - 477  [0.478] 1000
...      White vs Black: 664 - 390 - 946  [0.569] 2000
Elo difference: 32.8 +/- 11.0, LOS: 100.0 %, DrawRatio: 47.3 %
Finished match
90% of coding is debugging, the other 10% is writing bugs.