Sergio Vieri second net is out

[MikeB]

sf1134's "lucky" run continues ...

Code: Select all

Date: 07/24/20 : 12:06:02

4000 game(s) loaded
Rank Name    Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR
---------------------------------------------------------------------------------------------------------
   1 sf1134   3501   0.0    7    7  4000 2015.5  50.4  939  908 2153  23.5  53.8  3499
   2 sf1732   3499   2.6    7    7  4000 1984.5  49.6  908  939 2153  22.7  53.8  3501
---------------------------------------------------------------------------------------------------------
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
        sf sf
sf1134     76
sf1732  23

4000 game(s) loaded

sf1134 totals:
games are mostly 60+0.6 ( maybe all - did not check)
overall they are pretty tight and anyone of the first 5 listed below in the summary could be the true "topdog" ...progress has been slow of late.
only an Elo spread from top to bottom

Code: Select all

Rank Name    Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR
---------------------------------------------------------------------------------------------------------
   1 sf1134   3103   0.0    3    3 24000 12086.0  50.4 5680 5509 12811  23.7  53.4  3101
   2 sf2141   3102   1.7    4    4 12000 6017.5  50.1 2841 2806 6353  23.7  52.9  3101
   3 sf0123   3101   0.1    7    7  4000 1990.0  49.7  919  940 2141  23.0  53.5  3103
   4 sf1732   3101   0.7    7    7  4000 1984.5  49.6  908  939 2153  22.7  53.8  3103
   5 sf1240   3101   0.2    6    6  6000 2977.0  49.6 1380 1426 3194  23.0  53.2  3103
   6 sf0640   3097   3.2   10   10  2000  983.5  49.2  463  496 1041  23.1  52.0  3103
   7 sf1844   3095   2.5    7    7  4000 1962.5  49.1  927 1002 2071  23.2  51.8  3102
---------------------------------------------------------------------------------------------------------
  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

ResultSet-EloRating>los
        sf sf sf sf sf sf sf
sf1134     74 69 75 81 87 96
sf2141  25    50 57 59 76 96
sf0123  30 49    55 57 73 87
sf1732  24 42 44    51 70 84
sf1240  18 40 42 48    70 85
sf0640  12 23 26 29 29    64
sf1844   3  3 12 15 14 35
ResultSet-EloRating>

[MikeB]

Can someone with good hardware test this one?

Size=256. By ribbit on discord.

"My first little 256 network.. I tested it on Honey-XI-NN with pretty good result against stockfish-dev and Leela ... (d24 validation used, 6menTB)

ribbit_0.1 - https://rapidu.net/9571752717/nn.bin "

You posted a link for a 20Mb for a file to a site that downloads at 200Kb.sec - you can do better than that - google drive, dropbox - whatever. I tried to download- too painful for me - sorry.

[MMarco]

Can someone with good hardware test this one?

Size=256. By ribbit on discord.

"My first little 256 network.. I tested it on Honey-XI-NN with pretty good result against stockfish-dev and Leela ... (d24 validation used, 6menTB)

ribbit_0.1 - https://rapidu.net/9571752717/nn.bin "

Code: Select all

Games Completed = 1000 of 1000 (Avg game length = 23.424 sec)
Settings = RR/128MB/6000ms+100ms/M 700cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\3M_08_10.epd(395)
Time = 6015 sec elapsed, 0 sec remaining
 1.  SF Ribbit                	590.0/1000	416-236-348  	(L: m=1 t=0 i=0 a=235)	(D: r=188 i=72 f=30 s=5 a=53)	(tpm=184.7 d=16.68 nps=966844)
 2.  SF_dev                   	410.0/1000	236-416-348  	(L: m=0 t=0 i=0 a=416)	(D: r=188 i=72 f=30 s=5 a=53)	(tpm=185.5 d=18.25 nps=1730005)

The level of good Sergio nets, but not the best one.

Thanks for testing!

[MMarco]

Can someone with good hardware test this one?

Size=256. By ribbit on discord.

"My first little 256 network.. I tested it on Honey-XI-NN with pretty good result against stockfish-dev and Leela ... (d24 validation used, 6menTB)

ribbit_0.1 - https://rapidu.net/9571752717/nn.bin "

You posted a link for a 20Mb for a file to a site that downloads at 200Kb.sec - you can do better than that - google drive, dropbox - whatever. I tried to download- too painful for me - sorry.

Sorry, it was the link given on the discord channel. I reuploaded it on another server:
https://gofile.io/d/ry7AuA

[carldaman]

Net 20200723-1134.bin is still champion.

Strange that both Kai and Ed got a regression with 1134.

Not strange at at really - happens all the time - engines can run into an a lucky or unlucky run. 1134 hasn't lost since it came out for ME - , yomv and ymmv - I don't worry about ut , I'm sure they dont worry about it and you shouldn't worry about it either. 1134 will lose sat some point so eventually world peace and harmony will be restored. ;>)

Furthermore, if a hundred people test a thousand nets against one opponent, even if the two engines are “the same”, five hundred of those nets are going to show up stronger than.

The matches are short and headline results are, well, whatever, you get the idea.

You have a point, but I was under the (perhaps wrong) impression that the matches weren't so short.

[MikeB]

Can someone with good hardware test this one?

Size=256. By ribbit on discord.

"My first little 256 network.. I tested it on Honey-XI-NN with pretty good result against stockfish-dev and Leela ... (d24 validation used, 6menTB)

ribbit_0.1 - https://rapidu.net/9571752717/nn.bin "

You posted a link for a 20Mb for a file to a site that downloads at 200Kb.sec - you can do better than that - google drive, dropbox - whatever. I tried to download- too painful for me - sorry.

Sorry, it was the link given on the discord channel. I reuploaded it on another server:
https://gofile.io/d/ry7AuA

Thanks!

[MMarco]

Its getting scary!!

Posted by SVieri:

Code: Select all

I ran 2344 vs 2141 overnight. TC: 3m+2s, 92 threads.

Score of StockfishNNUE 2344 vs StockfishNNUE 20200722-2141: 14 - 5 - 33 [0.587]
...      StockfishNNUE 2344 playing White: 12 - 0 - 14  [0.731] 26
...      StockfishNNUE 2344 playing Black: 2 - 5 - 19  [0.442] 26
...      White vs Black: 17 - 2 - 33  [0.644] 52
Elo difference: 60.7 +/- 56.8, LOS: 98.1 %, DrawRatio: 63.5 %
52 of 100 games finished.

[MikeB]

Its getting scary!!

Posted by SVieri:

Code: Select all

I ran 2344 vs 2141 overnight. TC: 3m+2s, 92 threads.

Score of StockfishNNUE 2344 vs StockfishNNUE 20200722-2141: 14 - 5 - 33 [0.587]
...      StockfishNNUE 2344 playing White: 12 - 0 - 14  [0.731] 26
...      StockfishNNUE 2344 playing Black: 2 - 5 - 19  [0.442] 26
...      White vs Black: 17 - 2 - 33  [0.644] 52
Elo difference: 60.7 +/- 56.8, LOS: 98.1 %, DrawRatio: 63.5 %
52 of 100 games finished.

Wow - that is incredible!

While back on the farm, NNUE 1134's unbelievable "lucky" streak continues...

Code: Select all

tc/base+inc: 30+0.30
games planned: 4000

Current date : time (EDST)
Date: 07/25/20 : 00:05:53

Projected-> Time: 2h:8m:0s
Total->  RunTime: 1h:59m:9s

4000 game(s) loaded
Rank Name  Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR
---------------------------------------------------------------------------------------------------------

   1 1134   3501   0.0    7    7  4000 2016.5  50.4  997  964 2039  24.9  51.0  3499
   2 0545   3499   2.9    7    7  4000 1983.5  49.6  964  997 2039  24.1  51.0  3501
---------------------------------------------------------------------------------------------------------

  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
      11 05
1134     78
0545  21

I had temporarily skipped over 2344, but will do that one next..

[JohnS]

Its getting scary!!

Posted by SVieri:

Code: Select all

I ran 2344 vs 2141 overnight. TC: 3m+2s, 92 threads.

Score of StockfishNNUE 2344 vs StockfishNNUE 20200722-2141: 14 - 5 - 33 [0.587]
...      StockfishNNUE 2344 playing White: 12 - 0 - 14  [0.731] 26
...      StockfishNNUE 2344 playing Black: 2 - 5 - 19  [0.442] 26
...      White vs Black: 17 - 2 - 33  [0.644] 52
Elo difference: 60.7 +/- 56.8, LOS: 98.1 %, DrawRatio: 63.5 %
52 of 100 games finished.

Just did a quick test of 2344 against K14 using Nunn1 openings, G10s+0.2s - result +10 =9 -1! The loss was on the black side of a Winawer French in 57 moves. SFnnue won the reverse game in 29 moves.

Here is a great game by SFnnue - note the final position.

[pgn][Event "SF-NNUE - Komodo 14, Nunn1, G10s + 0.2s"] [Site "Home"] [Date "2020.07.25"] [Round "1"] [White "Stockfish+NNUE"] [Black "Komodo 14 64-bit"] [Result "1-0"] [TimeControl "10+0.2"] [Time "14:14:46"] [Board "15"] [Termination "adjudication by engines' scores"] [ECO "E99"] [Opening "King's Indian"] 1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 5. Be2 O-O 6. Nf3 e5 7. O-O Nc6 {E99: King's Indian, orthodox, Aronin-Taimanov, Benko attack} 8. d5 Ne7 9. Ne1 Ne8 10. Be3 {End of opening} f5 {-0.38/16 0.4 618934} 11. f3 {+0.65/19 1.2 1391417} c5 {-0.48/17 0.9 1467428} 12. Nd3 {+0.63/18 0.3 377773} Bd7 {-0.49/17 0.3 516060} 13. b4 {+0.89/17 0.2 252551} b6 {-0.54/17 0.6 1016905} 14. a4 {+1.06/18 1.1 1204265} a5 {-0.62/19 0.8 1334147} 15. bxc5 {+1.10/15 0.2 195816} bxc5 {-0.56/18 0.4 656201} 16. Rb1 {+0.76/21 3.5 3621517} f4 {-0.67/18 0.9 1463450} 17. Bf2 {+0.91/16 0.2 216667} h5 {-0.59/19 1.2 1863345} 18. Nxc5 {+1.93/18 0.4 479507} dxc5 {-0.82/16 0.2 408802} 19. Bxc5 {+2.27/19 0.1 164198} h4 {-0.98/16 0.2 365931} 20. Bb6 {+2.66/18 0.7 720791} Qc8 {-1.15/16 0.6 989696} 21. h3 {+2.36/16 0.2 226631} Kh7 {-1.09/17 0.3 498227} 22. c5 {+3.62/17 0.5 503384} Nf6 {-1.19/16 0.2 382084} 23. Nb5 {+3.63/16 0.4 392546} Bxb5 {-1.48/16 0.2 261530} 24. Bxb5 {+4.03/18 0.3 343746} Nd7 {-1.15/18 0.2 391155} 25. Bxd7 {+4.27/20 0.4 426614} Qxd7 {-1.31/20 0.6 954297} 26. Qc2 {+4.01/19 0.4 466334} Rf7 {-1.33/20 0.9 1465353} 27. c6 {+3.51/21 0.5 636578} Qe8 {-1.44/19 0.3 537145} 28. Qc3 {+4.99/23 1.2 1385561} Nc8 {-1.56/17 0.3 527751} 29. Bc5 {+5.04/20 0.2 223209} Bf8 {-1.65/19 0.2 346337} 30. Bxf8 {+4.79/21 0.6 672167} Rxf8 {-2.14/20 0.4 652274} 31. Rb7+ {+5.22/19 0.3 343815} Kh6 {-2.31/21 0.8 1456550} 32. Rfb1 {+5.35/19 0.2 177574} Nd6 {-2.34/19 0.3 485032} 33. Rd7 {+5.64/19 0.1 174131} Rf6 {-2.35/19 0.2 347856} 34. Kh2 {+6.02/18 0.3 351388} g5 {-1.47/18 0.3 630412} 35. Rb5 {+6.41/14 0.1 119822} Rc8 {-2.51/19 0.6 1080346} 36. Rxa5 {+6.29/18 0.3 346756} Nf7 {-2.55/18 0.3 497819} 37. Raa7 {+4.47/17 0.8 969218} Kg6 {-2.98/19 0.8 1462728} 38. Qc5 {+5.31/17 0.2 233415} g4 {-3.67/21 0.9 1880887} 39. hxg4 {+5.68/18 0.1 108279} Qh8 {-3.74/19 0.3 606130} 40. Rxf7 {+6.73/18 0.1 155666} Rxf7 {-4.10/17 0.2 464527} 41. Rxf7 {+6.56/16 0.1 152142} Kxf7 {-4.17/18 0.2 488041} 42. Qd6 {+7.04/15 0.1 207065} Qe8 {-2.01/17 0.2 454554} 43. Qh6 {+7.51/15 0.1 147087} Kg8 {-2.36/17 0.2 485455} 44. Kh3 {+7.83/16 0.1 202456} Rc7 {-2.65/18 0.4 911300} 45. Qd6 {+6.63/20 0.3 394130} Qc8 {-2.53/16 0.2 514872} 46. Qxe5 {+7.65/15 0.1 153608} Ra7 {-3.59/18 0.5 1140492} 47. Qxf4 {+8.53/17 0.4 510855} Rf7 {-3.49/18 0.3 702900} 48. Qe5 {+8.81/14 0.1 140761} Qc7 {-3.60/18 0.4 954921} 49. Qxc7 {+10.11/15 0.1 185229} Rxc7 {-5.29/18 0.1 383304} 50. Kxh4 {+10.21/15 0.1 184926} Kf7 {-5.73/19 0.2 572059} 51. Kg5 {+10.52/16 0.2 298133} Ke7 {-7.36/21 0.6 1749039} 52. e5 {+10.63/15 0.1 207669} Rc8 {-8.90/21 0.2 611988} 53. f4 {+10.71/15 0.2 234998} Rg8+ {-6.48/16 0.2 548227} 54. Kf5 {+11.02/15 0.1 225122} Rf8+ {-8.19/17 0.2 520910} 1-0[/pgn]

[Laskos]

Its getting scary!!

Posted by SVieri:

Code: Select all

I ran 2344 vs 2141 overnight. TC: 3m+2s, 92 threads.

Score of StockfishNNUE 2344 vs StockfishNNUE 20200722-2141: 14 - 5 - 33 [0.587]
...      StockfishNNUE 2344 playing White: 12 - 0 - 14  [0.731] 26
...      StockfishNNUE 2344 playing Black: 2 - 5 - 19  [0.442] 26
...      White vs Black: 17 - 2 - 33  [0.644] 52
Elo difference: 60.7 +/- 56.8, LOS: 98.1 %, DrawRatio: 63.5 %
52 of 100 games finished.

Too few games to say anything with high confidence. Not even clear that 2344 is stronger than 2141, cherry picked LOS of 98% doesn't qualify as a stopping rule. 99.9% or higher are needed when cherry picking to have some confidence in superiority, and even higher for small number of games.