Sergio Vieri second net is out

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
M ANSARI
Posts: 3500
Joined: Thu Mar 16, 2006 6:10 pm

Re: Sergio Vieri second net is out

Post by M ANSARI » Sat Jul 25, 2020 6:10 am

Never mind wrong score read. Is there a working version of this engine that we can try out on Fritz 14 GUI where it works properly without any editing?

Thanks

Milos
Posts: 4025
Joined: Wed Nov 25, 2009 12:47 am

Re: Sergio Vieri second net is out

Post by Milos » Sat Jul 25, 2020 6:15 am

Laskos wrote:
Sat Jul 25, 2020 4:40 am
MMarco wrote:
Sat Jul 25, 2020 3:11 am
Its getting scary!!

Posted by SVieri:

Code: Select all

I ran 2344 vs 2141 overnight. TC: 3m+2s, 92 threads.

Score of StockfishNNUE 2344 vs StockfishNNUE 20200722-2141: 14 - 5 - 33 [0.587]
...      StockfishNNUE 2344 playing White: 12 - 0 - 14  [0.731] 26
...      StockfishNNUE 2344 playing Black: 2 - 5 - 19  [0.442] 26
...      White vs Black: 17 - 2 - 33  [0.644] 52
Elo difference: 60.7 +/- 56.8, LOS: 98.1 %, DrawRatio: 63.5 %
52 of 100 games finished.
Too few games to say anything with high confidence. Not even clear that 2344 is stronger than 2141, cherry picked LOS of 98% doesn't qualify as a stopping rule. 99.9% or higher are needed when cherry picking to have some confidence in superiority, and even higher for small number of games.
There are indeed too few games, however, some things indicate that this is a bit more than just statistical noise. First, draw ratio is relatively high and second 3m+2s on 92 threads is equivalent to LTC on a single thread so games quality must be quite high and subsequently noise lower than your typical test games.

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Sergio Vieri second net is out

Post by Laskos » Sat Jul 25, 2020 6:29 am

Milos wrote:
Sat Jul 25, 2020 6:15 am
Laskos wrote:
Sat Jul 25, 2020 4:40 am
MMarco wrote:
Sat Jul 25, 2020 3:11 am
Its getting scary!!

Posted by SVieri:

Code: Select all

I ran 2344 vs 2141 overnight. TC: 3m+2s, 92 threads.

Score of StockfishNNUE 2344 vs StockfishNNUE 20200722-2141: 14 - 5 - 33 [0.587]
...      StockfishNNUE 2344 playing White: 12 - 0 - 14  [0.731] 26
...      StockfishNNUE 2344 playing Black: 2 - 5 - 19  [0.442] 26
...      White vs Black: 17 - 2 - 33  [0.644] 52
Elo difference: 60.7 +/- 56.8, LOS: 98.1 %, DrawRatio: 63.5 %
52 of 100 games finished.
Too few games to say anything with high confidence. Not even clear that 2344 is stronger than 2141, cherry picked LOS of 98% doesn't qualify as a stopping rule. 99.9% or higher are needed when cherry picking to have some confidence in superiority, and even higher for small number of games.
There are indeed too few games, however, some things indicate that this is a bit more than just statistical noise. First, draw ratio is relatively high and second 3m+2s on 92 threads is equivalent to LTC on a single thread so games quality must be quite high and subsequently noise lower than your typical test games.
I don't understand the noise other than statistical noise. Statistically, LTC and STC are the same, and there is no any quantification (estimation) of variance due to say "STC noise".

User avatar
MikeB
Posts: 4647
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Sergio Vieri second net is out

Post by MikeB » Sat Jul 25, 2020 6:37 am

MMarco wrote:
Sat Jul 25, 2020 3:11 am
Its getting scary!!

Posted by SVieri:

Code: Select all

I ran 2344 vs 2141 overnight. TC: 3m+2s, 92 threads.

Score of StockfishNNUE 2344 vs StockfishNNUE 20200722-2141: 14 - 5 - 33 [0.587]
...      StockfishNNUE 2344 playing White: 12 - 0 - 14  [0.731] 26
...      StockfishNNUE 2344 playing Black: 2 - 5 - 19  [0.442] 26
...      White vs Black: 17 - 2 - 33  [0.644] 52
Elo difference: 60.7 +/- 56.8, LOS: 98.1 %, DrawRatio: 63.5 %
52 of 100 games finished.
This is looking real to me - 1134's "lucky" streak will soon be no more

Code: Select all

pgn file: c:/cluster.mfb/pgn/2007250215-23441134.pgn
tc/base+inc: 60+0.60
games planned: 4000
Threads: 2
Hash: 256

Current date : time (EDST)
Date: 07/25/20 : 02:31:59

Projected-> Time: 7h:6m:40s
Running  -> Time: 0h:16m:52s

136 game(s) loaded
Rank Name  Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR
---------------------------------------------------------------------------------------------------------

   1 2344   3515   0.0   38   38   136   74.0  54.4   38   26   72  27.9  52.9  3485
   2 1134   3485  30.2   38   38   136   62.0  45.6   26   38   72  19.1  52.9  3515
---------------------------------------------------------------------------------------------------------

  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
      23 11
2344     93
1134   6

136 game(s) loaded

loops scheduled: 5/190

waiting: 128
  ...seconds remaining:   48
Last edited by MikeB on Sat Jul 25, 2020 6:45 am, edited 2 times in total.
Image

Milos
Posts: 4025
Joined: Wed Nov 25, 2009 12:47 am

Re: Sergio Vieri second net is out

Post by Milos » Sat Jul 25, 2020 6:39 am

Laskos wrote:
Sat Jul 25, 2020 6:29 am
Milos wrote:
Sat Jul 25, 2020 6:15 am
Laskos wrote:
Sat Jul 25, 2020 4:40 am
MMarco wrote:
Sat Jul 25, 2020 3:11 am
Its getting scary!!

Posted by SVieri:

Code: Select all

I ran 2344 vs 2141 overnight. TC: 3m+2s, 92 threads.

Score of StockfishNNUE 2344 vs StockfishNNUE 20200722-2141: 14 - 5 - 33 [0.587]
...      StockfishNNUE 2344 playing White: 12 - 0 - 14  [0.731] 26
...      StockfishNNUE 2344 playing Black: 2 - 5 - 19  [0.442] 26
...      White vs Black: 17 - 2 - 33  [0.644] 52
Elo difference: 60.7 +/- 56.8, LOS: 98.1 %, DrawRatio: 63.5 %
52 of 100 games finished.
Too few games to say anything with high confidence. Not even clear that 2344 is stronger than 2141, cherry picked LOS of 98% doesn't qualify as a stopping rule. 99.9% or higher are needed when cherry picking to have some confidence in superiority, and even higher for small number of games.
There are indeed too few games, however, some things indicate that this is a bit more than just statistical noise. First, draw ratio is relatively high and second 3m+2s on 92 threads is equivalent to LTC on a single thread so games quality must be quite high and subsequently noise lower than your typical test games.
I don't understand the noise other than statistical noise. Statistically, LTC and STC are the same, and there is no any quantification (estimation) of variance due to say "STC noise".
That's only because noise model we typically use in chess is just a crude (coin toss) approximation (just w/d/l and number of games) of real sources of noise.
Go ahead and run an experiment with 100 times 100 games match (of, in best case, 2 identical engines) at TC1 and 4xTC1 and plot Elo difference distribution (and calculate sigma of it) in both cases. You'd notice more difference in sigma than can be explained by just change of draw rate.

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Sergio Vieri second net is out

Post by Laskos » Sat Jul 25, 2020 7:01 am

Milos wrote:
Sat Jul 25, 2020 6:39 am
Laskos wrote:
Sat Jul 25, 2020 6:29 am
Milos wrote:
Sat Jul 25, 2020 6:15 am
Laskos wrote:
Sat Jul 25, 2020 4:40 am
MMarco wrote:
Sat Jul 25, 2020 3:11 am
Its getting scary!!

Posted by SVieri:

Code: Select all

I ran 2344 vs 2141 overnight. TC: 3m+2s, 92 threads.

Score of StockfishNNUE 2344 vs StockfishNNUE 20200722-2141: 14 - 5 - 33 [0.587]
...      StockfishNNUE 2344 playing White: 12 - 0 - 14  [0.731] 26
...      StockfishNNUE 2344 playing Black: 2 - 5 - 19  [0.442] 26
...      White vs Black: 17 - 2 - 33  [0.644] 52
Elo difference: 60.7 +/- 56.8, LOS: 98.1 %, DrawRatio: 63.5 %
52 of 100 games finished.
Too few games to say anything with high confidence. Not even clear that 2344 is stronger than 2141, cherry picked LOS of 98% doesn't qualify as a stopping rule. 99.9% or higher are needed when cherry picking to have some confidence in superiority, and even higher for small number of games.
There are indeed too few games, however, some things indicate that this is a bit more than just statistical noise. First, draw ratio is relatively high and second 3m+2s on 92 threads is equivalent to LTC on a single thread so games quality must be quite high and subsequently noise lower than your typical test games.
I don't understand the noise other than statistical noise. Statistically, LTC and STC are the same, and there is no any quantification (estimation) of variance due to say "STC noise".
That's only because noise model we typically use in chess is just a crude (coin toss) approximation (just w/d/l and number of games) of real sources of noise.
Go ahead and run an experiment with 100 times 100 games match (of, in best case, 2 identical engines) at TC1 and 4xTC1 and plot Elo difference distribution (and calculate sigma of it) in both cases. You'd notice more difference in sigma than can be explained by just change of draw rate.
The reason might be higher correlation between paired side-reversed games in LTC. which in pentanomial model is counted as lower variance. I do observe that in LTC with unbalanced opening positions (side-reversed) I am using, and in this case the difference between LTC and STC can indeed amount to some 10% in variance. But if one is using balanced openings, the difference in variances is much smaller, as the correlations of paired games are smaller in general. One cannot say in general "LTC decreases the statistical variance", as statistical variances are additive.

JohnS
Posts: 199
Joined: Sun Feb 24, 2008 1:08 am

Re: Sergio Vieri second net is out

Post by JohnS » Sat Jul 25, 2020 8:06 am

Another quick test of 2344 against H6.03 using Nunn1 openings, G10s+0.2s - result +11 =8 -1.

Here is a crazy sacrifice by SFnnue on move 13. Of course - usual disclaimers and as Mike would say yomv and ymmv.


carldaman
Posts: 2083
Joined: Sat Jun 02, 2012 12:13 am

Re: Sergio Vieri second net is out

Post by carldaman » Sat Jul 25, 2020 8:43 am

The games you posted are out of this world, JohnS. Fascinating stuff :)

User avatar
Rebel
Posts: 5819
Joined: Thu Aug 18, 2011 10:04 am

Re: Sergio Vieri second net is out

Post by Rebel » Sat Jul 25, 2020 9:56 am

MMarco wrote:
Fri Jul 24, 2020 1:41 pm
Can someone with good hardware test this one?

Size=256. By ribbit on discord.

"My first little 256 network.. I tested it on Honey-XI-NN with pretty good result against stockfish-dev and Leela ... (d24 validation used, 6menTB)

ribbit_0.1 - https://rapidu.net/9571752717/nn.bin "
This ribbit net is the same net as the much discussed 1134.
90% of coding is debugging, the other 10% is writing bugs.

JohnS
Posts: 199
Joined: Sun Feb 24, 2008 1:08 am

Re: Sergio Vieri second net is out

Post by JohnS » Sat Jul 25, 2020 11:26 am

The wins continue for 2344 - this time against Ethereal +13 =7 -0.

This was a nice game against the French with a nice rook sacrifice.


Post Reply