Tournament SF+NNUE, Lc0, Ethereal, SF+Classical Eval, Komodo, Stockfish+NNUE PO, (15m+15s) (8 Rounds)

[MMarco]

Looks like with this drawish opening book, even Ethereal is really hard to beat.

And less than 100 elo performance gap from top to bottom in this run so far. Small sample size, but it still tells us something.

What did this same drawish book tell you at 3m+2s?

I will show you, same "drawish opening book", same settings, only the time control has changed. 3m+2s vs here 15m+15s.

Code: Select all

Result:
----------------------------------------------------------------------------------------
  #  name                        games    wins   draws  losses   score    los%  elo+/-
  1. SF+NNUE PO 290720 x64 avx2     60      18      42       0    39.0   100.0   107.5
  2. Stockfish 260820               60      10      50       0    35.0    99.9    58.5
  3. Stockfish 240820               60       8      42      10    29.0    31.9   -11.6
  4. Lc0 v0.26.1                    60       6      51       3    31.5    84.1    17.4
  5. Komodo 14 64-bit               60       4      42      14    25.0     0.9   -58.5
  6. Ethereal 12.25 (POPCNT)        60       0      41      19    20.5     0.0  -113.9

Does SF NNUE "PO" stand for Ponder On? On 32 threads?

No.

sf-nnue po 290720
@FireFather FireFather released this on Jul 28

stockfish+nnue po (play only)

The download includes fast optimized PGO bmi2, avx2, & popc (sse42) binaries

https://github.com/FireFather/sf-nnue/releases

OK! That makes more sense!

Interesting because this is a Pure NNue compile, created before the hybridisation.

[mwyoung]

Looks like with this drawish opening book, even Ethereal is really hard to beat.

And less than 100 elo performance gap from top to bottom in this run so far. Small sample size, but it still tells us something.

What did this same drawish book tell you at 3m+2s?

I will show you, same "drawish opening book", same settings, only the time control has changed. 3m+2s vs here 15m+15s.

Code: Select all

Result:
----------------------------------------------------------------------------------------
  #  name                        games    wins   draws  losses   score    los%  elo+/-
  1. SF+NNUE PO 290720 x64 avx2     60      18      42       0    39.0   100.0   107.5
  2. Stockfish 260820               60      10      50       0    35.0    99.9    58.5
  3. Stockfish 240820               60       8      42      10    29.0    31.9   -11.6
  4. Lc0 v0.26.1                    60       6      51       3    31.5    84.1    17.4
  5. Komodo 14 64-bit               60       4      42      14    25.0     0.9   -58.5
  6. Ethereal 12.25 (POPCNT)        60       0      41      19    20.5     0.0  -113.9

Does SF NNUE "PO" stand for Ponder On? On 32 threads?

No.

sf-nnue po 290720
@FireFather FireFather released this on Jul 28

stockfish+nnue po (play only)

The download includes fast optimized PGO bmi2, avx2, & popc (sse42) binaries

https://github.com/FireFather/sf-nnue/releases

OK! That makes more sense!

Interesting because this is a Pure NNue compile, created before the hybridisation.

Yes, that is why it was being tested and hybrid Stockfish NNUE.

[mwyoung]

End of tournament.

Code: Select all

Result:
----------------------------------------------------------------------------------------
  #  name                        games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 260820               40       5      35       0    22.5    98.7    43.7
  2. SF+NNUE PO 290720 x64 avx2     40       4      36       0    22.0    97.7    34.9
  3. Lc0 v0.26.1                    40       2      38       0    21.0    92.1    17.4
  4. Stockfish 240820               40       1      36       3    19.0    15.9   -17.4
  5. Komodo 14 64-bit               40       1      35       4    18.5     9.0   -26.1
  6. Ethereal 12.25 (POPCNT)        40       0      34       6    17.0     0.7   -52.5

Cross table:
----------------------------------------------------------------------------------------
  #  name                           score   games         1         2         3         4         5         6
  1. Stockfish 260820                22.5      40         x  ========  ========  =======1  1=======  1=1===1=
  2. SF+NNUE PO 290720 x64 avx2      22.0      40  ========         x  ========  ======1=  ===11===  =1======
  3. Lc0 v0.26.1                     21.0      40  ========  ========         x  ==1=====  =====1==  ========
  4. Stockfish 240820                19.0      40  =======0  ======0=  ==0=====         x  ========  ======1=
  5. Komodo 14 64-bit                18.5      40  0=======  ===00===  =====0==  ========         x  =====1==
  6. Ethereal 12.25 (POPCNT)         17.0      40  0=0===0=  =0======  ========  ======0=  =====0==         x

Tech:
----------------------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name                          nodes/m         NPS  depth/m   time/m    moves     time
  1. Stockfish 260820              636654K    22885705     48.4     27.8     61.1   1699.0
  2. SF+NNUE PO 290720 x64 avx2    611621K    22756666     51.8     26.9     64.1   1722.8
  3. Lc0 v0.26.1                      820K       29309     12.5     28.0     63.1   1766.4
  4. Stockfish 240820             1084516K    40535112     54.3     26.8     67.8   1815.3
  5. Komodo 14 64-bit             1062519K    38130495     45.0     27.9     62.1   1730.4
  6. Ethereal 12.25 (POPCNT)      1132625K    40887572     36.9     27.7     62.5   1730.6
     all ---                       739922K    27565962     41.6     27.5     63.5   1744.1