SFNNue test, 3m + 1s

[MMarco]

I'm missing a few games in Stockfish Dev gauntlet because I'm going away, now but here is what I got:

Code: Select all

   # PLAYER                  :  RATING  ERROR  PLAYED   (%)   CFS    W    D    L  D(%)
   1 SFNNue + FiNN 0.2       :    3516     14     878  61.2    53  264  547   67  62.3
   2 SFNNue + GK 27-06       :    3515     13     876  61.1    84  261  549   66  62.7
   3 Stockfish Dev 110720    :    3506     13     632  49.2    58   86  450   96  71.2
   4 SFNNue + GK 12-07       :    3504     12     876  59.7    94  228  590   58  67.4
   5 Stockfish 11            :    3491     12     666  47.1   100   59  509   98  76.4
   6 Houdidit 6.03           :    3372     14     666  31.3    77   28  361  277  54.2
   7 Komodo 14               :    3363     15     666  30.2   ---   18  366  282  55.0

White advantage = 43.85 +/- 3.58
Draw rate (equal opponents) = 74.56 % +/- 1.07

Code: Select all

3) Stockfish Dev 110720 3506 :    632 (+86,=450,-96),  49.2 %

   vs.                        :  games (  +,   =,  -),   (%) :   Diff,   SD, CFS (%)
   SFNNue + FiNN 0.2          :    212 ( 27, 150, 35),  48.1 :    -10,   11,   18.2
   SFNNue + GK 27-06          :    210 ( 32, 139, 39),  48.3 :     -9,    9,   15.9
   SFNNue + GK 12-07          :    210 ( 27, 161, 22),  51.2 :     +2,    9,   58.5

Code: Select all

5) Stockfish 11           24 :    666 (+59,=509,-98),  47.1 %

   vs.                        :  games (  +,   =,  -),   (%) :   Diff,   SD, CFS (%)
   SFNNue + FiNN 0.2          :    222 ( 20, 167, 35),  46.6 :    -25,   10,    0.6
   SFNNue + GK 27-06          :    222 ( 19, 172, 31),  47.3 :    -24,    9,    0.3
   SFNNue + GK 12-07          :    222 ( 20, 170, 32),  47.3 :    -13,    9,    6.3

[MMarco]

For the sake of completeness:

Code: Select all

   # PLAYER                  :  RATING  ERROR  PLAYED   (%)   CFS    W    D    L  D(%)
   1 SFNNue + FiNN 0.2       :    3516     12     888  61.0    52  266  552   70  62.2
   2 SFNNue + GK 27-06       :    3515     12     888  61.0    83  264  555   69  62.5
   3 Stockfish Dev 110720    :    3507     13     666  49.2    58   92  472  102  70.9
   4 SFNNue + GK 12-07       :    3505     11     888  59.6    94  229  601   58  67.7
   5 Stockfish 11            :    3491     13     666  47.1   100   59  509   98  76.4
   6 Houdidit 6.03           :    3372     16     666  31.3    78   28  361  277  54.2
   7 Komodo 14               :    3363     15     666  30.2   ---   18  366  282  55.0

White advantage = 44.10 +/- 3.43
Draw rate (equal opponents) = 74.43 % +/- 0.93

Many find larger elo gap between SFNNue and Stckfish Dev (or SF-11). I think my processor isn't the best around (I get 43% of Stockfish speed with the 1507 binaries, while others get up to 60%), and the opening suite is not a "low draw" set, which tends to spread out the results.

The new binaries seems to yield about 10 extra elo:

Code: Select all

   # PLAYER                    :  RATING  ERROR  PLAYED   (%)   CFS    W    D    L  D(%)
   1 SFNNue 1907 + FiNN 0.2    :    3528     19     222  52.9    70   38  159   25  71.6
   2 SFNNue 1507 + FiNN 0.2    :    3518     19     222  51.6    81   37  155   30  69.8
   3 Stockfish Dev 110720      :    3507     12     444  47.7   ---   55  314   75  70.7

White advantage = 49.38 +/- 8.01
Draw rate (equal opponents) = 73.61 % +/- 2.00

Games: https://gofile.io/d/zlRKC7