Stockfish NNUE SV Tests

[mehmet123]

10 years change of chess engines :

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 310121 x64 bmi2 : 655 0 52 200 97.8 % 2100 4.5 %
2 Houdini 1.5a x64 : 0 52 0 200 2.2 % 2700 4.5 %


Individual statistics:

1 Stockfish 310121 x64 bmi2 : 655 200 (+191,= 9,- 0), 97.8 %

Houdini 1.5a x64 : 200 (+191,= 9,- 0), 97.8 %

2 Houdini 1.5a x64 : 0 200 (+ 0,= 9,-191), 2.2 %

Stockfish 310121 x64 bmi2 : 200 (+ 0,= 9,-191), 2.2 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/smw1sg4r ... 5.pgn/file

Houdini 1.5a x64 was the most powerful chess engine as of February 2011. Houdini 1.5a x64 won the TCEC Season1 (Houdini 1.5a - Rybka 4:23.5 -16.5) and TCEC Season 2 ( (Houdini 1.5a - Rybka 4.1 :22 - 18)

[mehmet123]

Fat Fritz 2.0 vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 160221 x64 bmi2 : 2412 8 8 6000 53.4 % 2388 11.2 %
2 Fat Fritz 2.0 x64 bmi2 : 2388 8 8 6000 46.6 % 2412 11.2 %

Individual statistics:

1 Stockfish 160221 x64 bmi2 : 2412 6000 (+2868,=672,-2460), 53.4 %

Fat Fritz 2.0 x64 bmi2 : 6000 (+2868,=672,-2460), 53.4 %

2 Fat Fritz 2.0 x64 bmi2 : 2388 6000 (+2460,=672,-2868), 46.6 %

Stockfish 160221 x64 bmi2 : 6000 (+2460,=672,-2868), 46.6 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 kn/s , Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/xk1604ng ... 6.pgn/file

[mehmet123]

Fat Fritz 2.0 vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 160221 x64 bmi2 : 2417 7 7 6000 54.8 % 2383 31.2 %
2 Fat Fritz 2.0 x64 bmi2 : 2383 7 7 6000 45.2 % 2417 31.2 %

Individual statistics:

1 Stockfish 160221 x64 bmi2 : 2417 6000 (+2352,=1872,-1776), 54.8 %

Fat Fritz 2.0 x64 bmi2 : 6000 (+2352,=1872,-1776), 54.8 %

2 Fat Fritz 2.0 x64 bmi2 : 2383 6000 (+1776,=1872,-2352), 45.2 %

Stockfish 160221 x64 bmi2 : 6000 (+1776,=1872,-2352), 45.2 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 10 kn/s , Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/uz1u2gpr ... 7.pgn/file

[mehmet123]

Fat Fritz 2.0 vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 160221 x64 bmi2 : 2407 7 7 3200 52.1 % 2393 63.8 %
2 Fat Fritz 2.0 x64 bmi2 : 2393 7 7 3200 47.9 % 2407 63.8 %

Individual statistics:

1 Stockfish 160221 x64 bmi2 : 2407 3200 (+646,=2041,-513), 52.1 %

Fat Fritz 2.0 x64 bmi2 : 3200 (+646,=2041,-513), 52.1 %

2 Fat Fritz 2.0 x64 bmi2 : 2393 3200 (+513,=2041,-646), 47.9 %

Stockfish 160221 x64 bmi2 : 3200 (+513,=2041,-646), 47.9 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 100 kn/s , Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/q2at6zch ... 8.pgn/file

[mehmet123]

Default Net vs. The White Rose Net:

Program Elo + - Games Score Av.Op. Draws

1 Eman 6.92 Default : 2405 12 12 300 51.3 % 2395 90.7 %
2 Eman 6.92 TWR : 2395 12 12 300 48.7 % 2405 90.7 %

Individual statistics:

1 Eman 6.92 Default : 2405 300 (+ 18,=272,- 10), 51.3 %

Eman 6.92 TWR : 300 (+ 18,=272,- 10), 51.3 %

2 Eman 6.92 TWR : 2395 300 (+ 10,=272,- 18), 48.7 %

Eman 6.92 Default : 300 (+ 10,=272,- 18), 48.7 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Move Opening Book, 512 Mb Hash, Ponder Off
https://www.mediafire.com/file/sw3iusdm ... 9.pgn/file

[mehmet123]

Default Net vs. The White Rose Net:

Program Elo + - Games Score Av.Op. Draws

1 Eman 6.92 Default : 2403 8 7 360 51.0 % 2397 95.3 %
2 Eman 6.92 TWR : 2397 7 8 360 49.0 % 2403 95.3 %

Individual statistics:

1 Eman 6.92 Default : 2403 360 (+ 12,=343,- 5), 51.0 %

Eman 6.92 TWR : 360 (+ 12,=343,- 5), 51.0 %

2 Eman 6.92 TWR : 2397 360 (+ 5,=343,- 12), 49.0 %

Eman 6.92 Default : 360 (+ 5,=343,- 12), 49.0 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 10 minute TC, Balsa 5 Move Opening Book, 512 Mb Hash, Ponder Off
https://www.mediafire.com/file/j9jnrvrf ... 0.pgn/file

Elo difference between Default Net and The White Rose Net:
30 sec + 0.5 sec TC (360 games) + 20 elo
1 min + 0.5 sec TC (180 games) + 14 elo
2 min + 0.5 sec TC (300 game) + 10 elo
10 minutes TC (360 games) + 6 elo

The White Rose Net scales better with increasing time according to Default Net according to my tests.

[mehmet123]

Fire vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Fire 8 x64 pext : 2410 18 18 500 53.0 % 2390 64.0 %
2 Stockfish 8 x64 bmi2 : 2390 18 18 500 47.0 % 2410 64.0 %

Individual statistics:

1 Fire 8 x64 pext : 2410 500 (+105,=320,- 75), 53.0 %

Stockfish 8 x64 bmi2 : 500 (+105,=320,- 75), 53.0 %

2 Stockfish 8 x64 bmi2 : 2390 500 (+ 75,=320,-105), 47.0 %

Fire 8 x64 pext : 500 (+ 75,=320,-105), 47.0 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/96f4x5ky ... 4.pgn/file

Other Fire 8 Tests:

Fire 8 vs Fire 7.1:

Program Elo + - Games Score Av.Op. Draws

1 Fire 8 x64 pext : 2471 21 21 540 69.3 % 2329 48.5 %
2 Fire 7.1 x64 popcnt : 2329 21 21 540 30.7 % 2471 48.5 %

Individual statistics:

1 Fire 8 x64 pext : 2471 540 (+243,=262,- 35), 69.3 %

Fire 7.1 x64 popcnt : 540 (+243,=262,- 35), 69.3 %

2 Fire 7.1 x64 popcnt : 2329 540 (+ 35,=262,-243), 30.7 %

Fire 8 x64 pext : 540 (+ 35,=262,-243), 30.7 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/zmcnwhvl ... 2.pgn/file


Fire vs Ethereal:

Program Elo + - Games Score Av.Op. Draws

1 Fire 8 x64 pext : 2425 20 20 500 57.1 % 2375 55.8 %
2 Ethereal 12.75 x64 pext : 2375 20 20 500 42.9 % 2425 55.8 %

Individual statistics:

1 Fire 8 x64 pext : 2425 500 (+146,=279,- 75), 57.1 %

Ethereal 12.75 x64 pext : 500 (+146,=279,- 75), 57.1 %

2 Ethereal 12.75 x64 pext : 2375 500 (+ 75,=279,-146), 42.9 %

Fire 8 x64 pext : 500 (+ 75,=279,-146), 42.9 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/i63bs6fv ... 3.pgn/file

[mehmet123]

Stockfish Dev vs Stockfish 13 (Fischer Random Chess)

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 190321 x64 bmi2 : 2403 12 12 800 50.9 % 2397 73.6 %
2 Stockfish 13 x64 bmi2 : 2397 12 12 800 49.1 % 2403 73.6 %

Individual statistics:

1 Stockfish 190321 x64 bmi2 : 2403 800 (+113,=589,- 98), 50.9 %

Stockfish 13 x64 bmi2 : 800 (+113,=589,- 98), 50.9 %

2 Stockfish 13 x64 bmi2 : 2397 800 (+ 98,=589,-113), 49.1 %

Stockfish 190321 x64 bmi2 : 800 (+ 98,=589,-113), 49.1 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min TC, Chess960 Book 3 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/drbjcfg4 ... 5.pgn/file

[mehmet123]

Stockfish Dev vs Stockfish 13 (Fischer Random Chess)

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 190321 x64 bmi2 : 2427 13 13 800 57.8 % 2373 68.4 %
2 Stockfish 13 x64 bmi2 : 2373 13 13 800 42.2 % 2427 68.4 %

Individual statistics:


1 Stockfish 190321 x64 bmi2 : 2427 800 (+189,=547,- 64), 57.8 %

Stockfish 13 x64 bmi2 : 800 (+189,=547,- 64), 57.8 %

2 Stockfish 13 x64 bmi2 : 2373 800 (+ 64,=547,-189), 42.2 %

Stockfish 190321 x64 bmi2 : 800 (+ 64,=547,-189), 42.2 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min TC, No Opening Book , 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/qvnsbvpg ... 7.pgn/file

There is a serious elo difference between two chess engines (+54 elo) in the test where an opening book is not used.

[mehmet123]

Stockfish Dev vs Stockfish 13 (Fischer Random Chess)

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 190321 x64 bmi2 : 2401 12 12 1000 50.4 % 2399 68.0 %
2 Stockfish 13 x64 bmi2 : 2399 12 12 1000 49.6 % 2401 68.0 %

Individual statistics:

1 Stockfish 190321 x64 bmi2 : 2401 1000 (+164,=680,-156), 50.4 %

Stockfish 13 x64 bmi2 : 1000 (+164,=680,-156), 50.4 %

2 Stockfish 13 x64 bmi2 : 2399 1000 (+156,=680,-164), 49.6 %

Stockfish 190321 x64 bmi2 : 1000 (+156,=680,-164), 49.6 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Chess960 Book 3 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/8ao33y0l ... 8.pgn/file

The performance of Stockfish 190321 isn't very good (+2 elo) with opening book at FRC in this test.