Stockfish NNUE SV Tests

[mehmet123]

CiChess vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 CiChess 021220 x64 bmi2 : 2401 8 8 1000 50.2 % 2399 85.2 %
2 Stockfish 051220 x64 bmi2 : 2399 8 8 1000 49.8 % 2401 85.2 %

Individual statistics:

1 CiChess 021220 x64 bmi2 : 2401 1000 (+ 76,=852,- 72), 50.2 %

Stockfish 051220 x64 bmi2 : 1000 (+ 76,=852,- 72), 50.2 %

2 Stockfish 051220 x64 bmi2 : 2399 1000 (+ 72,=852,- 76), 49.8 %

CiChess 021220 x64 bmi2 : 1000 (+ 72,=852,- 76), 49.8 %

Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
Cichess version:CiChess 021220 x64 E BMI2 (By Chessman based on Cfish and CorChess)\\ Both engines played by Default Nets.
http://www.mediafire.com/file/vxaxf652u ... 1.pgn/file

[mehmet123]

Eman vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 051220 x64 bmi2 : 2401 8 8 1200 50.4 % 2399 83.6 %
2 Eman 6.61 x64 bmi2 : 2399 8 8 1200 49.6 % 2401 83.6 %

Individual statistics:

1 Stockfish 051220 x64 bmi2 : 2401 1200 (+103,=1003,- 94), 50.4 %

Eman 6.61 x64 bmi2 : 1200 (+103,=1003,- 94), 50.4 %

2 Eman 6.61 x64 bmi2 : 2399 1200 (+ 94,=1003,-103), 49.6 %

Stockfish 051220 x64 bmi2 : 1200 (+ 94,=1003,-103), 49.6 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off//Default Net
https://www.mediafire.com/file/0zb34z42 ... 6.pgn/file

[mehmet123]

Eman vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Eman 6.61 Exp x64 bmi2 : 2405 7 7 1250 51.5 % 2395 85.0 %
2 Stockfish 051220 x64 bmi2 : 2395 7 7 1250 48.5 % 2405 85.0 %

Individual statistics:

1 Eman 6.61 Exp x64 bmi2 : 2405 1250 (+112,=1063,- 75), 51.5 %

Stockfish 051220 x64 bmi2 : 1250 (+112,=1063,- 75), 51.5 %

2 Stockfish 051220 x64 bmi2 : 2395 1250 (+ 75,=1063,-112), 48.5 %

Eman 6.61 Exp x64 bmi2 : 1250 (+ 75,=1063,-112), 48.5 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off//Default Net
Eman played with experience file
http://www.mediafire.com/file/5cm55wp8i ... 7.pgn/file

With the help of the experience file, Eman becomes a candidate to be the world's most powerful chess engine.

[mehmet123]

Eman vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Eman 6.61 Exp1 x64 bmi2 : 2404 9 9 1000 51.1 % 2396 82.4 %
2 Stockfish 051220 x64 bmi2 : 2396 9 9 1000 48.9 % 2404 82.4 %

Individual statistics:

1 Eman 6.61 Exp1 x64 bmi2 : 2404 1000 (+ 99,=824,- 77), 51.1 %

Stockfish 051220 x64 bmi2 : 1000 (+ 99,=824,- 77), 51.1 %

2 Stockfish 051220 x64 bmi2 : 2396 1000 (+ 77,=824,- 99), 48.9 %

Eman 6.61 Exp1 x64 bmi2 : 1000 (+ 77,=824,- 99), 48.9 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off//Default Net
Eman played with experience file
http://www.mediafire.com/file/jduq4rark ... 2.pgn/file

I have been running an exp file since yesterday. At my first test I got a better result (+8 elo ) against Stockfish Dev. than I expected and training is continue. At previous test the exp file used by Eman was run by Khalid Omar.

[mehmet123]

Sugar AI vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 140220 x64 bmi2 : 2410 12 12 600 52.9 % 2390 81.8 %
2 SugaR AI 1.00 bmi2 : 2390 12 12 600 47.1 % 2410 81.8 %


Individual statistics:

1 Stockfish 140220 x64 bmi2 : 2410 600 (+ 72,=491,- 37), 52.9 %

SugaR AI 1.00 bmi2 : 600 (+ 72,=491,- 37), 52.9 %

2 SugaR AI 1.00 bmi2 : 2390 600 (+ 37,=491,- 72), 47.1 %

Stockfish 140220 x64 bmi2 : 600 (+ 37,=491,- 72), 47.1 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off//Default Net
https://www.mediafire.com/file/ffkcitut ... 0.pgn/file

The performance of SugaR AI is close to ( -20 elo) latest Stockfish Dev. Very good performance for the first version of a chess engine. I guess that Sugar AI will perform much better in the near future.

[mehmet123]

Sugar AI vs. Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 SugaR AI 1.00 mi2 Exp2 : 2400 10 10 500 50.0 % 2400 89.6 %
2 Stockfish 140220 x64 bmi2 : 2400 10 10 500 50.0 % 2400 89.6 %

Individual statistics:

1 SugaR AI 1.00 mi2 Exp2 : 2400 500 (+ 26,=448,- 26), 50.0 %

Stockfish 140220 x64 bmi2 : 500 (+ 26,=448,- 26), 50.0 %

2 Stockfish 140220 x64 bmi2 : 2400 500 (+ 26,=448,- 26), 50.0 %

SugaR AI 1.00 mi2 Exp2 : 500 (+ 26,=448,- 26), 50.0 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off//Default Net
Sugar AI used a different experience file instead of default experience file.
https://www.mediafire.com/file/ctkv21zy ... 2.pgn/file

[mehmet123]

Let's make a correction. In my tests I used latest Stockfish Dev but I wrongly named the version as 140220.
The correct name of the engine used in my tests is Stockfish 141220.

[mehmet123]

Sugar AI vs. Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 SugaR AI 1.00 mi2 Exp2 : 2402 9 8 400 50.6 % 2398 93.8 %
2 Stockfish 141220 x64 bmi2 : 2398 8 9 400 49.4 % 2402 93.8 %

Individual statistics:

1 SugaR AI 1.00 mi2 Exp2 : 2402 400 (+ 15,=375,- 10), 50.6 %

Stockfish 141220 x64 bmi2 : 400 (+ 15,=375,- 10), 50.6 %

2 Stockfish 141220 x64 bmi2 : 2398 400 (+ 10,=375,- 15), 49.4 %

SugaR AI 1.00 mi2 Exp2 : 400 (+ 10,=375,- 15), 49.4 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off//Default Net
https://www.mediafire.com/file/g7geeqe9 ... 3.pgn/file

[mehmet123]

Sugar AI vs Cfish:

Program Elo + - Games Score Av.Op. Draws

1 SugaR AI 1.00 mi2 Exp2 : 2403 8 8 600 50.9 % 2397 91.8 %
2 Cfish 131220 x64 bmi2 : 2397 8 8 600 49.1 % 2403 91.8 %

Individual statistics:

1 SugaR AI 1.00 mi2 Exp2 : 2403 600 (+ 30,=551,- 19), 50.9 %

Cfish 131220 x64 bmi2 : 600 (+ 30,=551,- 19), 50.9 %

2 Cfish 131220 x64 bmi2 : 2397 600 (+ 19,=551,- 30), 49.1 %

SugaR AI 1.00 mi2 Exp2 : 600 (+ 19,=551,- 30), 49.1 %

Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off//Default Net
Cfish version:Cfish 131220 x64 E BMI2 mingw10 (ChessMan version)
https://www.mediafire.com/file/1f53hbsb ... 4.pgn/file

After a long time, a chess engine managed to beat Cfish in my tests. Sugar AI has a real great potential. Marco Zerbinati has done a great job.

[mehmet123]

Sugar AI vs Cfish:

Program Elo + - Games Score Av.Op. Draws

1 SugaR AI 1.00 x64 bmi2 v16 : 2400 8 8 320 50.0 % 2400 95.6 %
2 Cfish 131220 x64 bmi2 : 2400 8 8 320 50.0 % 2400 95.6 %

Individual statistics:

1 SugaR AI 1.00 x64 bmi2 v16: 2400 320 (+ 7,=306,- 7), 50.0 %

Cfish 131220 x64 bmi2 : 320 (+ 7,=306,- 7), 50.0 %

2 Cfish 131220 x64 bmi2 : 2400 320 (+ 7,=306,- 7), 50.0 %

SugaR AI 1.00 x64 bmi2 v16 : 320 (+ 7,=306,- 7), 50.0 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off//Default Net
Cfish version:Cfish 131220 x64 E BMI2 mingw10 (ChessMan version)
https://www.mediafire.com/file/phz4bvdd ... 5.pgn/file

This is a different experience file than Exp3 file. They are basically close files but there are some differences in game database and settings. Minimum experience depth is 16 for this file. For Exp3 file the game database is smaller than V16 file and minimum experience depth is 18.
Too many games in experience file does not mean the file will be stronger. Sometimes ineffective games weaken the experience file.