Stockfish NNUE SV Tests

[Modern Times]

Stockfish 14.1 vs Stockfish 14: (Fischer Random Chess)

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 14.1 x64 bmi2 : 2423 15 15 600 56.7 % 2377 69.0 %
2 Stockfish 14 x64 bmi2 : 2377 15 15 600 43.3 % 2423 69.0 %

That's impressive !

[mehmet123]

Stockfish PB 14.1 vs Stockfish 14.1:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish PB 14.1 x64 bmi2 : 2403 6 6 2000 50.8 % 2397 82.3 %
2 Stockfish 14.1 x64 bmi2 : 2397 6 6 2000 49.2 % 2403 82.3 %

Individual statistics:

1 Stockfish PB 14.1 x64 bmi2: 2403 2000 (+193,=1647,-160), 50.8 %

Stockfish 14.1 x64 bmi2 : 2000 (+193,=1647,-160), 50.8 %

2 Stockfish 14.1 x64 bmi2 : 2397 2000 (+160,=1647,-193), 49.2 %

Stockfish PB 14.1 x64 bmi2 : 2000 (+160,=1647,-193), 49.2 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, SuperGM 4 Moves Opening Book, 64 Mb Hash, Ponder Off
Stockfish PB 14.1 x64 E bmi2: " ChessMan" compile / Stockfish 14.1 x64 bmi2: Official Version
https://www.mediafire.com/file/wbyy6zgs ... 4.pgn/file

[mehmet123]

Kayra 1.0 vs Stockfish 081121:

Program Elo + - Games Score Av.Op. Draws

1 Kayra 1.0 v3 x64 bmi2 : 2402 12 11 200 50.5 % 2398 94.0 %
2 Stockfish 081121 x64 bmi2 : 2398 11 12 200 49.5 % 2402 94.0 %

Individual statistics:

1 Kayra 1.0 v3 x64 bmi2 : 2402 200 (+ 7,=188,- 5), 50.5 %

Stockfish 081121 x64 bmi2 : 200 (+ 7,=188,- 5), 50.5 %

2 Stockfish 081121 x64 bmi2 : 2398 200 (+ 5,=188,- 7), 49.5 %

Kayra 1.0 v3 x64 bmi2 : 200 (+ 5,=188,- 7), 49.5 %


Game Conditions: Arena Gui, 6 Cores (Core-i7 9750h), 1 min + 0.5 sec TC, SuperGM 4 Moves Opening Book, 1024 Mb Hash, Ponder Off
https://www.mediafire.com/file/ap5vokbt ... 5.pgn/file


Kayra chess engine is a Stockfish based engine and it's an early development version. I have been working on this chess engine for a short time.
Kayra is not an another Stockfish clone. I made very serious changes in the most critical parts of Stockfish, especially in the evaluation section.
I made very agressive changes more than 50 parameters in the evalution code. I made very aggressive changes at psqt and pawns sections. I changed more than 250 parameters at psqt and pawns section. I weakened Stockfish a lot at the start of all these critical changes, but I'm finally getting close to balancing it.
Although I made very critical changes in Stockfish codes and more than 300 parameters, the fact that Kayra didn' t perform badly gave me hope for the future. This version of Kayra is an early test version. If it reaches the level I want, there is no doubt that I' ll publish Kayra as open code source chess engine.

[mehmet123]

Kayra vs SugaR:

Program Elo + - Games Score Av.Op. Draws

1 Kayra 1.0 v4 bm2 : 2401 5 4 350 50.3 % 2399 98.3 %
2 SugaR AI 2.50 bmi2 : 2399 4 5 350 49.7 % 2401 98.3 %

Individual statistics:

1 Kayra 1.0 v4 bm2 : 2401 350 (+ 4,=344,- 2), 50.3 %

SugaR AI 2.50 bmi2 : 350 (+ 4,=344,- 2), 50.3 %

2 SugaR AI 2.50 bmi2 : 2399 350 (+ 2,=344,- 4), 49.7 %

Kayra 1.0 v4 bm2 : 350 (+ 2,=344,- 4), 49.7 %


Game Conditions: Fritz 17 Gui, 6 Cores (Core-i7 9750h), 1 min + 0. 5 sec TC, Balsa 5 Moves Opening Book, 1024 Mb Hash, Ponder Off
Learning is off for SugaR AI 2.50:


SugaR AI 2.50 is a very strong chess engine. This engine managed to beat the previous version of Kayra with a 96-95 score at same game conditions match. (SugaR AI 2.50 bmi2 – Kayra 1.0 v3b bmi2: 191 (+ 2,=188,- 1) %50.3 ).
After this defeat, I made some changes in the evaluation codes. These changes seem to have been positive for Kayra.


I've been developing Kayra for a short time. First of all I'm working on a chess engine with a different style than Stockfish. I made significant changes in critical codes and serious changes in more than 300 parameters. In fact, I deleted most of these parameters and rewrote them according to a plan so that vast majority of these parameters do not resemble Stockfish default parameters.
I have never had a goal of receiving donations. I'm still at the very beginning of the road becasue I have been developing Kayra only 3 weeks.
My plan is to sent this engine to some chess engine testers first, then to people who have made serious contributions to the forum and finally make it public with the source code.

[mehmet123]

Kayra vs SugaR:

Program Elo + - Games Score Av.Op. Draws

1 Kayra 1.0 v5 bmi2 : 2403 8 4 240 50.8 % 2397 98.3 %
2 SugaR AI 2.50 bmi2 : 2397 4 8 240 49.2 % 2403 98.3 %


Individual statistics:

1 Kayra 1.0 v5 bmi2 : 2403 240 (+ 4,=236,- 0), 50.8 %

SugaR AI 2.50 bmi2 : 240 (+ 4,=236,- 0), 50.8 %

2 SugaR AI 2.50 bmi2 : 2397 240 (+ 0,=236,- 4), 49.2 %

Kayra 1.0 v5 bmi2 : 240 (+ 0,=236,- 4), 49.2 %


Game Conditions: Fritz 17 Gui, 6 Cores (Core-i7 9750h), 1 min + 0. 5 sec TC, Balsa 5 Moves Opening Book, 1024 Mb Hash, Ponder Off
Learning is off for SugaR AI 2.50: / Kayra played with old NNUE net ( nn-13406b1dcbe0)
https://www.mediafire.com/file/lpd5kt85 ... 7.pgn/file


This is the best result for Kayra against SugaR in my tests. There are many problems to be solved for Kayra, but I am hopeful in terms of the development process.

[mehmet123]

Kayra vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Kayra 1.0 v6 bmi2 : 2402 6 3 300 50.5 % 2398 99.0 %
2 Stockfish 131221 x64 bmi2 : 2398 3 6 300 49.5 % 2402 99.0 %

Individual statistics:

1 Kayra 1.0 v6 bmi2 : 2402 300 (+ 3,=297,- 0), 50.5 %

Stockfish 131221 x64 bmi2 : 300 (+ 3,=297,- 0), 50.5 %

2 Stockfish 131221 x64 bmi2 : 2398 300 (+ 0,=297,- 3), 49.5 %

Kayra 1.0 v6 bmi2 : 300 (+ 0,=297,- 3), 49.5 %


Game Conditions: Arena Gui, 6 Cores (Core-i7 9750h), 1 min + 0. 5 sec TC, Balsa 5 Moves Opening Book, 1024 Mb Hash, Ponder Off
https://www.mediafire.com/file/wa0uhvr6 ... 8.pgn/file


I made a lot of changes in the critical Stockfish codes. Kayra game parameters and Stockfish game parameters don’t show any similarity.

The parameters changed in the Evaluation section:256
The parameters changed in the PSQT section:416
The parameters changed in the Pawns section:12
Total parameter changes:684

Although a serious mistake in one parameters could cause a significant loss of elo, I managed to keep this engine in a certain balance despite 684 parameters changes. I couldn’ t see any changes in these parameters in other open source Stockfish derivative chess engines.
Of course ı appreciate these chess engines. These engines provide important contributions such as faster compilations, polyglot opening book support, live opening book support, offering different playing styles etc.

i am working a full time job and I' m working on a different project than chess. I don't have a lot of time to develop Kayra. One of my three laptops is broken and I use the other for work. I can only use a 6 core/12 threads laptop to develop Kayra. Therefore, it is very difficult to say that these parameters used in Kayra are well optimized and it is impossible with these limited opportunities. It takes thousands or even tens of thousands of games to even optimize only a parameter. I can’ t say Kayra is the most powerful chess engine today, but I believe that Kayra will indisputably be the most powerful chess engine when these 684 parameters (many more parameters in the future) are well optimized with the help of powerful systems.