My test for Kayra 1.0 v6 bmi2

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

kranium
Posts: 2130
Joined: Thu May 29, 2008 10:43 am

Re: My test for Kayra 1.0 v6 bmi2

Post by kranium »

The results with NNUE=false (classical eval only)..I stopped early.

Code: Select all

1) Stockfish_21122211  3151 :   8444 (+8326,=83,-35),  99.1 %
   vs.                      :  games (    +,  =,  -),   (%) :   Diff,  SD, CFS (%)
   Kayra_1.0                :   8444 ( 8326, 83, 35),  99.1 :   +823,  13,  100.0

2) Kayra_1.0           2329 :   8444 (+35,=83,-8326),   0.9 %
   vs.                      :  games (  +,  =,    -),   (%) :   Diff,  SD, CFS (%)
   Stockfish_21122211       :   8444 ( 35, 83, 8326),   0.9 :   -823,  13,    0.0

   # PLAYER                : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish_21122211    : 3151.5   14.5   8367.5    8444   99.1%
   2 Kayra_1.0             : 2328.5   14.5     76.5    8444    0.9%
Stockfish is so highly tuned that simply changing even 1 eval parameter is likely to fail.
This clearly is the result of changing hundreds of them without testing each and every one.

It's quite surprising to me that so many changes to classic eval has affected NNUE eval so minimally.
I guess it demonstrates just how infrequent the SF uses the classic eval when NNUE is enabled.

Anyway, it doesn't hurt to try Mehmet!
That's what open-source is for...
Last edited by kranium on Thu Dec 23, 2021 8:24 pm, edited 2 times in total.
Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: My test for Kayra 1.0 v6 bmi2

Post by Guenther »

mehmet123 wrote: Thu Dec 23, 2021 7:57 pm
Guenther wrote: Thu Dec 23, 2021 7:46 pm I. guess the 'author' didn't know how to git clone from a certain timestamp/revision so he compiled just
his psqt changes (which have near zero effect due to nnue, but this was already told so often to no avail) compiled just his psqt changes (which have near zero effect due to nnue, but this was already told so often to no avail)
I wrote before
"Psqt codes are very important codes. In the test I made yesterday by giving strange values to the psqt codes, this engine (Kayra X NNUE ) was defeated by Stockfish NNUE with a 100-0 score."


Differences between Stockfish and Kayra codes.
https://github.com/Vizvezdenec/Stockfis ... baadf0a077

But this list is incomplete. They forgot to add some changes.
Instead of 'have near zero effect' I should have written have zero positive effect lol.
Ofc by hampering the eval values to absurdity you can reach that SF thinks it should change from NNUE to HCE.
But I doubt you get this. This is completely hilarious meanwhile.

I should quote you from outskirts where you confessed you have no clue what you are doing?
https://rwbc-chess.de

[Trolls n'existent pas...]
Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: My test for Kayra 1.0 v6 bmi2

Post by Guenther »

kranium wrote: Thu Dec 23, 2021 8:22 pm The results with NNUE=false (classical eval only)..I stopped early.

Code: Select all

1) Stockfish_21122211  3151 :   8444 (+8326,=83,-35),  99.1 %
   vs.                      :  games (    +,  =,  -),   (%) :   Diff,  SD, CFS (%)
   Kayra_1.0                :   8444 ( 8326, 83, 35),  99.1 :   +823,  13,  100.0

2) Kayra_1.0           2329 :   8444 (+35,=83,-8326),   0.9 %
   vs.                      :  games (  +,  =,    -),   (%) :   Diff,  SD, CFS (%)
   Stockfish_21122211       :   8444 ( 35, 83, 8326),   0.9 :   -823,  13,    0.0

   # PLAYER                : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish_21122211    : 3151.5   14.5   8367.5    8444   99.1%
   2 Kayra_1.0             : 2328.5   14.5     76.5    8444    0.9%
Stockfish is highly tuned that simply changing even 1 eval parameter id likely to fail.
This clearly the result of changing hundreds of them without testing each and every one.

It's quite surprising that some many changes to classic eval affected NNUE eval so minimally.
I guess it demonstrates just how infrequent the SF uses the classic eval when NNUE is enabled.
Well, you surely have seen that those 'changes' can only be seen as random work at all,
in case you looked at them...? SF test also called them random psqt changes 1/2/3.
https://rwbc-chess.de

[Trolls n'existent pas...]
mehmet123
Posts: 694
Joined: Sun Jan 26, 2020 10:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: My test for Kayra 1.0 v6 bmi2

Post by mehmet123 »

Guenther wrote: Thu Dec 23, 2021 8:23 pm
I should quote you from outskirts where you confessed you have no clue what you are doing?
I'm working on the Search part of Stockfish for Kayra 1.1 and be sure I'm doing much more ridiculous things.
it will be good for me to take a break from this forum for a while after the release of Kayra 1.1
mehmet123
Posts: 694
Joined: Sun Jan 26, 2020 10:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: My test for Kayra 1.0 v6 bmi2

Post by mehmet123 »

I will be very happy to release Kayra in a short time as open source. Stockfish developers have put a lot of effort into Stockfish and I don't want to be disrespectful to them.

I wrote in a Turkish Chess Forum under name of "Kaissa74" about chess computers. This forum wasn' t so active but my topic has been more than 600.000 views. In recent years there has been a very intense interest in chess computers especially among the young generation at Turkey. I didn't share any commercial program at this forum but I shared a few versions of the Raubfish. I was wrong to think that this engine will one day be released as open source. I admit that I made a mistake and stop the sharing of this non open source Stockfish derivative chess engine.
Eduard
Posts: 1439
Joined: Sat Oct 27, 2018 12:58 am
Location: Germany
Full name: N.N.

Re: My test for Kayra 1.0 v6 bmi2

Post by Eduard »

Unbeaten now for 692 games on playchess. :P

In the following game the opponent sees himself +1 in advantage and only creates a draw:

[pgn][Event "Rated game, 5 min"]
[Site "Engine Room"]
[Date "2021.12.23"]
[Round "?"]
[White "Blañk Queen, PidorDarko131221"]
[Black "SolistaChess, Kayra 1.0 v6 avx2"]
[Result "1/2-1/2"]
[ECO "B08"]
[WhiteElo "2731"]
[BlackElo "2733"]
[Annotator "0.91;0.52"]
[PlyCount "68"]
[EventDate "2021.12.23"]
[SourceTitle "playchess.com"]
[TimeControl "300"]

{PidorDarko131221 (4 threads): 27.3 plies; 2.372kN/s Intel(R) Core(TM) i7 CPU
X 980 @ 3.33GHz 3331MHz, (1 cores, 6 threads), Bigmaster.ctg, 4096 MB} 1. e4 {
B 0} d6 {B 0} 2. d4 {B 0} Nf6 {B 0} 3. Nc3 {B 0} g6 {B 0} 4. Be3 {B 0} Bg7 {B 0
} 5. Qd2 {B 0} a6 {B 0} 6. Nf3 {0.91/27 12} O-O {B 0} 7. O-O-O {1.03/25 3} b5 {
B 0} 8. Bd3 {0.91/24 0} c5 {B 0} 9. dxc5 {0.81/25 1} b4 {B 0} 10. Nd5 {0.68/27
2} Nxd5 {B 0} 11. exd5 {0.81/28 1} dxc5 {B 0} 12. Bxc5 {0.70/28 2} Qxd5 {B 0}
13. Qxb4 {0.69/30 1} Qxa2 {B 0} 14. Qa3 {0.60/27 1} Bh6+ {B 0} 15. Nd2 {
0.56/28 1} Qxa3 {B 0} 16. Bxa3 {0.59/27 1} Ra7 {B 0} 17. Rhe1 {0.58/30 5} Nc6 {
0.52/33 9} 18. Bc5 {0.65/28 2} Rc7 {0.38/36 6} 19. Bb6 {0.77/28 0} Rb7 {
0.51/37 2} 20. Be3 {0.62/27 3} Bg7 {0.47/38 0} 21. c3 {0.92/22 2} Ne5 {0.31/35
0} 22. Be4 {0.60/30 4} Bf5 {0.19/37 0} 23. Bxf5 {0.51/28 2} gxf5 {0.17/37 1}
24. Kc2 {0.52/30 0} Rc8 {0.18/36 5} 25. f4 {0.58/29 0 (Ra1)} Nc6 {0.19/35 4}
26. Bc5 {0.35/22 3} Rcc7 {0.00/34 0} 27. Nc4 {0.17/26 12 (g3)} Rb5 {0.00/41 3}
28. Ba3 {0.14/28 0} Nd4+ {0.00/43 3} 29. Kd3 {0.42/25 2 (cxd4)} Ne6 {0.00/48 5}
30. Ne3 {0.00/35 3} Nxf4+ {0.00/50 0} 31. Kc2 {0.00/34 1 SolistaChess,Kayra 1.
0 v6 avx2 offers a draw} h5 {0.00/51 2} 32. g3 {0.00/36 5 (Rf1)} Ng6 {0.00/43 4
} 33. Nd5 {0.00/40 0} Rcb7 {0.00/46 3} 34. Nxe7+ {0.00/43 0} Nxe7 {0.00/52 4}
1/2-1/2
[/pgn]

Or here, with 40 hreads (I play with 22) +0.84 and only a draw:

[pgn][Event "Rated game, 5 min"]
[Site "Engine Room"]
[Date "2021.12.23"]
[Round "?"]
[White "PurePower, Stockfish 14 x64 De"]
[Black "SolistaChess, Kayra 1.0 v6 avx2"]
[Result "1/2-1/2"]
[ECO "C11"]
[WhiteElo "2736"]
[BlackElo "2734"]
[Annotator "0.84;0.62"]
[PlyCount "145"]
[EventDate "2021.12.23"]
[SourceTitle "playchess.com"]
[TimeControl "300"]

{Stockfish 14 x64 Dev4 (40 threads): 46.5 plies; 43.916kN/s Intel(R) Xeon(R)
CPU E5-2673 v4 @ 2.30GHz 2295MHz, (20 cores, 40 threads), Bigmaster.ctg, 4096
MB} 1. e4 {B 0} e6 {B 0} 2. d4 {B 0} d5 {B 0} 3. Nc3 {B 0} Nf6 {B 0} 4. e5 {B 0
} Nfd7 {B 0} 5. f4 {B 0} c5 {B 0} 6. Nf3 {B 0} Nc6 {B 0} 7. Be3 {B 0} cxd4 {B 0
} 8. Nxd4 {B 0} a6 {B 0} 9. Qd2 {0.84/32 8} Bc5 {B 0} 10. a3 {B 0} Rb8 {B 0}
11. Qf2 {B 0} Nxd4 {B 0} 12. Bxd4 {0.53/34 10} Qb6 {B 0} 13. O-O-O {0.60/28 4}
Bxd4 {B 0} 14. Rxd4 {B 0} Qc5 {B 0} 15. Qd2 {B 0} O-O {0.62/34 33} 16. Bd3 {
0.62/28 6 (h4)} b5 {0.46/30 5} 17. Re1 {0.88/30 5} Kh8 {0.55/32 0} 18. Rb4 {
0.62/33 3} Nb6 {0.54/31 1} 19. Qe3 {0.70/34 3} Qxe3+ {0.75/34 5} 20. Rxe3 {
0.81/36 4} g6 {0.64/38 9} 21. Rd4 {0.80/38 3} Bd7 {0.67/34 0} 22. g4 {0.73/34 4
} Nc8 {0.60/36 1} 23. Bf1 {0.68/34 3} Rb7 {0.69/34 1} 24. Ne2 {0.75/36 18 (b3)}
Ne7 {0.62/31 3} 25. Rd1 {0.59/35 2 (b3)} b4 {0.71/37 11} 26. Nd4 {0.76/37 8} a5
{0.86/35 2} 27. axb4 {0.70/36 3} axb4 {0.65/33 0} 28. Be2 {0.68/35 11} Ra7 {
0.48/35 0} 29. h4 {0.64/31 4} Kg7 {0.55/33 0} 30. b3 {0.77/33 7 (Rb3)} Rc8 {
0.70/36 12} 31. Kd2 {0.73/37 3} Nc6 {0.66/36 0} 32. Rd3 {0.62/34 4} Ra2 {
0.80/38 4} 33. Rc1 {0.63/36 6} Na7 {0.62/38 1} 34. Rg3 {0.28/43 39 (Rf3)} Nc6 {
0.25/47 4} 35. Rd3 {0.42/38 0} Na7 {0.25/48 4} 36. Rf3 {0.29/43 3} Nc6 {
0.14/42 2} 37. Nxc6 {0.18/38 1} Bxc6 {0.21/43 3} 38. Rff1 {0.12/40 4} d4 {
0.20/43 0} 39. Ra1 {0.07/38 2} Rca8 {0.06/42 3} 40. Rxa2 {0.07/37 0} Rxa2 {
0.00/40 3} 41. Re1 {0.00/43 10 (Bd3)} Bd5 {0.12/46 5} 42. Kd3 {0.00/50 0 (Bc4)}
Bc6 {0.06/48 9} 43. Kd2 {0.10/50 0 (Bf1)} Bd5 {0.00/46 6} 44. Bc4 {0.00/51 0}
Bf3 {0.00/55 3} 45. g5 {0.00/54 0} h5 {0.00/61 3} 46. Kd3 {0.00/52 0 (Bd3)} Bg4
{0.00/60 4} 47. Rc1 {0.00/57 0} Bf5+ {0.00/61 3} 48. Kxd4 {0.00/58 0
SolistaChess,Kayra 1.0 v6 avx2 offers a draw} Bxc2 {0.00/62 5} 49. Kc5 {
0.00/58 0} Bf5 {0.00/64 2} 50. Kxb4 {0.00/57 0} Rh2 {0.00/63 4} 51. Kc5 {
0.00/61 0} Rxh4 {0.00/64 3} 52. b4 {0.00/62 0} Rxf4 {0.00/67 3} 53. b5 {
0.00/65 0} Be4 {0.00/65 2} 54. b6 {0.00/66 2} h4 {0.00/66 1} 55. Bb5 {0.00/66 1
} h3 {0.00/74 2} 56. Bc6 {0.00/73 0} Bxc6 {0.00/74 2} 57. Kxc6 {0.00/70 3} h2 {
0.00/80 0} 58. Kc5 {0.00/72 3 (Rh1)} Rg4 {0.00/77 3} 59. Rh1 {0.00/89 3} Rg2 {
0.00/81 7} 60. b7 {0.00/88 0} Rb2 {0.00/88 5} 61. Kc6 {0.00/88 0} Rc2+ {
0.00/94 3} 62. Kb5 {0.00/85 0 (Kb6)} Rb2+ {0.00/93 3} 63. Kc6 {0.00/90 6 (Ka6)}
Rc2+ {0.00/87 23} 64. Kb5 {0.00/108 0 (Kb6)} Rb2+ {0.00/83 3} 65. Ka6 {0.00/65
2} Ra2+ {0.00/98 0} 66. Kb6 {0.00/110 1} Rb2+ {0.00/97 0} 67. Kc7 {0.00/91 2}
Rc2+ {0.00/88 14} 68. Kb6 {0.00/98 0 (Kb8)} Rb2+ {0.00/92 1} 69. Kc7 {0.00/97
22} Rc2+ {0.00/108 0} 70. Kb8 {0.00/90 2} Rb2 {0.00/98 0} 71. Kc8 {0.00/98 3}
Rc2+ {0.00/97 12} 72. Kd7 {0.00/114 0} Rb2 {0.00/94 1} 73. Kc6 {0.00/245 7
(Kc8)} 1/2-1/2[/pgn]

Here +0.73 with 44 threads and only draw:

[pgn][Event "Rated game, 5 min"]
[Site "Engine Room"]
[Date "2021.12.24"]
[Round "?"]
[White "W32BlasteR, EMAN 7.80 64-bit AV"]
[Black "SolistaChess, Kayra 1.0 v6 avx2"]
[Result "1/2-1/2"]
[ECO "A93"]
[WhiteElo "2717"]
[BlackElo "2732"]
[Annotator "0.73;0.45"]
[PlyCount "68"]
[EventDate "2021.12.24"]
[SourceTitle "playchess.com"]
[TimeControl "300"]

{EMAN 7.80 64-bit AVX2 (44 threads): 46.1 plies; 19.912kN/s Intel(R) Xeon(R)
CPU E5-2690 v3 @ 2.60GHz 2594MHz, (24 cores, 48 threads), Bigmaster.ctg, 4096
MB} 1. d4 {B 0} f5 {B 0} 2. g3 {0.01/0 2} e6 {B 0} 3. Nf3 {0.73/33 23} Nf6 {
B 0 (Ae7)} 4. c4 {0.01/0 1} Be7 {B 0} 5. Bg2 {0.01/0 1} O-O {B 0} 6. O-O {
0.01/0 1} d5 {B 0} 7. b3 {0.01/0 1} Ne4 {B 0} 8. Qc2 {0.58/35 26} Nd7 {B 0} 9.
Nbd2 {0.61/33 9} c6 {0.45/37 27 (a5)} 10. Ne1 {0.51/36 24 (Bb2)} Bf6 {0.28/35 6
} 11. cxd5 {0.49/37 6} cxd5 {0.35/35 0} 12. Ndf3 {0.36/36 4} Nb8 {0.29/35 2}
13. Be3 {0.37/104 18 (Nd3)} Nc6 {0.21/38 9 (Ad7)} 14. Nd3 {0.39/36 8} Bd7 {
0.31/37 2 (De7)} 15. Rfc1 {0.29/37 8 (Qb2)} Qe7 {0.19/35 5} 16. Qd1 {0.28/37 1
(Qb2)} Rfc8 {0.17/36 4} 17. Nfe5 {0.31/36 1} Be8 {0.21/36 3} 18. a4 {0.24/35 4}
h6 {0.15/37 0 (Tc7)} 19. f3 {0.19/46 14} Ng5 {0.13/39 1} 20. Nxc6 {0.20/104 10
(Bf2)} Bxc6 {0.12/36 4 (Txc6)} 21. Bf2 {0.20/35 4} Nf7 {0.19/34 8} 22. e3 {
0.24/32 3 (Rc2)} Qe8 {0.19/32 4} 23. h4 {0.28/32 1} Be7 {0.17/31 4 (a5)} 24.
Bh3 {0.29/30 5 (Be1)} a5 {0.08/32 2 (Ad7)} 25. Be1 {0.37/42 6} b6 {0.01/33 3
(Ad7)} 26. g4 {0.16/45 3} Bd7 {0.12/36 0} 27. Bg3 {0.21/36 2} Qd8 {0.00/40 2}
28. gxf5 {0.08/45 4 (Qe1)} exf5 {0.00/44 4} 29. Qe1 {0.05/37 1} Rxc1 {0.00/45 7
} 30. Rxc1 {0.00/42 1} Rc8 {0.00/45 3} 31. Rxc8 {0.00/43 3} Qxc8 {0.00/44 2}
32. Qd2 {0.00/41 1 SolistaChess,Kayra 1.0 v6 avx2 offers a draw} g5 {0.00/40 3}
33. hxg5 {0.00/41 1} hxg5 {0.00/46 2} 34. Bf1 {0.00/39 1} Be6 {0.00/41 3 (Rg7)}
1/2-1/2[/pgn]
Uri Blass
Posts: 11078
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: My test for Kayra 1.0 v6 bmi2

Post by Uri Blass »

connor_mcmonigle wrote: Thu Dec 23, 2021 8:15 pm
mehmet123 wrote: Thu Dec 23, 2021 7:57 pm
Guenther wrote: Thu Dec 23, 2021 7:46 pm he compiled just his psqt changes (which have near zero effect due to nnue, but this was already told so often to no avail)
I wrote before
"Psqt codes are very important codes. In the test I made yesterday by giving strange values to the psqt codes, this engine (Kayra X NNUE ) was defeated by Stockfish NNUE with a 100-0 score."


Differences between Stockfish and Kayra codes.
https://github.com/Vizvezdenec/Stockfis ... baadf0a077

But this list is incomplete. They forgot to add some changes.
You seem to be confused. It's trivial to make SF massively weaker by choosing awful PSQT values as Stockfish switches to the classical evaluation function when handling winning/lost positions as decided by the classical evaluation function. However, disabling the classical evaluation function entirely only loses 1-2 elo. The classical evaluation function is irrelevant to Stockfish's overall strength (only 1-2 elo can be attributed to it).
1)I read in the stockfish forum a claim that it is 10 elo and not 1-2 elo

https://groups.google.com/g/fishcooking/c/v5cqRoHaetk

I do not know if it is correct.

2)Even if it is 1-2 elo it only means that the classical evaluation of today does not change the playing strength of stockfish by more than 1-2 elo.
It does not mean that the classical evaluation function is irrelevant because of the following reasons:

a)maybe by some change in the classical evaluation you can get 10 elo improvement in stockfish.

b)elo is not everything.
If stockfish can find mates faster in positions when one side has a big material advantage then it is important for people who use it for analysis and if I understand correctly with a big material advantage stockfish does not use NNUE but only classical evaluation.

I found in the past that stockfish is slow in mating the opponent with black when white starts without queen d1.
If you can make it faster relative to other engines thanks to improving the classical evaluation then I consider it as more important improvement for users then adding 5 elo.
Eduard
Posts: 1439
Joined: Sat Oct 27, 2018 12:58 am
Location: Germany
Full name: N.N.

Re: My test for Kayra 1.0 v6 bmi2

Post by Eduard »

12/25/2021 Kayra.bin (ZIP):
Kayra is a new young UCI engine, based on Stockfish Dev. The engine has its own style of play, according to the author Mehmet Karaman. I have tested Kayra on PlayChess with a test book with a lot of openings (including gambits) and have not yet lost a game (currently 692 games in a row). This little Kayra matchbook has now been created from the best online games as well as my own analyzes with game fragments. I wish everyone a Merry Christmas and a Happy New Year 2022!

Download:
https://filehorst.de/d/egFrDjsy

and on my Homepage:
https://solista-chess.jimdosite.com/

This book is only for this forum and my homepage. Please do not post alternative download links.
Eduard
Posts: 1439
Joined: Sat Oct 27, 2018 12:58 am
Location: Germany
Full name: N.N.

Re: My test for Kayra 1.0 v6 bmi2

Post by Eduard »

As I can see, Kayra cannot currently load any BIN books directly, maybe this can be changed in the future? In Banksia GUI, however, the BIN format is standard and the book can be loaded in the GUI. I'm planning a CTG matchbook with Kayra games, but it takes longer because I edit my books manually by hand.
User avatar
Sylwy
Posts: 4942
Joined: Fri Apr 21, 2006 4:19 pm
Location: IAȘI - the historical capital of MOLDOVA
Full name: Silvian Rucsandescu

Re: My test for Kayra 1.0 v6 bmi2

Post by Sylwy »

Eduard wrote: Sat Dec 25, 2021 5:12 am
Download:
https://filehorst.de/d/egFrDjsy
THANK YOU ! SUPER !

Using the .pgn Kayra matches collection you can made (easy) an opening book in .abk format under Arena GUI. Here is a short tutorial:

https://www.chess2u.com/t4215-how-to-ma ... p-tutorial

This book can be used like GUI book for all engines in matches/tournaments in Arena GUI ! :wink: