Lc0(Kayra4) Tests:

[mehmet123]

I made more parameter changes for Lc0 and for the first time Lc0 managed to beat Stockfish at these conditions.

Program Elo + - Games Score Av.Op. Draws
1 Lc0 591226 (Kayra4) : 2409 16 16 500 50.3 % 2407 71.8 %
2 Stockfish 050420 (c0) : 2407 16 16 500 49.7 % 2409 71.8 %

Previous Test:
1 Stockfish 050420 [c0] : 2407 9 9 2000 51.9 % 2393 65.1 %
2 Lc0 591226 [Kayra3] : 2402 13 13 1000 49.3 % 2407 64.4 %
3 Lc0 591226 [Default/Kiudee] : 2385 13 13 1000 46.8 % 2407 65.8 %


Individual statistics:

1 Lc0 591226 (Kayra4) : 2409 500 (+ 72,=359,- 69), 50.3 %

Stockfish 050420 (c0) : 500 (+ 72,=359,- 69), 50.3 %

2 Stockfish 050420 (c0) : 2407 500 (+ 69,=359,- 72), 49.7 %

Lc0 591226 (Kayra4) : 500 (+ 69,=359,- 72), 49.7 %


Arena Gui, 30'' + 0.5'' sec tc, Lc0 v.0.25.1 Balsa Opening Book (5 moves)
Stockfish: Core i7- 9750h (2 cores), Contempt 0
Lc0:Nvdia Gtx 1650
Hash:512 Mb , No Tablebase, Ponder:Off


Lc0 (Kayra4) is 7 elo better than Lc0 (Kayra3) and 24 elo better than Lc0 (Default/Kiudee) according to this test. (Elostat)


https://www.mediafire.com/file/zhsw2egs ... s.PNG/file

[mehmet123]

Program Elo + - Games Score Av.Op. Draws

1 Lc0 62676 [Kayra4] : 2432 26 26 200 55.8 % 2392 70.5 %
2 Stockfish 11 x64 bmi2 : 2392 26 26 200 44.2 % 2432 70.5 %

Individual statistics:

1 Lc0 62676 [Kayra4] : 2432 200 (+ 41,=141,- 18), 55.8 %

Stockfish 11 x64 bmi2 : 200 (+ 41,=141,- 18), 55.8 %

2 Stockfish 11 x64 bmi2 : 2392 200 (+ 18,=141,- 41), 44.2 %

Lc0 62676 [Kayra4] : 200 (+ 18,=141,- 41), 44.2 %


Arena Gui, 2 min + 0.5 sec tc, Lc0 v.0.25.1, Balsa Opening Book (5 moves)
Stockfish: Core i7- 9750h (1 core), Contempt 0
Lc0:Nvdia Gtx 1650
Hash:512 Mb , No Tablebase, Ponder:Off

Previous Tests

Program Elo + - Games Score Av.Op. Draws
1 Lc0 62676 [Kayra3] : 2416 34 34 120 53.3 % 2392 70.0 %
2 Stockfish 11 x64 bmi2 : 2392 34 34 120 46.7 % 2416 70.0 %


Program Elo + - Games Score Av.Op. Draws
1 Lc0 62676 [Kayra2] : 2412 23 21 150 52.7 % 2394 84.0 %
2 Lc0 62676 (Default/Kiudee) : 2395 23 24 150 49.0 % 2402 82.0 %
3 Stockfish 11 x64 bmi2 : 2392 31 31 150 48.3 % 2404 68.7 %


Lc0 (Kayra4) is 16 elo better than Lc0 (Kayra3) and 37 elo better than Lc0 (Default/Kiudee) according to this test. (Elostat)

http://www.mediafire.com/file/2yki2rw9v ... 5.pgn/file

[mehmet123]

Kayra4 Settings:
https://www.mediafire.com/file/zhsw2egs ... s.PNG/file

This settings is for suitable only latest Lc0 versions: (Lc0 v0.25.0, Lc0 v0.25.1 etc.)

[mehmet123]

I accidentally shared the link of the Kayra4 settings file instead of the pgn file of the first test.

This the pgn file of first test:
http://www.mediafire.com/file/zs13245e6 ... 1.pgn/file

Program Elo + - Games Score Av.Op. Draws
1 Lc0 591226 (Kayra4) : 2409 16 16 500 50.3 % 2407 71.8 %
2 Stockfish 050420 (c0) : 2407 16 16 500 49.7 % 2409 71.8 %

Arena Gui, 30'' + 0.5'' sec tc, Lc0 v.0.25.1 Balsa Opening Book (5 moves)
Stockfish: Core i7- 9750h (2 cores), Contempt 0
Lc0:Nvdia Gtx 1650
Hash:512 Mb , No Tablebase, Ponder:Off

[corres]

I made more parameter changes for Lc0 and for the first time Lc0 managed to beat Stockfish at these conditions.
Program Elo + - Games Score Av.Op. Draws
1 Lc0 591226 (Kayra4) : 2409 16 16 500 50.3 % 2407 71.8 %
2 Stockfish 050420 (c0) : 2407 16 16 500 49.7 % 2409 71.8 %
Previous Test:
1 Stockfish 050420 [c0] : 2407 9 9 2000 51.9 % 2393 65.1 %
2 Lc0 591226 [Kayra3] : 2402 13 13 1000 49.3 % 2407 64.4 %
3 Lc0 591226 [Default/Kiudee] : 2385 13 13 1000 46.8 % 2407 65.8 %
Individual statistics:
1 Lc0 591226 (Kayra4) : 2409 500 (+ 72,=359,- 69), 50.3 %
Stockfish 050420 (c0) : 500 (+ 72,=359,- 69), 50.3 %
2 Stockfish 050420 (c0) : 2407 500 (+ 69,=359,- 72), 49.7 %
Lc0 591226 (Kayra4) : 500 (+ 69,=359,- 72), 49.7 %
Arena Gui, 30'' + 0.5'' sec tc, Lc0 v.0.25.1 Balsa Opening Book (5 moves)
Stockfish: Core i7- 9750h (2 cores), Contempt 0
Lc0:Nvdia Gtx 1650
Hash:512 Mb , No Tablebase, Ponder:Off
Lc0 (Kayra4) is 7 elo better than Lc0 (Kayra3) and 24 elo better than Lc0 (Default/Kiudee) according to this test. (Elostat)

GTX 1650 is a rather weak GPU of NVIDIA and TC is too short.
We like to see results for NVIDIA RTX GPUs and with longer TC.

[mehmet123]

GTX 1650 is a rather weak GPU of NVIDIA and TC is too short.
We like to see results for NVIDIA RTX GPUs and with longer TC.

i want to see this too. My first goal is seeing the weaknesses of the Kayra settings and develop better settings for Lc0 chess engine.

And I give some information abot the speed of Lc0 and Stockfish.

Lc0 591226: ~24 kn/s at starting position (1 minute)
Lc0 63500: ~4 kn/s at starting position (1 minute)

Stockfish 11 (1 core) : 2000 kn/s at starting position (1 minute)
Stockfish 11 (2 core) : 3900 kn/s at starting position (1 minute)

[mehmet123]

Program Elo + - Games Score Av.Op. Draws
1 Lc0 63500 (Kayra4] : 2410 20 20 300 51.7 % 2399 73.3 %
2 Stockfish 110520 x64 : 2399 14 14 600 49.7 % 2401 74.0 %
3 Lc0 63500 (Default) : 2392 20 20 300 49.0 % 2399 74.7 %

Individual statistics:
1 Lc0 63500 (Kayra4] : 2410 300 (+ 45,=220,- 35), 51.7 %
Stockfish 110520 x64 bmi2 : 300 (+ 45,=220,- 35), 51.7 %

2 Stockfish 110520 x64 bmi2 : 2399 600 (+ 76,=444,- 80), 49.7 %
Lc0 63500 (Default) : 300 (+ 41,=224,- 35), 51.0 %
Lc0 63500 (Kayra4] : 300 (+ 35,=220,- 45), 48.3 %

3 Lc0 63500 (Default) : 2392 300 (+ 35,=224,- 41), 49.0 %
Stockfish 110520 x64 bmi2 : 300 (+ 35,=224,- 41), 49.0 %

Arena Gui, 1 min + 0.5 sec tc, Lc0 v.0.25.1, Balsa Opening Book (5 moves)
Stockfish: Core i7- 9750h (1 core), Contempt 0, Lc0:Nvdia Gtx 1650
Hash:512 Mb , No Tablebase, Ponder:Off

http://www.mediafire.com/file/29141qdog ... 2.pgn/file

[mehmet123]

Let's look at the results of Kayra4 tests made by me:

Kayra4 : 24 elo stronger than Default settings: (500 games)
Kayra4: 37 elo stronger than Default settings: (200 games)
Kayra4 : 18 elo stronger than Default settings: (300 games)

Average elo gain is +24.8 elo (Total 1000 games)
You can find all the pgn files at this section.

Let's look at the all indepence tests/ Kayra vs Stockfish: (Minimum 100 games)

This test was made by user 'Hluth' from Lc0 Discord Channel.

# PLAYER : RATING ERROR POINTS PLAYED (%) CFS(%) W D L D(%)
1 Lc0 v0.24.1 63082 Kayra2 : 49.39 42.70 57.0 100 57.00 72 25 64 11 64.00
2 Lc0 v0.24.1 63082 Default : 31.63 42.00 54.5 100 54.50 93 24 61 15 61.00
3 Stockfish 11 64 POPCNT : 0.00 ---- 88.5 200 44.25 --- 26 125 49 62.50

Match conditions: Permanent brain (ponder ON), Alternate colors.
Book: First 50 openings of Drawkiller_EloZoom_small500.pgn, of the Drawkiller_Openings_V3.1 by Stefan Pohl and Hauke Lutz, played by each engine as white and black (downloaded from: https://www.sp-cc.de/files/drawkiller_openings.zip).
Tablebases: Syzygy (GUI) sbases345.
Adjudication: Manual adjudication from the GUI of a few games, especially dead drawn positions misevaluated by the engines.
Software: FritzGUI 17.
Lc0 (Kayra2) is + 17. 8 elo better than Lc0 (Default) against Stockfish.

https://www.mediafire.com/file/k2co0sbc ... s.zip/file

This test was made by User 'Einyen' from Lc0 Discord Channel.

# PLAYER : RATING ERROR POINTS PLAYED (%) CFS(%) W D L D(%)
1 LS14.3 Kayra3 : 103.6 10.6 1283.0 2000 64.2 76 748 1070 182 53.5
2 LS14.3 Kiudee : 98.2 10.3 1269.0 2000 63.5 100 737 1064 199 53.2
3 Stockfish_20031719 : 0.0 ---- 1448.0 4000 36.2 --- 381 2134 1485 53.4

Match: Stockfish_20031719 vs LS 14.3 with Kayra3 settings
Hardware: i7-5960X + Geforce RTX 2080 LC0 version: v0.24.1
LC0 options: --backend=cudnn-fp16
Book: Chad's 6-ply book, openings-6ply-1000.pgn order=sequential -repeat
Adjudication: 6-man TB on SSD
Software: cutechess-cli (dev-version); Ordo 1.2.6

Lc0 (Kayra3) is + 5.4 elo better than Lc0 (Default) against Stockfish.

http://www.mediafire.com/file/24mwxp9su ... gn.7z/file

This test was made by Andreas Strangmüller:

Lc0 0.25.1 63305(Default) vs Stockfish 11 C0 : 250 ( 16, 199, 35), % 46.2
Lc0 0.25.1 63305 (Kayra4) vs Stockfish 11 C0 : 250 ( 28, 188, 34), % 48.8

Lc0 (Kayra4) is + 18 elo better than Lc0 (Default) against Stockfish. (Elostat)

http://www.fastgm.de/h2h16-60.html