T 59 Analysis-Settings beats kiudee

[Eduard]

I just did a short match with T59 59847. It played my (preliminary) analysis setting for this network against the kiudee.

CPU = AMD Ryzen 2700
GPU = GTX 1050 Ti
GUI = Fritz 17, Szyzygy 3+4+5+6+ some 7-men.
Level = 2m + 2s
Opening = Hert_250, first 10 variants.

My analysis setting played:

--threads=2
--minibatch-size=256
--max-prefetch=16
--nncache=2000000
--cpuct=2.850000
--cpuct-factor=2.000000
--policy-softmax-temp=2.200000
--fpu-value=0.750000
--move-overhead=70
--slowmover=0.900000
--time-steepness=3.50
--time-midpoint-move=18.50
--syzygy-paths=G:\3456men;G:\7men;F:\7men

kiudee setting played:

--threads=2
--minibatch-size=256
--max-prefetch=16
--nncache=2000000
--cpuct=2.147000
--cpuct-base=18368.000000
--cpuct-factor=2.815000
--policy-softmax-temp=1.607000
--fpu-value=0.443000
--move-overhead=70
--slowmover=0.900000
--time-steepness=3.50
--time-midpoint-move=18.50
--syzygy-paths=G:\3456men;G:\7men;F:\7men

After 20 games: T59analyse vs T59kiudee +3 =17 -0

Best game:

[pgn][Event "Ryzen 2700, Blitz 2m+2s"]
[Site "Ryzen 2700"]
[Date "2020.01.28"]
[Round "18"]
[White "Lc0 v0.23.0+git.02fc8e0 T58847-kiudee"]
[Black "Lc0 v0.23.0+git.02fc8e0 T59847-analyse"]
[Result "0-1"]
[ECO "B90"]
[Annotator "0.19;0.14"]
[PlyCount "108"]
[TimeControl "120+2"]

{AMD Ryzen 7 2700 Eight-Core Processor 3394 MHz W=11.4 plies; 29kN/s; 26
TBAs B=12.0 plies; 18kN/s; 22 TBAs} 1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6
5. Nc3 a6 6. Be3 e5 7. Nb3 Be6 8. f3 Be7 9. Qd2 O-O 10. O-O-O Nbd7 11. g4 b5
12. g5 Nh5 13. Nd5 Bxd5 14. exd5 {Beide letzter Buchzug} f6 {0.14/11 10} 15. h4
{0.19/13 18 (gxf6)} Ng3 {0.18/15 11} 16. Rg1 {0.23/15 4} Nxf1 {0.10/15 8} 17.
Rdxf1 {0.08/15 8} a5 {0.02/14 9} 18. g6 {0.26/15 28 (Dd3)} h6 {-0.04/14 28 (a4)
} 19. Kb1 {0.06/16 12} a4 {-0.07/16 8} 20. Nc1 {-0.03/15 6} Nb6 {-0.09/18 4}
21. b3 {-0.08/16 4} Qb8 {-0.34/16 17 (Dc7)} 22. Bxh6 {-0.21/15 15} Qb7 {
-0.48/17 9} 23. Be3 {-0.39/17 8} Nc4 {-0.49/17 1} 24. Qd3 {-0.43/17 5} Na3+ {
-0.84/15 12 (f5)} 25. Kb2 {-0.90/14 19} axb3 {-0.93/15 3} 26. cxb3 {-0.93/12 1}
Rfc8 {-1.08/13 8} 27. Rg2 {-1.16/13 16 (Td1)} Bd8 {-1.23/11 11} 28. b4 {
-1.26/14 3} Rc4 {-1.34/12 11 (Lb6)} 29. Nb3 {-1.42/11 9} Qc8 {-1.53/11 4} 30.
Rc1 {-1.66/11 8 (Th2)} Qh3 {-1.83/10 6} 31. Rf2 {-1.84/10 6} Rxb4 {-2.02/9 5}
32. Ka1 {-1.88/9 1} Nc4 {-2.05/10 4 (Tc4)} 33. Rcc2 {-1.89/9 4} f5 {-1.91/14 4}
34. Bg5 {-1.80/13 2} Bb6 {-1.93/13 2} 35. Rh2 {-1.85/13 2} Qg3 {-2.02/12 2} 36.
Qc3 {-1.85/12 2} Rxb3 {-2.27/12 6 (Dg1+)} 37. Qxb3 {-2.15/11 4} f4 {-2.29/12 0}
38. Rh1 {-2.27/11 3} Bd4+ {-2.30/11 2} 39. Kb1 {-2.33/11 1} Be3 {-2.37/10 2}
40. Qd3 {-2.39/10 3} Qxf3 {-2.40/10 3} 41. Re1 {-2.47/9 2} e4 {-2.53/9 4} 42.
Qc3 {-2.61/9 5 (Dd1)} Qh5 {-2.69/7 5} 43. Rf1 {-2.63/7 2} Qxg6 {-2.83/9 5 (Dg4)
} 44. Bxf4 {-2.83/8 2} Bxf4 {-2.97/9 1} 45. Rxf4 {-3.02/9 2} Qg1+ {-3.13/9 2}
46. Rc1 {-3.37/9 1} Qh2 {-3.35/9 1} 47. Rcf1 {-3.28/8 2} Nd2+ {-3.38/8 2} 48.
Kc1 {-3.44/8 3} Nxf1 {-3.44/7 2} 49. Rxf1 {-3.54/7 2} Qxa2 {-3.61/7 3} 50. Rg1
{-3.94/6 4} Qa1+ {-4.56/8 3} 51. Kd2 {-4.59/7 3} Qxg1 {-5.54/6 2} 52. Qc6 {
-5.29/5 3 (Dd4)} Qf2+ {-128.00/3 3} 53. Kc3 {-128.00/2 2 (Kd1)} Ra3+ {-128.00/
3 3} 54. Kb4 {-128.00/2 0} Qb2# {-128.00/2 2} 0-1[/pgn]

[mwyoung]

I just did a short match with T59 59847. It played my (preliminary) analysis setting for this network against the kiudee.

CPU = AMD Ryzen 2700
GPU = GTX 1050 Ti
GUI = Fritz 17, Szyzygy 3+4+5+6+ some 7-men.
Level = 2m + 2s
Opening = Hert_250, first 10 variants.

My analysis setting played:

--threads=2
--minibatch-size=256
--max-prefetch=16
--nncache=2000000
--cpuct=2.850000
--cpuct-factor=2.000000
--policy-softmax-temp=2.200000
--fpu-value=0.750000
--move-overhead=70
--slowmover=0.900000
--time-steepness=3.50
--time-midpoint-move=18.50
--syzygy-paths=G:\3456men;G:\7men;F:\7men

kiudee setting played:

--threads=2
--minibatch-size=256
--max-prefetch=16
--nncache=2000000
--cpuct=2.147000
--cpuct-base=18368.000000
--cpuct-factor=2.815000
--policy-softmax-temp=1.607000
--fpu-value=0.443000
--move-overhead=70
--slowmover=0.900000
--time-steepness=3.50
--time-midpoint-move=18.50
--syzygy-paths=G:\3456men;G:\7men;F:\7men

After 20 games: T59analyse vs T59kiudee +3 =17 -0

Best game:

[pgn][Event "Ryzen 2700, Blitz 2m+2s"]
[Site "Ryzen 2700"]
[Date "2020.01.28"]
[Round "18"]
[White "Lc0 v0.23.0+git.02fc8e0 T58847-kiudee"]
[Black "Lc0 v0.23.0+git.02fc8e0 T59847-analyse"]
[Result "0-1"]
[ECO "B90"]
[Annotator "0.19;0.14"]
[PlyCount "108"]
[TimeControl "120+2"]

{AMD Ryzen 7 2700 Eight-Core Processor 3394 MHz W=11.4 plies; 29kN/s; 26
TBAs B=12.0 plies; 18kN/s; 22 TBAs} 1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6
5. Nc3 a6 6. Be3 e5 7. Nb3 Be6 8. f3 Be7 9. Qd2 O-O 10. O-O-O Nbd7 11. g4 b5
12. g5 Nh5 13. Nd5 Bxd5 14. exd5 {Beide letzter Buchzug} f6 {0.14/11 10} 15. h4
{0.19/13 18 (gxf6)} Ng3 {0.18/15 11} 16. Rg1 {0.23/15 4} Nxf1 {0.10/15 8} 17.
Rdxf1 {0.08/15 8} a5 {0.02/14 9} 18. g6 {0.26/15 28 (Dd3)} h6 {-0.04/14 28 (a4)
} 19. Kb1 {0.06/16 12} a4 {-0.07/16 8} 20. Nc1 {-0.03/15 6} Nb6 {-0.09/18 4}
21. b3 {-0.08/16 4} Qb8 {-0.34/16 17 (Dc7)} 22. Bxh6 {-0.21/15 15} Qb7 {
-0.48/17 9} 23. Be3 {-0.39/17 8} Nc4 {-0.49/17 1} 24. Qd3 {-0.43/17 5} Na3+ {
-0.84/15 12 (f5)} 25. Kb2 {-0.90/14 19} axb3 {-0.93/15 3} 26. cxb3 {-0.93/12 1}
Rfc8 {-1.08/13 8} 27. Rg2 {-1.16/13 16 (Td1)} Bd8 {-1.23/11 11} 28. b4 {
-1.26/14 3} Rc4 {-1.34/12 11 (Lb6)} 29. Nb3 {-1.42/11 9} Qc8 {-1.53/11 4} 30.
Rc1 {-1.66/11 8 (Th2)} Qh3 {-1.83/10 6} 31. Rf2 {-1.84/10 6} Rxb4 {-2.02/9 5}
32. Ka1 {-1.88/9 1} Nc4 {-2.05/10 4 (Tc4)} 33. Rcc2 {-1.89/9 4} f5 {-1.91/14 4}
34. Bg5 {-1.80/13 2} Bb6 {-1.93/13 2} 35. Rh2 {-1.85/13 2} Qg3 {-2.02/12 2} 36.
Qc3 {-1.85/12 2} Rxb3 {-2.27/12 6 (Dg1+)} 37. Qxb3 {-2.15/11 4} f4 {-2.29/12 0}
38. Rh1 {-2.27/11 3} Bd4+ {-2.30/11 2} 39. Kb1 {-2.33/11 1} Be3 {-2.37/10 2}
40. Qd3 {-2.39/10 3} Qxf3 {-2.40/10 3} 41. Re1 {-2.47/9 2} e4 {-2.53/9 4} 42.
Qc3 {-2.61/9 5 (Dd1)} Qh5 {-2.69/7 5} 43. Rf1 {-2.63/7 2} Qxg6 {-2.83/9 5 (Dg4)
} 44. Bxf4 {-2.83/8 2} Bxf4 {-2.97/9 1} 45. Rxf4 {-3.02/9 2} Qg1+ {-3.13/9 2}
46. Rc1 {-3.37/9 1} Qh2 {-3.35/9 1} 47. Rcf1 {-3.28/8 2} Nd2+ {-3.38/8 2} 48.
Kc1 {-3.44/8 3} Nxf1 {-3.44/7 2} 49. Rxf1 {-3.54/7 2} Qxa2 {-3.61/7 3} 50. Rg1
{-3.94/6 4} Qa1+ {-4.56/8 3} 51. Kd2 {-4.59/7 3} Qxg1 {-5.54/6 2} 52. Qc6 {
-5.29/5 3 (Dd4)} Qf2+ {-128.00/3 3} 53. Kc3 {-128.00/2 2 (Kd1)} Ra3+ {-128.00/
3 3} 54. Kb4 {-128.00/2 0} Qb2# {-128.00/2 2} 0-1[/pgn]

Yes! Proving once again there is no best settings for every system and time control and NN. On the the other hand it is only 20 games.

[Eduard]

I agree.

[Alayan]

20 games is close to meaningless.

[marsell]

20 games is close to meaningless.

how many games will be played in the qualification of tcec?

[Eduard]

I agree.

After 40 games now +4 =33 -3 for Analysis. Enough for me to know that kiudee is not better.

[Alayan]

Eduard, if you take the last 20 games, you get +3=16-1 for kiudee. If this had been your 20 first games, what would your conclusion have been ?

The variability is high enough and the results close enough that you can't conclude. You know that neither settings lose by 200 elo, but one might be 50 elo better than the other and results like yours could still happen.

how many games will be played in the qualification of tcec?

30, and this is even worse because games against much weaker/stronger opponents have lower information value, and the opening they get can have a big impact. An engine can easily overperform or underperform by 50 elo in TCEC QL, League 2 or League 1.

Usually, the difference between the strongest and weakest engines in a league is at least 200 elo, so the order engine ends in is still somewhat reasonable. LTC games are also less noisy than STC games. But when two engines aren't too far apart in strength, upsets regularly happen.

[Eduard]

Eduard, if you take the last 20 games, you get +3=16-1 for kiudee. If this had been your 20 first games, what would your conclusion have been ?

Then Kiudee would have been better. But I thought Kiudee would be in the lead after 20 games. So the result of the analysis settings is now expected to be more than I expected after 40 games.

My stetting is not finished yet. I wait until T59 has ended. I will test both settings against Stockfish 11. I'll do that with the final version of T59.

[alex67a]

--cpuct-base for your settings ?

[Eduard]

Default (19652.000000).