Laskos wrote: ↑Fri Jul 13, 2018 8:38 pm
10 times longer TC:
600s + 10s
Score of Komodo 12.1.1 vs Lc0_11jul ID482: 6 - 1 - 3 [0.750] 10
Elo difference: 190.85 +/- 255.97
Finished match
Performance 3225 CCRL 40/4', even a regression compared earlier TC. LOS = 96.5% for Komodo, which I expected to be very similar in strength at this longer TC. Maybe I will leave overnight for another 10 games, if the result will be again weak, it is probably not bad luck.
2 times shorter TC from above:
Lc0 ID462 with latest CLOPed settings for this 450-460 series after
100 games
Code: Select all
Program CCRL Elo Error(cl 95%) Games Score
Lc0! ID462 3280.6 ±42.3 100 (+38,=46,-16), 61.0 %
vs. : games ( +, =, -), (%) : Diff, SD, CFS (%)
Fire 7.1 : 20 ( 4, 7, 9), 37.5 : -60.4, 21.6, 0.3
Andscacs 9.3 : 20 ( 7, 11, 2), 62.5 : +73.6, 21.6, 100.0
Gull 3 : 20 ( 5, 13, 2), 57.5 : +87.6, 21.6, 100.0
Texel 1.07 : 20 ( 10, 8, 2), 70.0 : +119.6, 21.6, 100.0
Laser 1.5 : 20 ( 12, 7, 1), 77.5 : +183.6, 21.6, 100.0
Lc0 kb3 b20 net with CLOPed settings for it after
100 games
Code: Select all
Program CCRL Elo Error(cl 95%) Games Score
Lc0 kb3 b20 3333.6 ±47.2 100 (+46,=43,-11) 67.5 %
vs. : games ( +, =, -), (%) : Diff, SD, CFS (%)
Fire 7.1 : 20 ( 4, 12, 4), 50.0 : -7.4, 24.1, 38.0
Andscacs 9.3 : 20 ( 8, 8, 4), 60.0 : +126.6, 24.1, 100.0
Gull 3 : 20 ( 9, 11, 0), 72.5 : +140.6, 24.1, 100.0
Texel 1.07 : 20 ( 13, 5, 2), 77.5 : +172.6, 24.1, 100.0
Laser 1.5 : 20 ( 12, 7, 1), 77.5 : +236.6, 24.1, 100.0
Lc0 ID390 with default CLOPed settings for this after
50 games
Code: Select all
Program CCRL Elo Error(cl 95%) Games Score
Lc0 ID390 3321.0 ±69.8 50 (+24,=18,-8) 66.0 %
vs. : games ( +, =, -), (%) : Diff, SD, CFS (%)
Fire 7.1 : 10 ( 1, 4, 5), 30.0 : -20.0, 35.6, 28.7
Andscacs 0.93 : 10 ( 7, 3, 0), 85.0 : +114.0, 35.6, 99.9
Gull 3 : 10 ( 4, 4, 2), 60.0 : +128.0, 35.6, 100.0
Texel 1.07 : 10 ( 8, 2, 0), 90.0 : +160.0, 35.6, 100.0
Laser 1.5 : 10 ( 4, 5, 1), 65.0 : +224.0, 35.6, 100.0
Conditions:
TC: 5'+5"
Ponder off
1 CPU for all AB engines with i5-6500 3.4 GHz and Nvidia GTX 1070 Ti with Lc0 for Leela.
My system with 1 CPU is 2.1 times faster than 1 CPU of CCRL and since 5+5 is close enough to 40/4 i used in the above Elo's the CCRL 40/4 Elo's of the AB engines.
In any case since Elo is relative you can just subtract any value you want and have the relative Elo of each gaunlet without connecting it to the world of CCRL Elo's.
I am a bit worried that Leela might scale badly in TCEC conditions. Maybe some folks have experience with Leela on longer time controls. Also, do they know whether in TCEC conditions the rig they are running will have enough RAM? Leela MCTS is very RAM hungry with fast GPUs and LTC.
Well Leela had the following 3 results in TCEC (All AB engines use 43 cores and Leela Lc0 with 2 GTX 1080 Ti):
Leela ID483 - Andscacs : 10.0 - 14.0 Elo -58
Leela ID483 - Laser : 11.5 - 12.5 Elo -15
Leela ID483 - Stockfish : 3.0 - 11.0 Elo -226
and Leela scored a win against SF! They said Stockfish has never lost in premier division from any engine other than Houdini or Komodo since season 7.
Andscacs is a premier divsion engine that finished 5th last year!
Laser is a 1st division engine that finished 6th last year.