Page 2 of 2
Re: LC0 11248 gauntlet vs new LC0 generation
Posted: Mon Jan 07, 2019 10:14 am
by majortom
LCZero 11248 (714 nodes per move) vs LCZero 32367 (714 nodes per move)
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 LC0 11248 N714 : 0.0 0.0 0.0 200 54.8 % -34 51.5 %
2 LC0 32367 N714 : -33.6 35.0 35.0 200 45.3 % 0 51.5 %
Code: Select all
1 LC0 11248 N714 : 0.0 200 (+ 58,=103,- 39), 54.8 %
LC0 32367 N714 : 200 (+ 58,=103,- 39), 54.8 %
2 LC0 32367 N714 :-33.6 200 (+ 39,=103,- 58), 45.3 %
LC0 11248 N714 : 200 (+ 39,=103,- 58), 45.3 %
Games
Re: LC0 11248 gauntlet vs new LC0 generation
Posted: Mon Jan 07, 2019 10:15 am
by majortom
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 LC0 11248 N714 : 0.0 0.0 0.0 1400 62.5 % -91 45.1 %
2 LC0 32367 N714 : -33.6 32.5 32.5 200 45.3 % 0 51.5 %
3 LC0 32177 N714 : -42.5 34.0 34.0 200 44.0 % 0 46.0 %
4 LC0 32015 N714 : -82.6 37.3 37.3 200 38.5 % 0 45.0 %
5 LC0 31741 N714 : -88.2 37.4 37.4 200 37.8 % 0 45.5 %
6 LC0 31885 N714 : -113.0 35.7 35.7 200 34.5 % 0 47.0 %
7 LC0 31687 N714 : -136.9 36.9 36.9 200 31.5 % 0 42.0 %
8 LC0 31633 N714 : -143.1 36.4 36.4 200 30.8 % 0 38.5 %
Re: LC0 11248 gauntlet vs new LC0 generation
Posted: Mon Jan 14, 2019 8:31 pm
by majortom
Number of nodes is increased in 10 times.
LCZero 11248 (7140 nodes per move) vs LCZero 32458(7140 nodes per move)
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 LC0 32458 N7140 : 77.0 81.2 81.2 38 60.5 % 0 63.2 %
2 LC0 11248 N7140 : 0.0 0.0 0.0 38 39.5 % 77 63.2 %
Code: Select all
1 LC0 32458 N7140 : 77.0 38 (+ 11,= 24,- 3), 60.5 %
LC0 11248 N7140 : 38 (+ 11,= 24,- 3), 60.5 %
2 LC0 11248 N7140 : 0.0 38 (+ 3,= 24,- 11), 39.5 %
LC0 32458 N7140 : 38 (+ 3,= 24,- 11), 39.5 %
Games
Re: LC0 11248 gauntlet vs new LC0 generation
Posted: Fri Jan 18, 2019 12:30 pm
by majortom
LCZero 11248 (7140 nodes per move) vs LCZero 32458(7140 nodes per move)
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 LC0 32458 N7140 : 54.2 61.9 61.9 60 57.5 % 0 61.7 %
2 LC0 11248 N7140 : 0.0 0.0 0.0 60 42.5 % 54 61.7 %
Code: Select all
1 LC0 32458 N7140 : 54.2 60 (+ 16,= 37,- 7), 57.5 %
LC0 11248 N7140 : 60 (+ 16,= 37,- 7), 57.5 %
2 LC0 11248 N7140 : 0.0 60 (+ 7,= 37,- 16), 42.5 %
LC0 32458 N7140 : 60 (+ 7,= 37,- 16), 42.5 %
Games
Re: LC0 11248 gauntlet vs new LC0 generation
Posted: Fri Jan 18, 2019 2:07 pm
by brianr
FWIW:
Code: Select all
Score of ID-32585 vs ID-11248: 61 - 63 - 176 [0.497] 300
Elo difference: -2.32 +/- 25.28
At 320 games it was exactly even.
Match with v20.1 net 32585 v 11248 at 0:10+0.4 with 6 piece TBs and 2 move book on a GTX-1070.
Of course, 32585 is not 32458.
Re: LC0 11248 gauntlet vs new LC0 generation
Posted: Sun Jan 20, 2019 5:25 pm
by brianr
brianr wrote: ↑Fri Jan 18, 2019 2:07 pm
FWIW:
Code: Select all
Score of ID-32585 vs ID-11248: 61 - 63 - 176 [0.497] 300
Elo difference: -2.32 +/- 25.28
At 320 games it was exactly even.
Match with v20.1 net 32585 v 11248 at 0:10+0.4 with 6 piece TBs and 2 move book on a GTX-1070.
Of course, 32585 is not 32458.
Update/correction. I ran a couple more matches. I changed the "new" net to 32747, which I think is the one selected for TCEC. The first was at the same time controls as the earlier one.
Code: Select all
Score of ID-32742 vs ID-11248: 19 - 25 - 56 [0.470] 100
Elo difference: -20.87 +/- 45.34
This result was in line with the earlier match with the other 30 series net.
However, there was talk on the Leela Discord chat about the newer nets scaling better than the older ones, so I did another match with tc=1:00+2.0 (1 min plus 2 seconds per move). The results were somewhat startling.
Code: Select all
Score of ID-32742 vs ID-11248: 27 - 12 - 61 [0.575] 100
Elo difference: 52.51 +/- 42.39
I suspect the difference will be even greater on stronger hardware.
Also, the 11248 net was trained with the 50 move bug.
Re: LC0 11248 gauntlet vs new LC0 generation
Posted: Sat Jan 26, 2019 9:35 am
by majortom
LCZero 11248 (7140 nodes per move) vs LCZero 32458(7140 nodes per move)
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 LC0 32458 N7140 : 66.0 52.1 52.1 100 59.0 % 0 62.0 %
2 LC0 11248 N7140 : 0.0 0.0 0.0 100 41.0 % 66 62.0 %
Code: Select all
1 LC0 32458 N7140 : 66.0 100 (+ 28,= 62,- 10), 59.0 %
LC0 11248 N7140 : 100 (+ 28,= 62,- 10), 59.0 %
2 LC0 11248 N7140 : 0.0 100 (+ 10,= 62,- 28), 41.0 %
LC0 32458 N7140 : 100 (+ 10,= 62,- 28), 41.0 %
Games