Discussion of computer chess matches and engine tournaments.
Moderators: hgm , Rebel , chrisw
majortom
Posts: 669 Joined: Mon Nov 04, 2013 10:19 pm
Post
by majortom » Mon Jan 07, 2019 10:14 am
LCZero 11248 (714 nodes per move) vs LCZero 32367 (714 nodes per move)
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 LC0 11248 N714 : 0.0 0.0 0.0 200 54.8 % -34 51.5 %
2 LC0 32367 N714 : -33.6 35.0 35.0 200 45.3 % 0 51.5 %
Code: Select all
1 LC0 11248 N714 : 0.0 200 (+ 58,=103,- 39), 54.8 %
LC0 32367 N714 : 200 (+ 58,=103,- 39), 54.8 %
2 LC0 32367 N714 :-33.6 200 (+ 39,=103,- 58), 45.3 %
LC0 11248 N714 : 200 (+ 39,=103,- 58), 45.3 %
Games
majortom
Posts: 669 Joined: Mon Nov 04, 2013 10:19 pm
Post
by majortom » Mon Jan 07, 2019 10:15 am
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 LC0 11248 N714 : 0.0 0.0 0.0 1400 62.5 % -91 45.1 %
2 LC0 32367 N714 : -33.6 32.5 32.5 200 45.3 % 0 51.5 %
3 LC0 32177 N714 : -42.5 34.0 34.0 200 44.0 % 0 46.0 %
4 LC0 32015 N714 : -82.6 37.3 37.3 200 38.5 % 0 45.0 %
5 LC0 31741 N714 : -88.2 37.4 37.4 200 37.8 % 0 45.5 %
6 LC0 31885 N714 : -113.0 35.7 35.7 200 34.5 % 0 47.0 %
7 LC0 31687 N714 : -136.9 36.9 36.9 200 31.5 % 0 42.0 %
8 LC0 31633 N714 : -143.1 36.4 36.4 200 30.8 % 0 38.5 %
majortom
Posts: 669 Joined: Mon Nov 04, 2013 10:19 pm
Post
by majortom » Mon Jan 14, 2019 8:31 pm
Number of nodes is increased in 10 times.
LCZero 11248 (7140 nodes per move) vs LCZero 32458(7140 nodes per move)
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 LC0 32458 N7140 : 77.0 81.2 81.2 38 60.5 % 0 63.2 %
2 LC0 11248 N7140 : 0.0 0.0 0.0 38 39.5 % 77 63.2 %
Code: Select all
1 LC0 32458 N7140 : 77.0 38 (+ 11,= 24,- 3), 60.5 %
LC0 11248 N7140 : 38 (+ 11,= 24,- 3), 60.5 %
2 LC0 11248 N7140 : 0.0 38 (+ 3,= 24,- 11), 39.5 %
LC0 32458 N7140 : 38 (+ 3,= 24,- 11), 39.5 %
Games
majortom
Posts: 669 Joined: Mon Nov 04, 2013 10:19 pm
Post
by majortom » Fri Jan 18, 2019 12:30 pm
LCZero 11248 (7140 nodes per move) vs LCZero 32458(7140 nodes per move)
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 LC0 32458 N7140 : 54.2 61.9 61.9 60 57.5 % 0 61.7 %
2 LC0 11248 N7140 : 0.0 0.0 0.0 60 42.5 % 54 61.7 %
Code: Select all
1 LC0 32458 N7140 : 54.2 60 (+ 16,= 37,- 7), 57.5 %
LC0 11248 N7140 : 60 (+ 16,= 37,- 7), 57.5 %
2 LC0 11248 N7140 : 0.0 60 (+ 7,= 37,- 16), 42.5 %
LC0 32458 N7140 : 60 (+ 7,= 37,- 16), 42.5 %
Games
brianr
Posts: 536 Joined: Thu Mar 09, 2006 3:01 pm
Post
by brianr » Fri Jan 18, 2019 2:07 pm
FWIW:
Code: Select all
Score of ID-32585 vs ID-11248: 61 - 63 - 176 [0.497] 300
Elo difference: -2.32 +/- 25.28
At 320 games it was exactly even.
Match with v20.1 net 32585 v 11248 at 0:10+0.4 with 6 piece TBs and 2 move book on a GTX-1070.
Of course, 32585 is not 32458.
brianr
Posts: 536 Joined: Thu Mar 09, 2006 3:01 pm
Post
by brianr » Sun Jan 20, 2019 5:25 pm
brianr wrote: ↑ Fri Jan 18, 2019 2:07 pm
FWIW:
Code: Select all
Score of ID-32585 vs ID-11248: 61 - 63 - 176 [0.497] 300
Elo difference: -2.32 +/- 25.28
At 320 games it was exactly even.
Match with v20.1 net 32585 v 11248 at 0:10+0.4 with 6 piece TBs and 2 move book on a GTX-1070.
Of course, 32585 is not 32458.
Update/correction. I ran a couple more matches. I changed the "new" net to 32747, which I think is the one selected for TCEC. The first was at the same time controls as the earlier one.
Code: Select all
Score of ID-32742 vs ID-11248: 19 - 25 - 56 [0.470] 100
Elo difference: -20.87 +/- 45.34
This result was in line with the earlier match with the other 30 series net.
However, there was talk on the Leela Discord chat about the newer nets scaling better than the older ones, so I did another match with tc=1:00+2.0 (1 min plus 2 seconds per move). The results were somewhat startling.
Code: Select all
Score of ID-32742 vs ID-11248: 27 - 12 - 61 [0.575] 100
Elo difference: 52.51 +/- 42.39
I suspect the difference will be even greater on stronger hardware.
Also, the 11248 net was trained with the 50 move bug.
majortom
Posts: 669 Joined: Mon Nov 04, 2013 10:19 pm
Post
by majortom » Sat Jan 26, 2019 9:35 am
LCZero 11248 (7140 nodes per move) vs LCZero 32458(7140 nodes per move)
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 LC0 32458 N7140 : 66.0 52.1 52.1 100 59.0 % 0 62.0 %
2 LC0 11248 N7140 : 0.0 0.0 0.0 100 41.0 % 66 62.0 %
Code: Select all
1 LC0 32458 N7140 : 66.0 100 (+ 28,= 62,- 10), 59.0 %
LC0 11248 N7140 : 100 (+ 28,= 62,- 10), 59.0 %
2 LC0 11248 N7140 : 0.0 100 (+ 10,= 62,- 28), 41.0 %
LC0 32458 N7140 : 100 (+ 10,= 62,- 28), 41.0 %
Games