brianr wrote: ↑Fri Jan 18, 2019 2:07 pm
FWIW:
Code: Select all
Score of ID-32585 vs ID-11248: 61 - 63 - 176 [0.497] 300
Elo difference: -2.32 +/- 25.28
At 320 games it was exactly even.
Match with v20.1 net 32585 v 11248 at 0:10+0.4 with 6 piece TBs and 2 move book on a GTX-1070.
Of course, 32585 is not 32458.
Update/correction. I ran a couple more matches. I changed the "new" net to 32747, which I think is the one selected for TCEC. The first was at the same time controls as the earlier one.
Code: Select all
Score of ID-32742 vs ID-11248: 19 - 25 - 56 [0.470] 100
Elo difference: -20.87 +/- 45.34
This result was in line with the earlier match with the other 30 series net.
However, there was talk on the Leela Discord chat about the newer nets scaling better than the older ones, so I did another match with tc=1:00+2.0 (1 min plus 2 seconds per move). The results were somewhat startling.
Code: Select all
Score of ID-32742 vs ID-11248: 27 - 12 - 61 [0.575] 100
Elo difference: 52.51 +/- 42.39
I suspect the difference will be even greater on stronger hardware.
Also, the 11248 net was trained with the 50 move bug.