LCZero: Progress and Scaling. Relation to CCRL Elo

[Milos]

I wonder if it's just an accident that the regression started around the time v8 was released?

I played a quick match v0.8 (without TBs) vs v0.7 and they came pretty close, so I don't think there is some bug introduces. Puct was changed from 0.6 to 0.75 but that makes no much difference.
Btw. I also played a match of cuDNN Win version 050518 with batch size 256 on my old GTX 770 vs v0.7, TC=60'+0.6' and cuDNN ended up with 95Elo advantage. Around 5% losses on time, most of them v0.7.

[Milos]

duplicate

[mirek]

I played a quick match v0.8 (without TBs) vs v0.7 and they came pretty close, so I don't think there is some bug introduces. Puct was changed from 0.6 to 0.75 but that makes no much difference.
Btw. I also played a match of cuDNN Win version 050518 with batch size 256 on my old GTX 770 vs v0.7, TC=60'+0.6' and cuDNN ended up with 95Elo advantage. Around 5% losses on time, most of them v0.7

There could also possibly be a bug that shows only in in training though.
About cuDNN yes my results are also similar in OCL vs. cuDNN lc0 but when testing against different engines the cuDNN version had only like 20, 30 elo edge, basically still within the 1 sigma range. So I would be actually curious if there is anyone who had made some serious testing (not just form 100 games as I did) of cuDNN against other engines and what are the results.

[Albert Silver]

I played a quick match v0.8 (without TBs) vs v0.7 and they came pretty close, so I don't think there is some bug introduces. Puct was changed from 0.6 to 0.75 but that makes no much difference.
Btw. I also played a match of cuDNN Win version 050518 with batch size 256 on my old GTX 770 vs v0.7, TC=60'+0.6' and cuDNN ended up with 95Elo advantage. Around 5% losses on time, most of them v0.7

There could also possibly be a bug that shows only in in training though.
About cuDNN yes my results are also similar in OCL vs. cuDNN lc0 but when testing against different engines the cuDNN version had only like 20, 30 elo edge, basically still within the 1 sigma range. So I would be actually curious if there is anyone who had made some serious testing (not just form 100 games as I did) of cuDNN against other engines and what are the results.

I had good results, +60 Elo difference, when testing cuDNN (May 5) with puct 3.0 instead of its default 1.7.

[carldaman]

I got this elo graph from someone who come from "Future" https://ibb.co/cxzk67

Not a crazy prediction so far...

[Ralph Stoesser]

Seems Leela has fallen in love with Moron

[peter]

Hi Albert!

I had good results, +60 Elo difference, when testing cuDNN (May 5) with puct 3.0 instead of its default 1.7.

In WAC- tactical postions test-suite with NW 265 from today and the same 24 threads of the 12x3GHz CPU and 10"/move as always, the worst result of all my runs till now.

Having started such with something like NW120 and version 0.7 then, there were about 150 out of 300 poitions solved, best was 159 before making the network bigger, latest till now was 144 and today 140 only.

[Laskos]

Progress to ID265 with v0.8:

Code: Select all

LCZero v0.8
6s/position 4 CPU threads (equivalent to 1s on GTX 1060)

WAC300 tactical:
ID227: 120/300
ID258: 110/300
ID265: 109/300
performance below 1800 Elo points AB engines, worsening

Openings200 positional:
ID227:  98/200
ID258: 111/200
ID265  114/200
performance above 3300 Elo points AB engines, improving

The eval improvement seems to be going on, comparing LC0 with 1 playout against Komodo 11.3.1 at depth=1:



But strength improvement at fixed time compared to ID227 (first of the larger nets) is lagging behind:

[Albert Silver]

Hi Albert!

I had good results, +60 Elo difference, when testing cuDNN (May 5) with puct 3.0 instead of its default 1.7.

In WAC- tactical postions test-suite with NW 265 from today and the same 24 threads of the 12x3GHz CPU and 10"/move as always, the worst result of all my runs till now.

Having started such with something like NW120 and version 0.7 then, there were about 150 out of 300 poitions solved, best was 159 before making the network bigger, latest till now was 144 and today 140 only.

You are aware that the cuDNN versions cannot run on just CPUs? They also have different code and behavior than the normal LCZero. It is not 'just faster'.

[peter]

You are aware that the cuDNN versions cannot run on just CPUs? They also have different code and behavior than the normal LCZero. It is not 'just faster'.

I have the CPU- version only. My graphic card wouldn't be bad neither, but it's AMD und doesn't support OpenCl.