Lc0 and future of rating lists

[Hugo]

Hi All

I am curious about what the rating lists are thinking about Lc0?

A list is quite incomplete, as long one of the top 1-2-3 is missing.

Isn't it time to add Lc0 based on e.g. RTX 2060?

regards, C.K.

[ThatsIt]

http://cegt.forumieren.com/t1082-testin ... 20-2-32930

Best wishes,
G.S.
(CEGT team)

[Jouni]

WOW new engine with start rating of 3400 - this is history .

[ThatsIt]

WOW new engine with start rating of 3400 - this is history .

Not really new:

http://www.cegt.net/40_4_Ratinglist/40_ ... e/668.html

http://www.cegt.net/40_4_Ratinglist/40_ ... /1635.html

Best wishes,
G.S.
(CEGT team)

[jorose]

Leela is also on CCRL eg.

http://www.computerchess.org.uk/ccrl/40 ... ating+list

Issue is these rating lists are normalized to some decades old CPU. Comparing an Athlon 64 X2 4600+ with a GTX 2060 is not really fair. Furthermore normalizing hardware becomes nontrivial, what happens when you have a fast GPU paired with a lightweight net paired with a slow CPU which could result in a CPU bottleneck?

If you can't normalize hardware then the issue becomes that you can't have different testers with different hardware. These rating lists only really work due to distributed effort being put into them.

That being said, I am sure the different teams have been thinking a lot of this and have their own ways they intend to solve the issues. I'm just pointing out that it is nontrivial to do so.

[ThatsIt]

@jorose

Its no problem for our lists in my view.
We will test LCZero as a "black box", only the same
two machines in the next 1 or 2 years. The so-called
Leela-Ratio is 1.25-1.3 @ 10" per move and 1.60-1.75
@ 120" per move. For our FTC-lists (40/4 and 3'+1")
the first value is important. Fair enough i guess.

Best wishes,
G.S.
(CEGT team)

[Modern Times]

It is certainly difficult for an equal hardware list, where equality is based on CPU benchmarks. SSDF is in the best position, they have engines listed running on different hardware already. No issues for them at all.

[dkappe]

Lc0 with 11258-112x9-se distilled network is probably tops for CPU. http://www.cegt.net/40_40%20Rating%20Li ... n/355.html

[carldaman]

It is certainly difficult for an equal hardware list, where equality is based on CPU benchmarks. SSDF is in the best position, they have engines listed running on different hardware already. No issues for them at all.

Is it totally correct to think of CCRL as equal hardware where you have 1, 2 and 4CPU entities competing and being rated together?
Why can't a GPU become something like a new category of hardware? I know a lot depends on the actual GPU, and benchmarking that can become tricky, but maybe a 'most common denominator' GPU could be used as a new category of engine hardware (?)

[Modern Times]

Is it totally correct to think of CCRL as equal hardware where you have 1, 2 and 4CPU entities competing and being rated together?
Why can't a GPU become something like a new category of hardware? I know a lot depends on the actual GPU, and benchmarking that can become tricky, but maybe a 'most common denominator' GPU could be used as a new category of engine hardware (?)

Very good point, I hadn't thought of it that way.