Lc0 and future of rating lists

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Hugo
Posts: 772
Joined: Tue Dec 01, 2009 10:10 am

Lc0 and future of rating lists

Post by Hugo » Sun Feb 10, 2019 1:11 pm

Hi All

I am curious about what the rating lists are thinking about Lc0?

A list is quite incomplete, as long one of the top 1-2-3 is missing.

Isn't it time to add Lc0 based on e.g. RTX 2060?

regards, C.K.

ThatsIt
Posts: 887
Joined: Thu Mar 09, 2006 1:11 pm
Contact:

Re: Lc0 and future of rating lists

Post by ThatsIt » Sun Feb 10, 2019 1:18 pm


Jouni
Posts: 1953
Joined: Wed Mar 08, 2006 7:15 pm

Re: Lc0 and future of rating lists

Post by Jouni » Mon Feb 11, 2019 9:28 am

WOW new engine with start rating of 3400 - this is history :!: :D :o .
Jouni

ThatsIt
Posts: 887
Joined: Thu Mar 09, 2006 1:11 pm
Contact:

Re: Lc0 and future of rating lists

Post by ThatsIt » Mon Feb 11, 2019 10:22 am

Jouni wrote:
Mon Feb 11, 2019 9:28 am
WOW new engine with start rating of 3400 - this is history :!: :D :o .
Not really new:

http://www.cegt.net/40_4_Ratinglist/40_ ... e/668.html

http://www.cegt.net/40_4_Ratinglist/40_ ... /1635.html

Best wishes,
G.S.
(CEGT team)

jorose
Posts: 232
Joined: Thu Jan 22, 2015 2:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: Lc0 and future of rating lists

Post by jorose » Mon Feb 11, 2019 1:09 pm

Leela is also on CCRL eg.

http://www.computerchess.org.uk/ccrl/40 ... ating+list

Issue is these rating lists are normalized to some decades old CPU. Comparing an Athlon 64 X2 4600+ with a GTX 2060 is not really fair. Furthermore normalizing hardware becomes nontrivial, what happens when you have a fast GPU paired with a lightweight net paired with a slow CPU which could result in a CPU bottleneck?

If you can't normalize hardware then the issue becomes that you can't have different testers with different hardware. These rating lists only really work due to distributed effort being put into them.

That being said, I am sure the different teams have been thinking a lot of this and have their own ways they intend to solve the issues. I'm just pointing out that it is nontrivial to do so.
-Jonathan

ThatsIt
Posts: 887
Joined: Thu Mar 09, 2006 1:11 pm
Contact:

Re: Lc0 and future of rating lists

Post by ThatsIt » Mon Feb 11, 2019 5:11 pm

@jorose

Its no problem for our lists in my view.
We will test LCZero as a "black box", only the same
two machines in the next 1 or 2 years. The so-called
Leela-Ratio is 1.25-1.3 @ 10" per move and 1.60-1.75
@ 120" per move. For our FTC-lists (40/4 and 3'+1")
the first value is important. Fair enough i guess.

Best wishes,
G.S.
(CEGT team)

Modern Times
Posts: 2292
Joined: Thu Jun 07, 2012 9:02 pm

Re: Lc0 and future of rating lists

Post by Modern Times » Mon Feb 11, 2019 5:43 pm

It is certainly difficult for an equal hardware list, where equality is based on CPU benchmarks. SSDF is in the best position, they have engines listed running on different hardware already. No issues for them at all.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

dkappe
Posts: 150
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Lc0 and future of rating lists

Post by dkappe » Tue Feb 12, 2019 11:47 pm

Lc0 with 11258-112x9-se distilled network is probably tops for CPU. http://www.cegt.net/40_40%20Rating%20Li ... n/355.html

carldaman
Posts: 1640
Joined: Sat Jun 02, 2012 12:13 am

Re: Lc0 and future of rating lists

Post by carldaman » Wed Feb 13, 2019 1:43 am

Modern Times wrote:
Mon Feb 11, 2019 5:43 pm
It is certainly difficult for an equal hardware list, where equality is based on CPU benchmarks. SSDF is in the best position, they have engines listed running on different hardware already. No issues for them at all.
Is it totally correct to think of CCRL as equal hardware where you have 1, 2 and 4CPU entities competing and being rated together?
Why can't a GPU become something like a new category of hardware? I know a lot depends on the actual GPU, and benchmarking that can become tricky, but maybe a 'most common denominator' GPU could be used as a new category of engine hardware (?)

Modern Times
Posts: 2292
Joined: Thu Jun 07, 2012 9:02 pm

Re: Lc0 and future of rating lists

Post by Modern Times » Wed Feb 13, 2019 5:07 am

carldaman wrote:
Wed Feb 13, 2019 1:43 am

Is it totally correct to think of CCRL as equal hardware where you have 1, 2 and 4CPU entities competing and being rated together?
Why can't a GPU become something like a new category of hardware? I know a lot depends on the actual GPU, and benchmarking that can become tricky, but maybe a 'most common denominator' GPU could be used as a new category of engine hardware (?)
Very good point, I hadn't thought of it that way.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Post Reply