CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
konsolas
Posts: 182
Joined: Sun Jun 12, 2016 3:44 pm
Location: London
Full name: Vincent
Contact:

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by konsolas » Mon Feb 25, 2019 10:02 pm

For the most part, I believe CCRL currently uses actual time controls closer to 40/20 rather than 40/40 due to the huge advancements made in CPU hardware.

A modern quad core at 40/20 would be comparable to a 1050 at 40/20 because even though both pieces of hardware are much newer than the Athlon 4600+, they both have half as much time to match.

The problem of different GPU models could potentially be solved with asymmetric time controls.

e.g. CCRL 40/40 could mean:
  • 40 moves in 40 minutes with an Athlon64 4600+, benchmarked with (some named) version of Crafty
  • 40 moves in 20 minutes with a GTX 1050, benchmarked with (some named) version of Lc0

jorose
Posts: 269
Joined: Thu Jan 22, 2015 2:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by jorose » Mon Feb 25, 2019 10:32 pm

konsolas wrote:
Mon Feb 25, 2019 10:02 pm
For the most part, I believe CCRL currently uses actual time controls closer to 40/20 rather than 40/40 due to the huge advancements made in CPU hardware.

A modern quad core at 40/20 would be comparable to a 1050 at 40/20 because even though both pieces of hardware are much newer than the Athlon 4600+, they both have half as much time to match.

The problem of different GPU models could potentially be solved with asymmetric time controls.

e.g. CCRL 40/40 could mean:
  • 40 moves in 40 minutes with an Athlon64 4600+, benchmarked with (some named) version of Crafty
  • 40 moves in 20 minutes with a GTX 1050, benchmarked with (some named) version of Lc0
Ok, but how do you deal with the reality that GPU based engines also need a CPU? While Leela is certainly quite GPU heavy, I am quite sure even in this case the CPU is non-negligible. To make matters worse, you could easily imagine an engine which is roughly evenly bottlenecked by the GPU and CPU. How are you going to deal with that?
-Jonathan

Modern Times
Posts: 2417
Joined: Thu Jun 07, 2012 9:02 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by Modern Times » Tue Feb 26, 2019 6:35 am

The GTX 1050 has been bought because of low cost and power efficiency and with decent performance, and that my 8 year old noisy and power hungry GPU needs replacing.

However it then opens up the opportunity to try some blitz-only GPU testing (not 40/40). In the short term it is easy enough but yes there are medium to longer term issues to resolve as well. I think it is a good fit for the list, but if it isn't welcomed then I simply won't bother.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

User avatar
hgm
Posts: 23772
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by hgm » Tue Feb 26, 2019 6:57 am

jorose wrote:
Mon Feb 25, 2019 10:32 pm
To make matters worse, you could easily imagine an engine which is roughly evenly bottlenecked by the GPU and CPU. How are you going to deal with that?
You could benchmark with the individual engine, and give each GPU-using engine its own time-odds factor depending on its speed on the standard and actual hardware.

henk2
Posts: 30
Joined: Mon Jan 14, 2019 6:55 am
Full name: Henk Verbaasdonk

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by henk2 » Tue Feb 26, 2019 8:06 am

konsolas wrote:
Mon Feb 25, 2019 10:02 pm
For the most part, I believe CCRL currently uses actual time controls closer to 40/20 rather than 40/40 due to the huge advancements made in CPU hardware.

A modern quad core at 40/20 would be comparable to a 1050 at 40/20 because even though both pieces of hardware are much newer than the Athlon 4600+, they both have half as much time to match.

The problem of different GPU models could potentially be solved with asymmetric time controls.

e.g. CCRL 40/40 could mean:
  • 40 moves in 40 minutes with an Athlon64 4600+, benchmarked with (some named) version of Crafty
  • 40 moves in 20 minutes with a GTX 1050, benchmarked with (some named) version of Lc0
CCRL is 40/20 with my i5-4460. (I benchmarked it)

I think it's closer to 40/15 or 40/12 with a current gen i5-9600.

User avatar
xr_a_y
Posts: 778
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by xr_a_y » Tue Feb 26, 2019 8:08 am

Where can I find the procedure to benchmark CCRL time control on my hardware ?

Modern Times
Posts: 2417
Joined: Thu Jun 07, 2012 9:02 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by Modern Times » Tue Feb 26, 2019 9:11 am

henk2 wrote:
Tue Feb 26, 2019 8:06 am


I think it's closer to 40/15 or 40/12 with a current gen i5-9600.
Our current machines range from 40/15 up to 40/30 I think.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

User avatar
Graham Banks
Posts: 33216
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by Graham Banks » Tue Feb 26, 2019 9:46 am

xr_a_y wrote:
Tue Feb 26, 2019 8:08 am
Where can I find the procedure to benchmark CCRL time control on my hardware ?
http://kirill-kryukov.com/chess/discuss ... f=7&t=1486
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

User avatar
xr_a_y
Posts: 778
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by xr_a_y » Tue Feb 26, 2019 1:54 pm

Ok thanks get this
Crafty 19.17 BH benchmark can be downloaded here. (Version 19.17, Brian Hoffman compile, 32-bit, single-CPU). Please note that we should use the same version and compile because different versions may be slightly faster or slower and will give different benchmark time.
but the link is a windows only version. I guess some CCRL members are testing under linux ?

User avatar
xr_a_y
Posts: 778
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by xr_a_y » Tue Feb 26, 2019 1:58 pm

I compiled it by hand, linux64 (so not really the same as CCRL) and get this

Code: Select all

Running benchmark. . .
......
Total nodes: 80680468
Raw nodes per second: 4482248
Total elapsed time: 18
SMP time-to-ply measurement: 35.555556
So I guess, 40/3 will be quite good to emulate CCRL 40/4 for me.

Post Reply