CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Graham Banks
Posts: 33252
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by Graham Banks » Sat Feb 23, 2019 11:26 pm

The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://computerchess.org.uk/ccrl/4040/ (40/40)
http://www.computerchess.org.uk/ccrl/404/ (40/4)
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)

Please note that the three lists are often updated separately to each other. The FRC list is only updated when a new engine or engine version is being/has been tested.
Also, please note that the 40/40 and 40/4 lists are constructed from totally different databases and not comparable to each other.

Thanks to all of our currently active testers (Graham, Ray, Tirsa, Gabor, Sergio, Andreas and Brent), plus to Chessdom.com for providing an online server to assist with resources.

40/40 testing this week that I'm aware of will include (with live broadcast port for TLCV noted where applicable):

73rd Amateur Series Division 1 Tournament (finishing Friday 16002)
73rd Amateur Series Division 2 Tournament (continuing 16001)
73rd Amateur Series Division 3 Tournament (starting Friday 16002)
Jumbo And The Monster Tournament (continuing 16053)
Asymptote 0.4.1 64-bit Gauntlet (continuing 16063)
Topple 0.3.5 64-bit Gauntlet (continuing 16064)
Jumbo 0.6.96 64-bit Gauntlet (continuing 16065)
SugaR 1.6 64-bit Gauntlet (finishing Thursday 16091)
Laser 1.7 64-bit Gauntlet (continuing 16092)
Ethereal 11.25 64-bit Gauntlet (continuing 16093)
RubiChess 1.3 64-bit Gauntlet (continuing 16094)
The Red Jewel 4CPU Tournament (continuing 16083)
Tongues Of Fire 4CPU Tournament (continuing 16084)
RofChade 2.0 64-bit 4CPU Gauntlet (being run by Tirsa)
Minic 0.40 64-bit Gauntlet (being run by Tirsa)
Barbarossa 0.5.0 64-bit Gauntlet (being run by Tirsa)
Various other 1CPU Gauntlets (being run by Tirsa)
Francesca MAD 0.21 64-bit Gauntlet (being run by Gabor)
Deuterium 2019.1.36.50 64-bit Gauntlet (being run by Gabor)
Various 1CPU Tournaments (being run by Sergio and Andreas)

40/4 testing since the last update report has included (thanks to Gabor and Sergio):

Laser 1.7 64-bit
Robocide 0.4 64-bit
Ethereal 11.25 64-bit 4CPU
Laser 1.7 64-bit 4CPU
Asymptote 0.4.2 64-bit
Bumblebee 1.0 64-bit
Minic 0.34 64-bit
RubiChess 1.3 64-bit
Franky 1.0 64-bit
Topple 0.3.4 64-bit
ECE-X3.5 64-bit
Various Tournaments (run by Sergio)

FRC 40/4 testing since the last update report has included (thanks to Ray):

Arminius 2018-12-23 64-bit
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

User avatar
Graham Banks
Posts: 33252
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by Graham Banks » Sat Feb 23, 2019 11:47 pm

Brent did some testing using the time ratio and if the results had been included, Lc0 GPU would slot into 3rd spot, below SF and Houdini on the CCRL 40/4 list.
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

Modern Times
Posts: 2421
Joined: Thu Jun 07, 2012 9:02 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by Modern Times » Sun Feb 24, 2019 6:32 am

I'm keen that we get GPU testing with Lc0 and others on our lists and I'm certain we will at some point. But is has to be done the right way, and we could not agree with Brent on the approach to take. So we wait a bit longer !

None of our current testers have decent GPUs, because in the past when building a chess machine you could choose a basic GPU and put the money into the CPU and the rest of the machine instead. Perhaps once we get into hardware replacement cycles that will be the time, and with luck GPU prices will come down as well.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Modern Times
Posts: 2421
Joined: Thu Jun 07, 2012 9:02 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by Modern Times » Sun Feb 24, 2019 7:02 am

On CEGT 40/4, comparing to 4CPU engines, Lc0 on GPU is in 4th place behind Stockfish, Houdini and Komodo. I'm not sure what GPU they use.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

konsolas
Posts: 182
Joined: Sun Jun 12, 2016 3:44 pm
Location: London
Full name: Vincent
Contact:

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by konsolas » Sun Feb 24, 2019 10:39 am

Possible suggestion:

The GTX 1050 has a low TDP doesn't use any additional PCI-E power connectors so it would probably be an easy upgrade for most of your existing systems (as long as they're not laptops).

In the UK, it's priced very similarly to a modern quad core (the i3 8100 costs £113, the 1050 costs £117), so it may be a fair comparison with CCRL 4CPU test conditions.

Modern Times
Posts: 2421
Joined: Thu Jun 07, 2012 9:02 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by Modern Times » Sun Feb 24, 2019 10:55 am

konsolas wrote:
Sun Feb 24, 2019 10:39 am
Possible suggestion:

The GTX 1050 has a low TDP doesn't use any additional PCI-E power connectors so it would probably be an easy upgrade for most of your existing systems (as long as they're not laptops).

In the UK, it's priced very similarly to a modern quad core (the i3 8100 costs £113, the 1050 costs £117), so it may be a fair comparison with CCRL 4CPU test conditions.
Thanks for the suggestion, much appreciated - so when you say "no additional connectors" does that mean none at all, it draws it all from the PCI-E slot ? Or it needs one connector ? It has been so long since I bought CPU components and built a machine I am completely clueless now. At least on this machine I'm pretty sure I have one connector from memory, 1x 6-pin (ATI Radeon HD 6850 from 2011 !!)

Edit: Ah I see: "Power Supply: Powered directly from the PCIe x16 slot, no additional PCIe connector necessary"

For the 1050 would you need a 2048Mb card or a 4096MB ?
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

konsolas
Posts: 182
Joined: Sun Jun 12, 2016 3:44 pm
Location: London
Full name: Vincent
Contact:

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by konsolas » Sun Feb 24, 2019 11:12 am

Modern Times wrote:
Sun Feb 24, 2019 10:55 am
konsolas wrote:
Sun Feb 24, 2019 10:39 am
Possible suggestion:

The GTX 1050 has a low TDP doesn't use any additional PCI-E power connectors so it would probably be an easy upgrade for most of your existing systems (as long as they're not laptops).

In the UK, it's priced very similarly to a modern quad core (the i3 8100 costs £113, the 1050 costs £117), so it may be a fair comparison with CCRL 4CPU test conditions.
Thanks for the suggestion, much appreciated - so when you say "no additional connectors" does that mean none at all, it draws it all from the PCI-E slot ? Or it needs one connector ? It has been so long since I bought CPU components and built a machine I am completely clueless now. At least on this machine I'm pretty sure I have one connector from memory, 1x 6-pin (ATI Radeon HD 6850 from 2011 !!)

For the 1050 would you need a 2048Mb card or a 4096MB ?
Yes, it uses no additional connectors at all and draws all of its power from the PCI-E slot. It only uses 75W. This shouldn't be a problem with most motherboards, but some PCI-E slots may have a reduced power limit.

The GTX 1050 has 2GB of GDDR5, although there is also a 3GB version, but that one does not appear to be widely available.

The 4GB model you're talking about is actually a 1050Ti, which is a bit faster and a bit more expensive at around £130 to £160 in my country.
The 1050Ti is the fastest GPU that NVIDIA sells (excluding the Quadros) which does not require additional power connectors. It also has a TDP of 75W.

Modern Times
Posts: 2421
Joined: Thu Jun 07, 2012 9:02 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by Modern Times » Sun Feb 24, 2019 11:40 am

Seriously tempted by this (1050) at those prices, whilst waiting for the higher performance cards to be more affordable. Do they work in PCIe 2.0 x16 slots, and does that affect performance ?

Edit: Sorry, googled this and yes it would run, and it seems with little or no performance effect with a card at this level
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Modern Times
Posts: 2421
Joined: Thu Jun 07, 2012 9:02 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by Modern Times » Mon Feb 25, 2019 8:21 pm

konsolas wrote:
Sun Feb 24, 2019 10:39 am
Possible suggestion:

The GTX 1050 has a low TDP doesn't use any additional PCI-E power connectors so it would probably be an easy upgrade for most of your existing systems (as long as they're not laptops).

In the UK, it's priced very similarly to a modern quad core (the i3 8100 costs £113, the 1050 costs £117), so it may be a fair comparison with CCRL 4CPU test conditions.
I ordered an ASUS GTX 1050 card today, £100 delivered. Given that our reference CPU is an ancient Athlon64 4600+, the GTX 1050 should be more than adequate and very fair. The good folks at CEGT, who are already doing GPU testing, have been very helpful and giving me advice on set-up and how they do their testing. Hopefully this weekend I'll be up and running with it.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

jorose
Posts: 269
Joined: Thu Jan 22, 2015 2:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: CCRL 40/40, 40/4 and FRC lists updated (23rd February 2019)

Post by jorose » Mon Feb 25, 2019 8:57 pm

Modern Times wrote:
Mon Feb 25, 2019 8:21 pm
konsolas wrote:
Sun Feb 24, 2019 10:39 am
Possible suggestion:

The GTX 1050 has a low TDP doesn't use any additional PCI-E power connectors so it would probably be an easy upgrade for most of your existing systems (as long as they're not laptops).

In the UK, it's priced very similarly to a modern quad core (the i3 8100 costs £113, the 1050 costs £117), so it may be a fair comparison with CCRL 4CPU test conditions.
I ordered an ASUS GTX 1050 card today, £100 delivered. Given that our reference CPU is an ancient Athlon64 4600+, the GTX 1050 should be more than adequate and very fair. The good folks at CEGT, who are already doing GPU testing, have been very helpful and giving me advice on set-up and how they do their testing. Hopefully this weekend I'll be up and running with it.
In general I think it's great that you guys want to test engines with GPU, but there are some issues that I think should at least be discussed.
Are you intending to only ever have a single GPU model you test with, or how would you rescale to ensure consistency between testers? If both CPU and GPU differ between testers, how would you rescale between them?

Also, by what metric would the GTX 1050 be in any way a fair comparison to the 13 year old reference CPU? I understand that if we used GPUs from 2006 then basically there wouldn't be a benefit at all (no real NN support), but maybe it is time to upgrade the reference CPU in that case?

Im aware that these may not be solvable problems, but to me that is a sign that perhaps a new list is needed, not that the issues should get ignored.
-Jonathan

Post Reply