CCRL adapted TC to current hardware

[Guenther]

It seems I missed this change or was it reported before?
40/40 => 40/15 and 40/4 => 40/2
(The links itself are not yet changed though)

This reflects more the real tc played on newer hardware - I would still welcome adding the real tc always
in the pgn header and not just the generic one, at least for the future, as it is probably impossible w/o huge work
to have it for past games too.

http://ccrl.chessdom.com/ccrl/404/rating_list_all.html

[Modern Times]

It was done yesterday, I think one of us was going to post it.

Years ago CEGT re-named their 40/40 list to 40/20 and we've been thinking of doing that ever since then, and talking about it internally on and off, but only got around to it now. In the interim we had put a comment on the main page.

No the URLs have not changed and that is deliberate, so that existing bookmarks and favourites out there continue to work.

I suspect we'll have to re-name it again in the not too distant future, but now we know how to do it (several configuration files and templates had to be changed) it won't be an issue. It now represents "more current" hardware rather than "current" hardware, as there are faster CPUs out there, but it represents what we currently have.

We'll also change the benchmark software to something like Stockfish 10 instead of the old Crafty version used currently.

[Alayan]

I noticed it yesterday. In a way, I find it nice as it's clearer for a less informed audience that might not pay attention to the benchmark CPU. A 4770K is not as fast as the latest CPUs, but it's still faster than the average CPU out there with laptops and budget desktops, I don't think the reference would require another update until at least 2 or 3 CPU generations.

But I'm puzzled by how 40/40 becomes 40/15 and 40/4 becomes 40/2. The ratio between both lists go from 10:1 to 7.5:1. What's the thought process behind this ?

[Gabor Szots]

But I'm puzzled by how 40/40 becomes 40/15 and 40/4 becomes 40/2. The ratio between both lists go from 10:1 to 7.5:1. What's the thought process behind this ?

The reason is that very few GUI's support fraction of a minute time controls. I myself use Arena, Fritz 17 and Shredder 13 and in neither of them can I select 40 moves in 1 minute 30 seconds.

[Modern Times]

Yes, with the blitz time control there has always been rounding even beforehand - say your machine benchmarked as 40 moves in 3.5 minutes - with most GUis you had to choose 40/3 or 40/4. But also even if you could round cleanly, i.e. 3.3 down to 3, you were still playing slightly short. So the time control has always been approximated anyway.

[Alayan]

Thanks for the explanations, both of you.

When I do some limited engine testing/tournaments, I'm using cutechess-cli which allows me to even use fractions of seconds, so I wasn't thinking in terms of GUI limitations. Having to stick with full minutes for base time is a massive limitation. If I was a tester, that would motivate me to try and use something else.

This actually makes the CPU benchmarking process close to meaningless for the blitz (bullet ?) list, because between 1m30s and 2m30s (assuming rounding to closest) you have a 67% TC increase, which at this short of a TC can noticeably impact relative performance.

My understanding is that CCRL is sticking to the old repeating time control (which at equal total time used leads to lower quality compared to standard increment, or another way to put it is it's reducing sample size by limiting the number of games) for historical compatibility reasons. But would actually switching to increment TC break it so much more than what is already induced with hardware variability ?

I have another question while I'm at it : how is TC scaled when GPU engines are playing ?

The list says e.g. a RTX 2080 is used, but there is no information on the TC math. The CPU benchmark is a terrible measure here. If tester A uses 2 minutes and tester B uses 3 minutes, for example, but both use the same GPU, the tester with the slower CPU that uses 3 minutes will actually allocate more GPU time to the GPU engine and have it score much better relatively to results on tester A's box.

I think this ought to be clarified.

[Modern Times]

GPU testing is a minefield, but only two testers have done GPU testing and both have different GPUs, so the situation you describe doesn't exist (yet). Now there is only one tester, who has the RTX 2080, so at the moment life is even easier. In the future there will be difficult issues to resolve though.

In terms of the blitz CPU time control, It is a necessary compromise when different testers have different hardware and use different GUIs. Here, the individual lists like Stefan Pohl's have an advantage. But frankly in the world of CPU engines the vast majority of engines perform the same at short and longer time controls so I don't believe it distorts results significantly. The purpose of the blitz list anyway is just to accumulate games quickly as an approximation of strength.

[Guenther]

Thanks for the explanations, both of you.

...

My understanding is that CCRL is sticking to the old repeating time control (which at equal total time used leads to lower quality compared to standard increment,
...

I see it exactly the opposite, usually inc games lead to low quality in the endgame, or even long before, if a game lasts much longer
than 40 or 50 moves. IMO it is ugly to have a game finally decided by the low inc for dozens of moves.

[Alayan]

I see it exactly the opposite, usually inc games lead to low quality in the endgame, or even long before, if a game lasts much longer
than 40 or 50 moves. IMO it is ugly to have a game finally decided by the low inc for dozens of moves.

Then, use higher increment, and maybe base time too ? My claim is for equal average time per game, if you just use the repeating base time as the base time for base+inc, you typically shorten significantly the average game duration.

Repeating time control induce "time scramble" every 40 moves, and you'll find plenty of blunders in moves 30 to 40 that wouldn't have happened with the base+increment TC.

In human play, one can find similar complaints about increment and endgame quality, and they mostly boil down to complaint against TCs being often shorter nowadays, while happily disregarding time-blunders from moves 30 to 40.

Of course, another point is that using equal time for all moves leads to lower quality games - you can increase the endgame quality with an even bigger increment, but if you compensate with a lower base time to keep the same average game length, you will lower opening/midgame quality, and lower overall quality. But even with a high increment/base ratio, you'll get better quality than with repeating.

By the way, I'm not using quality as an abstract concept. You can match up engines with asymmetric time controls with cutechess, so my claims are testable by doing a tournament with an engine (or several) using different settings. One could of course argue that an engine has bad TM for one given setting limits the method's usefulness, but with several engines this shouldn't be a major issue.

[Modern Times]

Repeating time control is a historical thing - back in the day many engines did not support Fischer time control - so it assured maximum compatibility. But not many of us are a fan of it any more, including me. Short of abandoning the current list and starting again, I'm not sure what we can do about it. Certainly we don't want more than two lists, that is difficult enough as it is.