Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 8:44 pm
Location: Amman,Jordan

Re: Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Post by Dr.Wael Deeb »

beram wrote:It is a great effort and a great list, the CCRL list(s), no doubt about that.
People who are still asking or hammering about even longer LTC, should realise that what we call now blitzTC is LongTC in 10 years
The level of play of the games now played at 40/40 TC is incredibly high, with average plydepths in middlegame that for 10 years ago really no one could imagine.
What I want to make clear is that it really makes no sense to compare engine games TC's with human games TC's. It is far fetched and illogical.
Well said regards,
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Post by ouachita »

beram wrote:What I want to make clear is that it really makes no sense to compare engine games TC's with human games TC's. It is far fetched and illogical.
That's exactly my point. All time controls referred to on this site are engine "testing" time controls, not engine game "playing" time controls. The moves made by engines at popular testing time controls (Bullet 1m+1s, Blitz 3+3 or 4+2, etc.) will most often not be the moves made in correspondence games at game LTC.
SIM, PhD, MBA, PE
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Post by ouachita »

Said differently, the results of this or any engine test are only relevant to that particular test, and cannot be interpolated or extrapolated with any degree of certainty. For example, here's a set of recent engine (K, H and SF) test conditions and criteria:

SYSTEM---i3 M380|2 Cores@2.53|Win7(64bit)|6GB RAM
FMARK-----Speed 4.82/Kns 2313
TB'S-------None
PONDER----Off
GUI--------Deep Fritz 14(64 Bit)
HASH------256 MB
BOOK------SalvoSuite50 (8 move reversing)
TIME------4 min + 2 sec

Whatever the test results were or may be, they cannot be interpolated back to hypothetical, i.e., 1+2 results, or extrapolated forward to 40/120 or 40/4000 or any other results. These 4+2 test results are (only) what they are.

Engine test results also seem to be of greater significance to the tester than to objective observers, other testers and game players.
SIM, PhD, MBA, PE
lkaufman
Posts: 6284
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Post by lkaufman »

beram wrote:It is a great effort and a great list, the CCRL list(s), no doubt about that.
People who are still asking or hammering about even longer LTC, should realise that what we call now blitzTC is LongTC in 10 years
The level of play of the games now played at 40/40 TC is incredibly high, with average plydepths in middlegame that for 10 years ago really no one could imagine.
What I want to make clear is that it really makes no sense to compare engine games TC's with human games TC's. It is far fetched and illogical.
I don't agree with this. Comparison to old computers is irrelevant. Chess programs are bought for use by human players, and the pace at which they use the machines is not related to the speed of the hardware. For watching engine games or for quick review of one's own games, the relevant time limit is probably a minimum of 5' + 3" or so, while for serious analysis of openings or for overnight review of a game even 90' + 30" is a bit fast. For use in correspondence play we really need results at much longer time controls yet, but that is impractical. Engine strength should be measured at similar time controls to the way they will be used by human players.
PaulieD
Posts: 242
Joined: Tue Jun 25, 2013 8:19 pm

Re: Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Post by PaulieD »

It is interesting that the author's who's engines are only good at very LTC speak primarily of that and how "it" is the primary way in chess. The author's who's engines are better at blitz speak of how time controls are changing and getting faster and faster (true) and that is why their engine is best.

The top three are so close, depending on the time control selected you can have a different winner. When H3 was #1 in the beginning, it was totally dominant at all time controls which eliminated a lot of this bantering.

Time controls and life are getting faster and faster.
PaulieD
Posts: 242
Joined: Tue Jun 25, 2013 8:19 pm

Re: Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Post by PaulieD »

ouachita wrote:Said differently, the results of this or any engine test are only relevant to that particular test, and cannot be interpolated or extrapolated with any degree of certainty. For example, here's a set of recent engine (K, H and SF) test conditions and criteria:

SYSTEM---i3 M380|2 Cores@2.53|Win7(64bit)|6GB RAM
FMARK-----Speed 4.82/Kns 2313
TB'S-------None
PONDER----Off
GUI--------Deep Fritz 14(64 Bit)
HASH------256 MB
BOOK------SalvoSuite50 (8 move reversing)
TIME------4 min + 2 sec

Whatever the test results were or may be, they cannot be interpolated back to hypothetical, i.e., 1+2 results, or extrapolated forward to 40/120 or 40/4000 or any other results. These 4+2 test results are (only) what they are.

Engine test results also seem to be of greater significance to the tester than to objective observers, other testers and game players.
This could be said about every time control in existence, there are so many variables in time that, of course, no "set" time control speaks for all other time controls. But correspondence chess is not the standard that engines should be getting measured at, it as not as large a part of chess as "time controlled" chess is and is a very limited "view".
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Post by ouachita »

lkaufman wrote:Engine strength should be measured at similar time controls to the way they will be used by human players.
Agree 100%. I have assumed all along that engines are tested primarily to be used in actual games. Thus, Condition A, Test, should be ~ the same as Condition B, Game.

For example, a STC engine test at 2+2 would be most relevant and probably of most interest to players in events with the exact or similar hardware, settings and time controls. However, there's no scientific reason to believe that such a test is relevant with any degree of precision to shorter TC or LTC such as 90+30, 40/120+30, or 40/4000, etc.
SIM, PhD, MBA, PE
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Post by ouachita »

PaulieD wrote: correspondence chess is not the standard that engines should be getting measured at, it as not as large a part of chess as "time controlled" chess is and is a very limited "view".
LTC (>=90+) testing on the fastest hardware (6, 16, 24, or 32 cores) available is the best standard that engines should be measured for my needs.

I am primarily concerned abouting winning chess games, rather than test results.

Also, "banter" is presumably what this site is here for.
SIM, PhD, MBA, PE
PaulieD
Posts: 242
Joined: Tue Jun 25, 2013 8:19 pm

Re: Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Post by PaulieD »

The problem with that admirable standard is that it takes years to get enough data to say anything with certainty, especially now with how close the top 3 are. Hence the compromise between the extreme quality of long time control and the volume of data available at lesser time controls. That is why there are so very many rating lists with various time controls, to attempt to provide data to satisfy all players needs or desires.
Congrats on your IM achievement BTW.
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: Komodo TCEC 64-bit Gauntlet for CCRL 40/40

Post by ouachita »

PaulieD wrote:. . . the author's who's engines are only good at very LTC speak primarily of that and how "it" is the primary way in chess. . .

The top three are so close . . .
These two statements are inconsistent, unreasonable and unfair, because, as you have said, "The top three are so close". Indeed, they are reasonably close at all TC.

Another expression of the practical relevance of testing is to run a 50 game match/test with white at 1+1 TC and black at 40/120. What would you guess the cross table will look like?

And . . . thanks.
SIM, PhD, MBA, PE