MCERL (Mac Chess Engines Rating List) September 25th 2022

[Dariusz]

Link: https://www.chessengeria.com/mcerl-2022-09-25

I would like to extend warm thanks to enthusiasts of this topic such as Alessandro Morales for his many excellent compilations of chess engines, and Stefan Pohl and Thomas Plaschke for providing a tool called: Sacrifice Games Search and Statistics Tool on the Stefan Pohl Computer Chess website.

Gentlemen, thank you sincerely for the excellent fruits of your work !

[yeni_sekme]

Thanks for the rating list. 1 minute + 0 seconds is very odd TC. I'm suggesting you to use 60 seconds + 0.6 seconds increment for each move

[Dariusz]

Thanks for the rating list. 1 minute + 0 seconds is very odd TC. I'm suggesting you to use 60 seconds + 0.6 seconds increment for each move

Thank you for your suggestion. I will take it under consideration for the next edition of MCERL.

[lkaufman]

Thanks for the rating list. 1 minute + 0 seconds is very odd TC. I'm suggesting you to use 60 seconds + 0.6 seconds increment for each move

Thank you for your suggestion. I will take it under consideration for the next edition of MCERL.

Yes, many engines are apt to time forfeit some games with 1 minute and zero increment, especially with default parameter settings. Your ratings look generally reasonable, so I'm not saying this was a major issue, but I imagine it at least had some impact. Even if they don't forfeit, many engines will just do 3 ply searches or something trivial like that after a hundred moves or so under these conditions, so the results get randomized.

[Dariusz]

Thanks for the rating list. 1 minute + 0 seconds is very odd TC. I'm suggesting you to use 60 seconds + 0.6 seconds increment for each move

Thank you for your suggestion. I will take it under consideration for the next edition of MCERL.

Yes, many engines are apt to time forfeit some games with 1 minute and zero increment, especially with default parameter settings. Your ratings look generally reasonable, so I'm not saying this was a major issue, but I imagine it at least had some impact. Even if they don't forfeit, many engines will just do 3 ply searches or something trivial like that after a hundred moves or so under these conditions, so the results get randomized.

Larry, thank you very much for your constructive feedback.
In the next MCERL publication, the engines will be tested with the formula 60 seconds + 0.6 seconds increment for each move.

[kranium]

Let's say for purposes of argument, that the average engine vs engine game is 60 moves per side (120 ply).
That means that with a TC base of 60 secs., each engine would play a move approx. every 1 sec.

If however, the game ran more than 60 moves, and an engine had exhausted it's 60 secs. base time...it would be forced to play the remainder of the game on increment only (this being 0.60 sec. /move). The pace of play would increase into a noticeable (increment only) time scramble, and possibly lower quality of play.

However, with a 1 sec. increment the game would/should continue until end in a smooth fashion.
So it seems reasonable to conclude that 60 sec + 1 sec is an improvement to 60 secs + 0.60 sec. (IF the average moves per game is 60).

If however, the average game is 100 moves per side (200 ply), then 60 secs + .60 sec would be ideal.

[lkaufman]

Let's say for purposes of argument, that the average engine vs engine game is 60 moves per side (120 ply).
That means that with a TC base of 60 secs., each engine would play a move approx. every 1 sec.

If however, the game ran more than 60 moves, and an engine had exhausted it's 60 secs. base time...it would be forced to play the remainder of the game on increment only (this being 0.60 sec. /move). The pace of play would increase into a noticeable (increment only) time scramble, and possibly lower quality of play.

However, with a 1 sec. increment the game would/should continue until end in a smooth fashion.
So it seems reasonable to conclude that 60 sec + 1 sec is an improvement to 60 secs + 0.60 sec. (IF the average moves per game is 60).

If however, the average game is 100 moves per side (200 ply), then 60 secs + .60 sec would be ideal.

That would only be valid if the increment is added starting at move 60 (in your example). This is occasionally done in human tournaments, never to my knowledge in engine tournaments or rating lists. With increment added from the start, a smart engine will use it, so it will always have to play faster once main time is exhausted after the 60 moves or so. This is actually desirable, since there is a good chance that the game may already be effectively decided by move 60. Obviously a longer increment will produce better play, but the question is what base to increment ratio produces the best play for a given average game duration. This cannot be answered by abstract reasoning, only by testing. Past testing in Komodo puts the ideal ratio at somewhere in the 100 to 200 range, but it will of course depend on the time control and specific engine. With very short time controls like one minute, where considerations of GUI overhead and startup time per move may be an issue, I think that the 100 to 1 ratio is a good practical compromise, and is the standard for many rating lists.

[kranium]

Let's say for purposes of argument, that the average engine vs engine game is 60 moves per side (120 ply).
That means that with a TC base of 60 secs., each engine would play a move approx. every 1 sec.

If however, the game ran more than 60 moves, and an engine had exhausted it's 60 secs. base time...it would be forced to play the remainder of the game on increment only (this being 0.60 sec. /move). The pace of play would increase into a noticeable (increment only) time scramble, and possibly lower quality of play.

However, with a 1 sec. increment the game would/should continue until end in a smooth fashion.
So it seems reasonable to conclude that 60 sec + 1 sec is an improvement to 60 secs + 0.60 sec. (IF the average moves per game is 60).

If however, the average game is 100 moves per side (200 ply), then 60 secs + .60 sec would be ideal.

That would only be valid if the increment is added starting at move 60 (in your example). This is occasionally done in human tournaments, never to my knowledge in engine tournaments or rating lists. With increment added from the start, a smart engine will use it, so it will always have to play faster once main time is exhausted after the 60 moves or so. This is actually desirable, since there is a good chance that the game may already be effectively decided by move 60. Obviously a longer increment will produce better play, but the question is what base to increment ratio produces the best play for a given average game duration. This cannot be answered by abstract reasoning, only by testing. Past testing in Komodo puts the ideal ratio at somewhere in the 100 to 200 range, but it will of course depend on the time control and specific engine. With very short time controls like one minute, where considerations of GUI overhead and startup time per move may be an issue, I think that the 100 to 1 ratio is a good practical compromise, and is the standard for many rating lists.

Of course it does depend on the exact ratios used in the individual engine's TC implementation. Many will still have base time left after 60 or even 100 moves and never reach a situation where it's running on 'empty' (increment only). I think there must exist an ideal ratio, something empirical yet to be demonstrated, where an engine could get an ideally consistent amount of time for each move for the entire game. Many of the TCs I see being utilized are unsystematic. For ex CCRL blitz is now 2'+1" (ratio 120), which in a 60 move game would be 2 secs per move, but only 1 sec per move when running increment only...a significant difference. 2'+2" (ratio 60) just make more sense to me...perhaps it's the 60 secs factor. It's easy to calculate and remember. When extrapolated, you get the pleasing TCs: 1'+1", 2'+2", 3'+3", 4'+'4", etc.

[lkaufman]

Let's say for purposes of argument, that the average engine vs engine game is 60 moves per side (120 ply).
That means that with a TC base of 60 secs., each engine would play a move approx. every 1 sec.

If however, the game ran more than 60 moves, and an engine had exhausted it's 60 secs. base time...it would be forced to play the remainder of the game on increment only (this being 0.60 sec. /move). The pace of play would increase into a noticeable (increment only) time scramble, and possibly lower quality of play.

However, with a 1 sec. increment the game would/should continue until end in a smooth fashion.
So it seems reasonable to conclude that 60 sec + 1 sec is an improvement to 60 secs + 0.60 sec. (IF the average moves per game is 60).

If however, the average game is 100 moves per side (200 ply), then 60 secs + .60 sec would be ideal.

That would only be valid if the increment is added starting at move 60 (in your example). This is occasionally done in human tournaments, never to my knowledge in engine tournaments or rating lists. With increment added from the start, a smart engine will use it, so it will always have to play faster once main time is exhausted after the 60 moves or so. This is actually desirable, since there is a good chance that the game may already be effectively decided by move 60. Obviously a longer increment will produce better play, but the question is what base to increment ratio produces the best play for a given average game duration. This cannot be answered by abstract reasoning, only by testing. Past testing in Komodo puts the ideal ratio at somewhere in the 100 to 200 range, but it will of course depend on the time control and specific engine. With very short time controls like one minute, where considerations of GUI overhead and startup time per move may be an issue, I think that the 100 to 1 ratio is a good practical compromise, and is the standard for many rating lists.

Of course it does depend on the exact ratios used in the individual engine's TC implementation. Many will still have base time left after 60 or even 100 moves and never reach a situation where it's running on 'empty' (increment only). I think there must exist an ideal ratio, something empirical yet to be demonstrated, where an engine could get an ideally consistent amount of time for each move for the entire game. Many of the TCs I see being utilized are unsystematic. For ex CCRL blitz is now 2'+1" (ratio 120), which in a 60 move game would be 2 secs per move, but only 1 sec per move when running increment only...a significant difference. 2'+2" (ratio 60) just make more sense to me...perhaps it's the 60 secs factor. It's easy to calculate and remember. When extrapolated, you get the pleasing TCs: 1'+1", 2'+2", 3'+3", 4'+'4", etc.

In my opinion the CCRL blitz time limit of 2' + 1" is just about ideal, in terms of maximizing the quality of play for a given amount of average time per game. If anyone want to test this, just run matches of Stockfish vs Stockfish at 2' + 1" vs some 60 to 1 ratio that is about the same on average (1.5' + 1.5" would be equal time for a 60 move game, but the average SF vs SF game will be longer than 60 moves, so you might have to experiment a bit to get the average time per game to be equal). I am reasonably certain that 2' + 1" will defeat whatever 60 to 1 ratio takes the same average time. 2' + 1" also is very easy to double, triple, etc. As compared with repeating time controls like 40 moves per minute, it is vastly more efficient.