TCEC has no rules regarding engine uniqueness or data training

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Madeleine Birchfield
Posts: 76
Joined: Tue Sep 29, 2020 2:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

TCEC has no rules regarding engine uniqueness or data training

Post by Madeleine Birchfield » Sun Oct 11, 2020 12:54 pm

Here are the rules for the upcoming TCEC Cup 7:
1. Event

a. The TCEC Cup 7 is a knockout format played in TCEC, including all qualifying participants.
b. The 32 qualifying engines consist of the top 32 ending participants of TCEC Season 19, dropping those that are out of apparent development.
c. These engines will be seeded according to their respective final results in TCEC Season 19's final standings.
d. The winner of TCEC Cup 6 will be seeded number 1.

2. TCEC Cup knockout matches format

a. The TCEC Cup 7 will take place after the TCEC Season 19 Superfinal.
b. It will be a knockout championship with the 32 participants divided into brackets.
c. Each participant will be equal-distance seeded according to its final standing in the TCEC league event with the #1 seed meeting the #17 seed, #2 meeting #18, etc.
d. The TCEC Cup 7 will consist of a total of 32 matches divided into 5 rounds: sixteenth-finals, eighth-finals, quarter-finals, semi-finals and final.
e. For an engine to win the TCEC Cup gold medal it must win a total of five matches. The two losing semifinalists will play a match for 3rd and 4th place (`small final’).

3. Matches and tiebreaks

a. Each of the matches will initially consist of 2 pairs of games (4 games, every second with reversed colors and the same opening). Cutechess-cli will adjudicate the match if a decisive result has been reached within 4 games.
b. The lower seeded engine will start playing with the white pieces.
c. In case of an equal score after these 4 games, tiebreakers will be played out at the end of the Cup round. Tiebreakers consist of additional game-pairs; a decisive pair of tiebreaker games decides a match.
d. If a match is tied after its scheduled regular 4 games, more game-pairs will be played until there is a decisive game-pair result.
e. Whether a game is immediately restarted or played rather at the end of the round will be decided by the responsible TCEC staff.

4. Openings books

a. All matches are played with a book exit from one of the five books in use: A through E.
b. No bookless games will be played in the TCEC Cup.
c. All games take place in pairs in which both engines each play each side of every book exit. Engines will play both sides of each position selected. Openings chosen will roughly approximate human games in terms of their frequency. A new opening should thus occur every other game, then the reverse game is played, so two games in a row from the same starting book position. Because each engine will thus play both sides of every opening, fairness is guaranteed.
d. BOOK A created by Nelson Hernandez, consisting of two-move (four-ply) exits, used in the first round (round of 32).
e. BOOK B created by Nelson Hernandez, consisting of four-move (eight-ply) exits, used in the second round(round of 16).
f. BOOK C created by Nelson Hernandez, consisting of six-move (twelve-ply) exits, used in the quarter finals.
g. BOOK D created by Nelson Hernandez, consisting of eight-move (sixteen-ply) exits. This book is used in the semi-final rounds.
h. BOOK E is a compilation of over 400 positions chosen for Season 9-18 Superfinals by Jeroen Noomen. This book will be used in the Cup final and third-place matches.
i. All five books will be used randomized.

5. Time control

a. Time control will be 30 mins + 5 secs increment per move for all games in a match, including the tiebreakers.
b. If an engine loses on time, that results in a loss and will be registered as such. The game will not be replayed.
c. Whether a game is immediately restarted or played rather at the end of the round will be decided by the responsible TCEC staff.

6. Game adjudication

a. A game can be drawn by the normal 3-fold repetition rule or the 50-move rule.
b. A game can also be drawn at move 35 or later if the eval from both playing engines are within +0.15 to -0.15 pawns for the last 5 moves, or 10 plies. If there is a pawn advance, or a capture of any kind, this draw rule will reset and start over. In the website this rule is shown as "TCEC draw rule" with a number indicating how many plies there are left until it kicks in.
c. Cutechess-cli will adjudicate 6-men or less endgame positions automatically with Syzygy tablebases.

7. Crashes

a. Engine crashes are counted as losses.
b. If an engine loses on time, the result will not be changed nor the game replayed.
c. In case of a server disconnect, or time loss due to lag, or other interruptions not caused by the engines there are five possible scenarios:

i. If the web server crashes, the game continues unaffected, and broadcasting will be resumed as soon as possible;
ii. If the evaluation of both engines is more than ABS(10) at the moment of game server interruption, the game is scored as a win;
iii. If the evaluation of both engines is less than ABS(0.10) at the moment of game server interruption, and both engines have completed 35 moves, the game is scored as a draw;
iv. If a game interrupts with 7 pieces on the board, position on the board at the instant of game server interruption will be adjudicated according to 7-man EGTB;
v. In all other cases the game is restarted from the position that the two engines reached before the disconnect, with time compensation to fill up the cache.

8. Engine Updates

a. Under no circumstances are updates and fixes to engines allowed once the Cup competition has started.
and here are the rules for the upcoming TCEC Season 20:
1. Season

The Top Chess Engine Championship (TCEC) Season 20 is the premier championship for chess software.
TCEC Season 20 is a tournament, between engines with ELO 3000+, and it is divided into 6 Events, consisting of four Leagues, the Premier Division and the Superfinal.
Each of the Leagues, the Premier Division and the Superfinal, is hereafter called: an Event.
TCEC Season 20 runs 24/7 until all its games have been played. One game is played at a time - the next one starts automatically.
As soon as an Event starts, it will run 24/7 until all its games have been played. One game is played at a time - the next one starts automatically.
There will be a short break between the Events, to test and make sure everything is ok with the TCEC game server and to prepare for the next Event.

2. Qualification League

The Qualification League is the first Event of TCEC Season 20.
For the Qualification League Event, engines will be invited by the organizers.
The Qualification League or QL consists of 12 engines recently submitted by their authors, all of which are under active development.
It is a 1x double round robin at TC 30+5 with a randomized 2-move opening book by Nelson Hernandez (Book A).
The top 2 engines promote to League 3.

3. League 3

The League 3 is the second Event of TCEC Season 20.
League 3 or L3 consists of 10 engines including the 2 promotions from QL.
It is a 1x double round robin at TC 30+5 with a randomized 4-move opening book by Nelson Hernandez (Book B).
The bottom 2 engines relegate and the top 2 promote to League 2.

4. League 2

The League 2 is the third Event of TCEC Season 20.
League 2 or L2 consists of 10 engines including the 2 promotions from L3.
It is a 1x double round robin at TC 30+5 with a consecutive various depth opening book by Eduardo Sauceda (Book C).
The bottom 2 engines relegate and the top 2 promote to League 1.

5. League 1

The League 1 is the fourth Event of TCEC Season 20.
League 1 or L1 consists of 10 engines including the 2 promotions from L2.
It is a 1x double round robin at TC 45+5 with a randomized 8-move opening book by Nelson Hernandez (Book D).
The bottom 2 engines relegate and the top 2 promote to Premier Division.

6. Premier Division

The Premier Division is the fifth Event of TCEC Season 20.
Premier Division or DivP consists of 8 engines including the 2 promotions from L1.
It is a 3x double round robin at TC 90+10 with a randomized 8-move opening book by Nelson Hernandez (Book E).
The top 2 engines promote to the Superfinal and the bottom 2 engines are relegated to League 1.

8. Superfinal

The Superfinal consists of a 2 engine 50DRR, 100 games at TC 120+10, with a consecutive various depth opening book by Jeroen Noomen (Book F).
If the match is theoretically won for one side before game 100, the match will still continue until all 100 games have been played.
In the case of a drawn match there will be a rapid match of 16 games with a time control of 25' + 10" with random openings selected from earlier in the same Season.
In case it is still tied there will be sets of Blitz matches of 8 games each, with a time control of 3' + 2" until a winner is found.
When the Superfinal is over, the current Season ends.

9. The TCEC Grand Champion

The winner of the Superfinal will be crowned the TCEC Grand Champion and will keep this title until there is a winner in the next Superfinal.
There is no automatic qualification for the reigning Grand Champion, it will have to go all the way through the Premier Division of the next Season for it to be able to defend the title.

10. Time control

Different classical time controls will be used throughout the season and the time control is increased as the season progresses.
For the Qualification League, League 3 and League 2 Events, the time control is 30 minutes + 5 seconds added per move for the whole game.
For League 1 Event, the time control is 45 minutes + 5 seconds added per move for the whole game.
For Premier Division, the time control is 90 minutes + 10 seconds added per move for the whole game.
For the Superfinal, the time control is 120 minutes + 10 seconds added per move for the whole game.

11. Game adjudication

A game can be drawn by the normal 3-fold repetition rule or the 50-move rule.
However, a game can also be drawn at move 35 or later if the eval from both playing engines are within +0.15 to -0.15 pawns for the last 5 moves, or 10 plies. If there is a pawn advance, or a capture of any kind, this special draw rule will reset and start over. On the GUI interface, this rule is shown as "TCEC draw rule" with a number indicating how many plies there are left until it becomes official.
A game will be adjudicated as won for one side if both playing engines have an eval of at least 10.00 pawns (or -10.00 in case of a black win) for 5 consecutive moves, or 10 plies - this rule is in effect as soon as the game starts. On the GUI interface, this rule is shown as "TCEC win rule" with a number indicating how many plies there are left until it becomes official.
Cutechess will automatically adjudicate endgame positions with 6-men Syzygy tablebases.

12. Tiebreaks

If necessary, tiebreaks shall be used to determine advancement, based on the final standings of a League or Division. For all Events except the Superfinal, the following criteria will be used, in the order as presented below:
In case of engines being tied, then the direct encounter(s) between the tied engines decides first.
r-mobility tiebreak is the second criterion.
The third criterion is the greatest number of wins.
The Sonneborn-Berger is the fourth criterion.
In the unlikely event they are even then still tied, then the tournament organizers will decide which engine gets the promotion or relegation.

13. Engine Updates

All engines that promote to the next Event are allowed to update their engine, engine parts, and/or settings before the next Event's submission deadline.
Except when promoting from QL to L3 and from L2 to L1, all engines that promote to the next Event are allowed to update before the Event's submission deadline, no matter if it is to fix bugs uncovered during the previous Event, or to submit a better version. To summarize:
No testing will be done and provided, nor is updating allowed, between Qualification League and League 3.
No testing will be done and provided, nor is updating allowed, between League 2 and League 1.
Testing will be done and provided, and updating allowed, between League 3 and League 2.
Testing will be done and provided, and updating allowed, between League 1 and Premier Division.
Testing will be done and provided, and updating allowed, between Premier Division and Superfinal.
The deadline for engine submission is the last game of the current Event unless the programmer is given a specific deadline from the Tournament Director - the goal is to be able to start the next Event as soon as possible without any significant delay.
Note that despite testing between certain Events will be provided, engine authors update at their own risk.
Once an Event is being played, no updates are allowed during it.

14. Engine Ratings

The TCEC engine ratings can be found at https://tcec-chess.com/bayeselo.txt
This list is updated live after every official game and includes all games, including rapid or faster time control games from current and previous season.
Most testing games with sufficiently long time control are also used.
Version numbers are ignored and all games are included, including games lost on time or due to crash.
New engines are assigned a temporary rating based on testing, until an official rating can be calculated after they played in an Event.

15. Crashes

If an engine loses on time, the result will not be changed or the game replayed.
In case of a server disconnect, or time loss due to lag, or other interruptions not caused by the engines there are five possible scenarios:
If the web server crashes, the game continues unaffected, and broadcasting will be resumed as soon as possible;
If at the moment of game server interruption the evaluation of both engines is more than +10, or if the evaluation of both engines is less than -10, the game is scored as a win;
If the evaluation of both engines is less than ABS(0.15) at the moment of game server interruption, and both engines have completed 35 moves, the game is scored as a draw;
If a game interrupts with 7 pieces on the board, position on the board at the instant of game server interruption will be adjudicated according to 7-man EGTB;
In all other cases the game is restarted from the position that the two engines reached before the disconnect, with time compensation to fill up the cache.
An engine that crashes 3 times during an Event is disqualified and all its game results (points) will be annulled, also for its opponents.
In case of disqualification crashes are counted for all engines until the end of an Event.
Crash counting is only reset at the next Event.
Crashes during the Superfinal are counted as losses.
The TCEC cup rules only require that the engines have participated in the previous TCEC season, while the regular TCEC season rules specifically mention that engines will be invited by the organizers, making TCEC an invitational. The only requirement for all engines at TCEC mentioned in the rules is that they have to be strong enough, i.e. above 3000 elo. There is no mention of engine uniqueness or data training in any of the rules for TCEC events whatsoever.

The supposed NNUE 'rules' that TCEC came up with that everybody has been fighting over the past few days are
1. only applicable for the TCEC Cup 7, and
2. guidelines, not rules, which means they are not binding.

Post Reply