Cursed win at TCEC

[elpapa]

There's simply no room for discussion.

And yet here we are on page eight.

[mwyoung]

Because I said dubious opens. As in not sound.

I still don't see why you think that would skew the results, if each engine has to play both sides. In the worst case the start position is a certain win, and then they would just get a 1-1 on that line, and it is as if the game was never played, and there were just fewer games.

And I don't see why you don't think that it does skew results.

And the games are played and do count in TCEC.

Lets make an absurd example to show the point. I play GM Carlsen a 6 game match with 6 fixed and very unsound openings. So unsound I am able to win with white every game. And so is GM Carlsen.

You would say that is fair. Each side played the same openings and that does not skew results.

But I scored 3 wins and drew the match against GM Carlsen. Hmmm.

If GM Carlsen played those 6 fixed openings against GM Caruana the results would be the same. 3 wins for GM Caruana and 3 wins for GM Carlsen. And you would say that is to be expected anyway, and that is fair they played the same openings.

The problem is it does skews results. And the wider the strength deference between the players. The more such a fixed opening design choice to cause more decisive games can skew results.

[syzygy]

Because I said dubious opens. As in not sound.

I still don't see why you think that would skew the results, if each engine has to play both sides. In the worst case the start position is a certain win, and then they would just get a 1-1 on that line, and it is as if the game was never played, and there were just fewer games.

And I don't see why you don't think that it does skew results.

And the games are played and do count in TCEC.

Lets make an absurd example to show the point. I play GM Carlsen a 6 game match with 6 fixed and very unsound openings. So unsound I am able to win with white every game. And so is GM Carlsen.

And that is exactly what is not happening in TCEC.

[MikeB]

There's simply no room for discussion.

And yet here we are on page eight.

+1 it's clearly one of those cases where the obvious solution is so clear and recognizable by most people - yet the people running the show, I'm sure very smart people in their own right , show to the entire world - their utter lack of common sense and sound judgement . At this point it's not about the 1/2 point at all - not only are they displaying a total lack of judgment, but they are also displaying to the world , their circle the wagons mentality and are even not even considering the possibility they missed the boat , that's it even possible that they could even be wrong. One has learned more about them in this one episode , than anyone ever desired 😳

[basil00]

FWIW here is one possible continuation of the game assuming Syzygy/DTZ perfect play (there may be other equally good solutions) found using the Fathom probe tool. Sure enough the game ends in a draw by the 50 move rule. Since both Stockfish and Houdini both use the Syzygy tablebases, the game would have ended in a draw, assuming no bugs in either engine's TB implementation.


[pgn]
[Event ""]
[Site ""]
[Date "??"]
[Round "-"]
[White "Syzygy"]
[Black "Syzygy"]
[Result "1/2-1/2"]
[FEN "K5Q1/8/8/8/5bb1/6k1/8/8 b - - 0 72"]

72... Be5 73. Kb7 Kf4 74. Kc6 Kf5 75. Kb5 Kf4 76. Kb4 Kf5 77. Qc4 Bf3 78. Qf1 Ke4 79. Kc4 Bf6 80. Qf2 Be5 81. Qh4+ Kf5 82. Qh7+ Kg4 83. Kd3 Bf4 84. Qd7+ Kh4 85. Qe6 Kg3 86. Qf5 Bg2 87. Ke2 Bf3+ 88. Kf1 Bg2+ 89. Ke1 Bf3 90. Qd3 Be5 91. Qe3 Bf6 92. Kd2 Bh4 93. Qg1+ Kf4 94. Qg7 Bf2 95. Kd3 Bh4 96. Qf7+ Kg4 97. Kd4 Bf2+ 98. Ke5 Bg3+ 99. Ke6 Be4 100. Qf1 Kg5 101. Qc1+ Kg4 102. Kf6 Bf4 103. Qc3 Bf3 104. Qe1 Bh2 105. Kg6 Bg3 106. Qe3 Bf4 107. Qg1+ Bg3 108. Kf6 Kf4 109. Qc1+ Ke4 110. Kg5 Be2 111. Qc2+ Ke3 112. Qc3+ Bd3 113. Kg4 Bd6 114. Qf6 Be2+ 115. Kf5 Bd3+ 116. Kg5 Bg3 117. Kg4 Be1 118. Qe6+ Kf2 119. Qe5 Be2+ 120. Kf4 Bd2+ 121. Ke4 Bf1 122. Qf5+ {Draw by fifty move rule} 1/2-1/2
[/pgn]

[mwyoung]

Because I said dubious opens. As in not sound.

I still don't see why you think that would skew the results, if each engine has to play both sides. In the worst case the start position is a certain win, and then they would just get a 1-1 on that line, and it is as if the game was never played, and there were just fewer games.

And I don't see why you don't think that it does skew results.

And the games are played and do count in TCEC.

Lets make an absurd example to show the point. I play GM Carlsen a 6 game match with 6 fixed and very unsound openings. So unsound I am able to win with white every game. And so is GM Carlsen.

And that is exactly what is not happening in TCEC.

I respect your opinion.

And I will let TCEC make my point.

From TCEC themselves on the opening book choices played in TCEC season 9.

"For this season I plan to deviate from what I have done in the past. Instead of providing balanced positions that always favor white I will come up with a set of positions that may, in some cases, provoke outcries of alarm. Keep in mind the engines get to play both sides and optimal chess theory isn’t the goal here—we’re trying to produce an outcome that puts the most deserving two engines into the final. Expect the unusual and more blood on the board than we’ve seen this late in the tournament in recent seasons!"

[bnemias]

it's clearly one of those cases where the obvious solution is so clear and recognizable by most people

The solution in THIS case is clear, amend the result so the adjudication metric aligns with the metric the engines were fed.

The wider TCEC solution isn't so clear because it is possible to have one engine using Gaviota while the other is using Syzygy (or no TB). Luckily we don't have that case, because I doubt it is possible to adjudicate that case fairly under the current rules.

However, recognizing the possibility of the latter is important in preventing this problem in subsequent seasons.

[Norm Pollock]

Here are the rules about adjudicating wins:

Code: Select all

 It will adjudicate as won for one side if both playing engines have an eval of at least 6.50 pawns (or -6.50 in case of a black win) for 4 consecutive moves, or 8 plies - this rule is in effect as soon as the game starts. In the website this rule is shown as "TCEC win rule" with a number indicating how many plies there are left until it kicks in. Cutechess will also adjudicate 5-men or less tablebase endgame positions automatically.

Both rules are there to save time and get on with the next game.

The problem that occurred in the finals showed that the second rule is defective. So the simple solution is to remove the last sentence. There will still be one rule left to adjudicate a win.

[Dirt]

The wider TCEC solution isn't so clear because it is possible to have one engine using Gaviota while the other is using Syzygy (or no TB). Luckily we don't have that case, because I doubt it is possible to adjudicate that case fairly under the current rules.

Nonsense. Syzygy is always right as far as I know. Gaviota can be wrong. If an engine uses Gaviota it has to expect it being wrong sometimes.

[bnemias]

Code: Select all

... Cutechess will also adjudicate 5-men or less tablebase endgame positions automatically.

...

The problem that occurred in the finals showed that the second rule is defective.

I don't think the 2nd rule is defective. The problem (in this case) is not adjudication per se, but rather that the adjudication was based on different TB than the engines were using.

As I mentioned above, we don't have the more difficult case here, where one side uses Gaviota and the other uses one (or none) that obeys the 50 move rule.

But in this case, the 50 move rule doesn't matter wrt adjudication because both engines were using TB that said it was a draw, and the tournament software used a different set that said it was a win for one side.