Cursed win at TCEC

[Ralph Stoesser]

I can't see why it should be broken. It's not the question if someone may be able to show that a checkmate is unavoidable. That someone has to be the engine that plays, and as long as the adjucation tb is state of the art there will be no problem. I would call that progress.

[syzygy]

I can't see why it should be broken. It's not the question if someone may be able to show that a checkmate is unavoidable. That someone has to be the engine that plays, and as long as the adjucation tb is state of the art there will be no problem. I would call that progress.

It has to be the engine that plays?

In game 17, both engines were able to show that the position was a draw.

How will you modify your condition now to make it fit your ideal?

In the time loss example white was able to show that the position was a win. Why should he not have been awarded the win?

And do you realise that you are skewing the game rules in favour of the side making the prediction that it can mate (no matter the time it would take him or the number of moves it would take him) over the side that actually tries to play out the win over the board (but then runs into a 50-move draw or runs out of time)?

[Ralph Stoesser]

I can't see why it should be broken. It's not the question if someone may be able to show that a checkmate is unavoidable. That someone has to be the engine that plays, and as long as the adjucation tb is state of the art there will be no problem. I would call that progress.

It has to be the engine that plays?

In game 17, both engines were able to show that the position was a draw.

How will you modify your condition now to make it fit your ideal?

In the time loss example white was able to show that the position was a win. Why should he not have been awarded the win?

And do you realise that you are skewing the game rules in favour of the side making the prediction that it can mate (no matter the time it would take him or the number of moves it would take him) over the side that actually tries to play out the win over the board (but then runs into a 50-move draw or runs out of time)?

I have absolutely no idea why you still think somebody and especially me would want to change FIDE rules? They are fine, nearly perfect, for ever, written in golden letters. We are talking about engine tournaments. FIDE rules do not apply in that case.

Then there are the TCEC rules, in particular the TB adjucation rule. There is obviously something broken. Game 17 shouldn't have been adjucated as a win, because both engines are playing with a TB setup that includes the 50 move draw rule, but the adjucation system doesn't. That wrong adjucation (actually there are two cases) should have been manually corrected.

In the end we would of course like to have an adjucation system that would be fair without any invention from a tournament director.

My wish for the next TCEC tournament would be:

Cutechess with 6 piece TB adjucation but without 50 move draw rule restriction. All engines playing with the same TB setup, also without 50 move draw rule restriction (or without TB at all). No other changes. Especially no changes in FIDE tournaments where humans are involved. Lol.

Now what exactly would be the problem with this setup?

[syzygy]

I can't see why it should be broken. It's not the question if someone may be able to show that a checkmate is unavoidable. That someone has to be the engine that plays, and as long as the adjucation tb is state of the art there will be no problem. I would call that progress.

It has to be the engine that plays?

In game 17, both engines were able to show that the position was a draw.

How will you modify your condition now to make it fit your ideal?

In the time loss example white was able to show that the position was a win. Why should he not have been awarded the win?

And do you realise that you are skewing the game rules in favour of the side making the prediction that it can mate (no matter the time it would take him or the number of moves it would take him) over the side that actually tries to play out the win over the board (but then runs into a 50-move draw or runs out of time)?

I have absolutely no idea why you still think somebody and especially me would want to change FIDE rules? They are fine, nearly perfect, for ever, written in golden letters. We are talking about engine tournaments. FIDE rules do not apply in that case.

Where did I mention FIDE?

Then there are the TCEC rules, in particular the TB adjucation rule. There is obviously something broken. Game 17 shouldn't have been adjucated as a win, because both engines are playing with a TB setup that includes the 50 move draw rule, but the adjucation system doesn't. That wrong adjucation (actually there are two cases) should have been manually corrected.

OK, it wasn't obvious to me that you were in agreement all along.

In the end we would of course like to have an adjucation system that would be fair without any invention from a tournament director.

I don't think correcting an adjudication result once or at most twice per season is a big deal. But of course it is nicer if the automatic adjudication system gets fixed.

My wish for the next TCEC tournament would be:

Cutechess with 6 piece TB adjucation but without 50 move draw rule restriction. All engines playing with the same TB setup, also without 50 move draw rule restriction (or without TB at all). No other changes. Especially no changes in FIDE tournaments where humans are involved. Lol.

Now what exactly would be the problem with this setup?

One problem is that this is unfair to engines that do not use TBs. They rely on the 50-move rule being applied in all positions.

Separately from the 50-move rule, I personally don't think 6-piece TB adjudication is a good idea if one of the engines does not use TBs. There are many 6-piece positions that (some) engines are not able to play correctly, so adjudicating those is helping TB-less engines too much. This problem also exists for 5-piece positions, but to a much smaller extent.

The best solution in my eyes is to replace TB adjudication with a request to authors of engines that use TBs to greatly limit thinking time on TB positions. Alternatively, TCEC could artificially limit their thinking time (as suggested by HGM), but that requires changes in cutechess and might be tricky to get right. Of course without TB adjudication the 50-move rule obviously continues to be in place.

[hgm]

It should not be difficult to get that right, if the adjudication code already exists. Instead of terminating the game you switch on a depth limit. That is all. But it is only a fair method if both engines use tablebases, and thus are in principle capable of moving instantly.

[syzygy]

It should not be difficult to get that right, if the adjudication code already exists. Instead of terminating the game you switch on a depth limit. That is all. But it is only a fair method if both engines use tablebases, and thus are in principle capable of moving instantly.

The difficulty is not so much in the technical implementation part as in the configuration part. Only those engines should be limited in time or depth that use TBs. Unfortunately such configuration issues are bound to go wrong and when this happens the game that was almost completed is ruined.

[Ralph Stoesser]

Ok, re TB-less engines: That's why I suggested the 'player', the engine of course, should prove to 'see' the unavoidable checkmate itself. If the engine shows a checkmate score with it's last move and the cutechess TB adjudication system agrees -> adjudicate the win. Now you could again say it is unfair to engines that don't use TB information at all. Well, it would (and should) be a disadvantage to not use TBs, but it would not be unfair if the rules would clearly state what happens.

[Lyudmil Tsvetkov]

no one knows how the game would have ended, as top engine analysis is imperfect, even in relatively simple positions.

I am afraid you have far too little knowledge of the engines and how they deal with tablebases.

It is 100% certain that the game would have ended in a draw. Thanks to its tablebases, H5 knew how to keep the mate beyond the 50-move horizon.

Try it out for yourself:
https://syzygy-tables.info/?fen=K5Q1/8/ ... _b_-_-_0_1

You are white and may play whatever moves you like. Black always plays the move shown at the top.

You won't get DTZ down by more than 1 ply on each move and black won't let DTZ go down by more than 1 ply on each move. Since you start at DTZ=124, you need more than 60 moves to get to a capture or mate. The 50-move rule kicks in after 50 moves. QED.

Now that you have learned something new, you might want to reconsider your earlier statements in this thread.

Rigth, I just start learning about engines and engine code, and often have hell of a time while reading and trying to understand some stuff: for example your SF tbs code, not the easiest code to read, but still understandable.

Nothing to reconsider for me, the GUI already adjudicated. (definitely this will be the longest thread of the year, and Nuno desrves some special prize for starting it)

Whether the game was won or draw depends on where you start analysing, you migth start a few plies earlier or later; in any case, if we are going to follow engine scores, the same way you proceed with your tablebases, then 105cps in the mg is certainly won. But, again, the point is not so much this particular game, but rather why on earth a KNOWN WIN should be adjudicated as a DRAW? Simply because there is some ancient rule concerning fortress drawing?

If we have witnessed at least a couple of similar developments in TCEC with only couple of hundred games played, then you can certainly imagine how many objective wins are adjudicated as artificial draws in a much larger pool of games that is statistically relevant. Why artificially distort the result?

btw., FIDE rules of chess are quite adamant on this: an existing mate on the board overrides any other possible outcome. A tablebase mate in 72 is certainly an existing mate, it should definitely override any other specific ad-hoc rule, like current TCEC draw rule, and the end position, as we all know perfectly well, was simply a mate in 72, rigth?

So, what should be done is for tbs adjudication to fall in line with FIDE rules, and the stupid 50-moves restriction be lifted as soon as possible, only in pawnless endgames.

[Lyudmil Tsvetkov]

[d]4k1n1/8/8/8/8/8/8/2B1KB2 w - - 0 1

why adjudicate above position as a draw, whatever the adjudication rules?
if tablebases are in force, that simply means there is mate on the board. if some software prunes that mate, it is the fault of the software, of course.

do you know how often similar positions occur? (above is mate in 66)

[d]2k1r3/2r5/8/8/8/8/5R2/2BR1K2 w - - 0 1

or, this one (again longer than 50 mate)

do you know how often above correlation occurs?
If I have seen this myself while watching/browsing through engine games at least 5 or 6 times, then it should be relatively frequent. Obviously, any wrong tbs adjudication will only distort the result.

[Uri Blass]

no one knows how the game would have ended, as top engine analysis is imperfect, even in relatively simple positions.

I am afraid you have far too little knowledge of the engines and how they deal with tablebases.

It is 100% certain that the game would have ended in a draw. Thanks to its tablebases, H5 knew how to keep the mate beyond the 50-move horizon.

Try it out for yourself:
https://syzygy-tables.info/?fen=K5Q1/8/ ... _b_-_-_0_1

You are white and may play whatever moves you like. Black always plays the move shown at the top.

You won't get DTZ down by more than 1 ply on each move and black won't let DTZ go down by more than 1 ply on each move. Since you start at DTZ=124, you need more than 60 moves to get to a capture or mate. The 50-move rule kicks in after 50 moves. QED.

Now that you have learned something new, you might want to reconsider your earlier statements in this thread.

Rigth, I just start learning about engines and engine code, and often have hell of a time while reading and trying to understand some stuff: for example your SF tbs code, not the easiest code to read, but still understandable.

Nothing to reconsider for me, the GUI already adjudicated. (definitely this will be the longest thread of the year, and Nuno desrves some special prize for starting it)

Whether the game was won or draw depends on where you start analysing, you migth start a few plies earlier or later; in any case, if we are going to follow engine scores, the same way you proceed with your tablebases, then 105cps in the mg is certainly won. But, again, the point is not so much this particular game, but rather why on earth a KNOWN WIN should be adjudicated as a DRAW? Simply because there is some ancient rule concerning fortress drawing?

If we have witnessed at least a couple of similar developments in TCEC with only couple of hundred games played, then you can certainly imagine how many objective wins are adjudicated as artificial draws in a much larger pool of games that is statistically relevant. Why artificially distort the result?

btw., FIDE rules of chess are quite adamant on this: an existing mate on the board overrides any other possible outcome. A tablebase mate in 72 is certainly an existing mate, it should definitely override any other specific ad-hoc rule, like current TCEC draw rule, and the end position, as we all know perfectly well, was simply a mate in 72, rigth?

So, what should be done is for tbs adjudication to fall in line with FIDE rules, and the stupid 50-moves restriction be lifted as soon as possible, only in pawnless endgames.

chess programs played based on the assumption that there is 50 move rule.
Chess programs are programmed to use rules of game A(with the 50 move rule).

Adjudicating based on rule B without the 50 move rule in 5 piece positions is not fair when you did not tell the programmers in advance about the rules.

Note that using distance to mate and not using the 50 move rule in tablebases also is not going to solve the problem because in this case a program may force 6 piece tablebases position that is mate in 80 but draw by the 50 move rule and TCEC does not adjudicate 6 piece tablebases positions.

If you want programs not to use the 50 move rule you are free to organize your own tournament when you declare the rules before the tournament
and invite programmers to change their programs in order to participate.