Engine authors, beware of false-draw-claim forfeits!

Discussion of chess software programming and technical issues.

Moderator: Ras

User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Engine authors, beware of false-draw-claim forfeits!

Post by hgm »

bob wrote:I do not follow your point about the "result" keyword.
RESULT is not a keyword, but a non-terminal grammar symbol the WinBoard protocol document uses for the strings "1-0", "0-1", or "1/2-1/2", which the engine sends to the GUI to end the game. This is not to be confused with the 'result RESULT' command the GUI sends to both engines (for the purpose of learning) to tell them how the game ended. See the WinBoard protocol definition on Tim Mann's website.
It does not have to come before or after a move.
No, but it comes before a comment field, where you can state the reason for the result claim.
And it certainly isn't attached to a move. To properly claim a draw in real chess you write down your move but do not make it, you stop both clocks, and inform the TD. There is no such mechanism for online play or automated play using the winboard match option. Sending the "result" keyword in winboard ends the game instantly, correct or not.
Sending a 'result' keyword to WinBoard does not have any specified effect. You should sennd 1-0, 0-1 or 1/2-1/2 to instantly end the game.
I can show you a bunch of games where Arasan 9 had a bug and would send the wrong result when resigning, and winboard was giving it the victory instead of a loss.
No need. You won't be able to pull such a trick in WinBoard_F.
ICC has addressed this by allowing you to first offer a draw, then making a move. The server will then notice (say) a 3-fold repetition or 50-move draw and enforce it so long as the move is entered after the draw command.
This certainly would be an alternative to handle it for WinBoard as well. Would this way of rep-draw claiming be preferrable over specifying the move in the COMMENT field of the result? It would certainly be easier to implement in the GUI, so if there are no major objections, I will declare even draw claims in non-drawn positions that do specify the move in the COMMENT field as false claims that forfeit the game. The only safe way to claim a repdraw would then be through a draw offer.
User avatar
Matthias Gemuh
Posts: 3245
Joined: Thu Mar 09, 2006 9:10 am

ChessGUI approach

Post by Matthias Gemuh »

If ChessGUI receives a claim without/before the drawing move, engine forteits ... full stop (period).


Matthias.
My engine was quite strong till I added knowledge to it.
http://www.chess.hylogic.de
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: ChessGUI approach

Post by hgm »

Yes, this is what WinBoard_F does too (with claim verification enabled). But considering a draw offer not only as a request to the opponent to grant a draw, but also to the arbiter (=GUI), does not sound like a bad alternative. It is easy to implement, both for engine and GUI programmers. And it would not suffer from sync problems.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Engine authors, beware of false-draw-claim forfeits!

Post by bob »

hgm wrote:
bob wrote:I do not follow your point about the "result" keyword.
RESULT is not a keyword, but a non-terminal grammar symbol the WinBoard protocol document uses for the strings "1-0", "0-1", or "1/2-1/2", which the engine sends to the GUI to end the game. This is not to be confused with the 'result RESULT' command the GUI sends to both engines (for the purpose of learning) to tell them how the game ended. See the WinBoard protocol definition on Tim Mann's website.

Turns out you can send "result {1-0 white wins on time}" if you want to and it works perfectly. Always has in the winboard protocol. So "result" actually is a keyword that the engine can send to the GUI or vice-versa.

It does not have to come before or after a move.
No, but it comes before a comment field, where you can state the reason for the result claim.



So what? As I mentioned, the result claim is taken at face value by winboard, right or wrong. That is not addressing the problem we are talking about here, which is how to follow the rules of chess and make them work in a GUI. To date, only the ICC gui handles draw offers in a way that is consistent with FIDe, because they directly addressed the timing issue I mentioned, while no GUI I have run under understands the idea of a move and a repetition claim being tied together. If you send the claim first, they complain that it is invalid. If you send the move first, the GUI can't wait around to see if you are going to send a draw claim next because that introduces artificial lag and screws up the time-keeping...
And it certainly isn't attached to a move. To properly claim a draw in real chess you write down your move but do not make it, you stop both clocks, and inform the TD. There is no such mechanism for online play or automated play using the winboard match option. Sending the "result" keyword in winboard ends the game instantly, correct or not.
Sending a 'result' keyword to WinBoard does not have any specified effect. You should sennd 1-0, 0-1 or 1/2-1/2 to instantly end the game.
What it "should do" vs what it "actually does" might well be two different things, but the result string from the engine to the GUI will certainly terminate the game. Right, wrong or indifferent. And that lets engines produce bogus results which has given me a bit of grief in all the cluster testing I have done, to the point that I wrote my own non-X referee program that only trusts Crafty to send the result of a game, since some opponents are unreliable in that regard.






I can show you a bunch of games where Arasan 9 had a bug and would send the wrong result when resigning, and winboard was giving it the victory instead of a loss.
No need. You won't be able to pull such a trick in WinBoard_F.

Does that mean programs can't claim a win draw or loss themselves?

and what is "winboard_f"???




ICC has addressed this by allowing you to first offer a draw, then making a move. The server will then notice (say) a 3-fold repetition or 50-move draw and enforce it so long as the move is entered after the draw command.
This certainly would be an alternative to handle it for WinBoard as well. Would this way of rep-draw claiming be preferrable over specifying the move in the COMMENT field of the result? It would certainly be easier to implement in the GUI, so if there are no major objections, I will declare even draw claims in non-drawn positions that do specify the move in the COMMENT field as false claims that forfeit the game. The only safe way to claim a repdraw would then be through a draw offer.[/quote]


I don't like artificial implementations like "move in comment field". For a repetition, I have to actually choose and make a move to justify the claim. And the move has to become part of the PGN game record as in "33. Rh1+ {Draw by repetition}. The only caveat is that if a programmer is sloppy, he has to be prepared for his 3-fold repetition claim to be denied due to a timing issue, so a program can't claim a draw and stop playing, else it might well lose on time (as happened to me back when this problem first came to light).
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Engine authors, beware of false-draw-claim forfeits!

Post by hgm »

bob wrote:Turns out you can send "result {1-0 white wins on time}" if you want to and it works perfectly. Always has in the winboard protocol. So "result" actually is a keyword that the engine can send to the GUI or vice-versa.
What 'turns out you can send' and what is part of the protocol are two entirely different things, wouldn't you agree? That is works or some versions of WinBoard, or in the single version you happen to use, is no guarantee whatsoever that it will work in all versions of WinBoard, in future versions of WinBoard, or in other GUIs using WinBoard protocol.

Sending result { 1-0 white wins on time } would in fact give you a big fat zero in WinBoard 4.3.13. And rightly so, as the protocol very explicitly states that you are not allowed to claim results based on things happening outside the board, such as the clocks.

It does not have to come before or after a move.
No, but it comes before a comment field, where you can state the reason for the result claim.
So what? As I mentioned, the result claim is taken at face value by winboard, right or wrong.
Hellowwww! Anybody awake there?
Not in WinBoard 4.3.13


The entire purpose of this thread is to inform you and others that such claims are _no longer_ taken at face value but will result in a forfeit.
That is not addressing the problem we are talking about here, which is how to follow the rules of chess and make them work in a GUI. To date, only the ICC gui handles draw offers in a way that is consistent with FIDe, because they directly addressed the timing issue I mentioned, while no GUI I have run under understands the idea of a move and a repetition claim being tied together. If you send the claim first, they complain that it is invalid. If you send the move first, the GUI can't wait around to see if you are going to send a draw claim next because that introduces artificial lag and screws up the time-keeping...
We have discussed now already two work-arounds for this: the draw offer before the move, and giving the move in the comment field. In fact there even is a third: the GUI can judge the validity of a claim both in the position before and after the last move of the opponent, if the the side making the claim is on move, and accept the claim if it would be valid in either of these positions.
What it "should do" vs what it "actually does" might well be two different things,
This will not be two different things, as I personally will see to that.
but the result string from the engine to the GUI will certainly terminate the game. Right, wrong or indifferent.
A result string has to terminate the game, as engines are in general not prepared to play on after sending one. But you can be sure that many existing and future GUIs will end the game right, i.e. forfeit it for the engine sending an improper result string. Such as a rep-draw claim before the move realizing it.
Does that mean programs can't claim a win draw or loss themselves?
They can claim it, but if the claim has no basis, it will be treated as a 'resign', and the game will be forfeited.
and what is "winboard_f"???
The latest version of WinBoard, of course.
I don't like artificial implementations like "move in comment field". For a repetition, I have to actually choose and make a move to justify the claim. And the move has to become part of the PGN game record as in "33. Rh1+ {Draw by repetition}. The only caveat is that if a programmer is sloppy, he has to be prepared for his 3-fold repetition claim to be denied due to a timing issue, so a program can't claim a draw and stop playing, else it might well lose on time (as happened to me back when this problem first came to light).
Well, then we will settle on the draw-offer solution.

That means that making the claim before the move will result in a forfeit under any conditions, even when you specify a move in the COMMENT. Due to a protocol violation, rather than a FIDE rules violation.

So engine programmers, beware!
User avatar
Matthias Gemuh
Posts: 3245
Joined: Thu Mar 09, 2006 9:10 am

Re: Engine authors, beware of false-draw-claim forfeits!

Post by Matthias Gemuh »

hgm wrote: That means that making the claim before the move will result in a forfeit under any conditions, even when you specify a move in the COMMENT. Due to a protocol violation, rather than a FIDE rules violation.

So engine programmers, beware!


Absolutely same in ChessGUI !

Matthias.
My engine was quite strong till I added knowledge to it.
http://www.chess.hylogic.de
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Engine authors, beware of false-draw-claim forfeits!

Post by bob »

hgm wrote:
bob wrote:Turns out you can send "result {1-0 white wins on time}" if you want to and it works perfectly. Always has in the winboard protocol. So "result" actually is a keyword that the engine can send to the GUI or vice-versa.
What 'turns out you can send' and what is part of the protocol are two entirely different things, wouldn't you agree? That is works or some versions of WinBoard, or in the single version you happen to use, is no guarantee whatsoever that it will work in all versions of WinBoard, in future versions of WinBoard, or in other GUIs using WinBoard protocol.

Sending result { 1-0 white wins on time } would in fact give you a big fat zero in WinBoard 4.3.13. And rightly so, as the protocol very explicitly states that you are not allowed to claim results based on things happening outside the board, such as the clocks.

It does not have to come before or after a move.
No, but it comes before a comment field, where you can state the reason for the result claim.
So what? As I mentioned, the result claim is taken at face value by winboard, right or wrong.
Hellowwww! Anybody awake there?
Not in WinBoard 4.3.13


The entire purpose of this thread is to inform you and others that such claims are _no longer_ taken at face value but will result in a forfeit.
OK, in that same tone, "HELLO..." Who gives a damn about some non-standard version of winboard that drastically changes the way things work from the 100 previous releases?
That is not addressing the problem we are talking about here, which is how to follow the rules of chess and make them work in a GUI. To date, only the ICC gui handles draw offers in a way that is consistent with FIDE, because they directly addressed the timing issue I mentioned, while no GUI I have run under understands the idea of a move and a repetition claim being tied together. If you send the claim first, they complain that it is invalid. If you send the move first, the GUI can't wait around to see if you are going to send a draw claim next because that introduces artificial lag and screws up the time-keeping...
We have discussed now already two work-arounds for this: the draw offer before the move, and giving the move in the comment field. In fact there even is a third: the GUI can judge the validity of a claim both in the position before and after the last move of the opponent, if the the side making the claim is on move, and accept the claim if it would be valid in either of these positions.
And I explained why that won't work in the general case where a _program_ is allowed to claim an actual result. You can't claim a draw before you repeat the position for the third time. The GUI has no business holding up the game for what has to be an indeterminate amount of time waiting to see if the engine is also going to send a move after the draw offer. And, in fact, the idea is broken. Does that mean while it is my move, I can't simply offer my opponent a draw and see how he response, as the rules of chess specify?

The _right_ way to do this would be the following:

1. A specific string to claim a draw by repetition or by 50-move rule, where the claim is valid _right now_ without my having to make a move. In real chess, I can claim a forced draw before or after making my move.

2. A specific string to claim a draw by repetition or by 50 move rule, where the claim is valid _after_ I make a move, and the move has to be sent with the claim (but not as a comment, since it is a real move and needs to be entered in the PGN correctly).

3. A mechanism to offer draws to my opponent, and accept draw offers from my opponent without having to make a move.

4. A mechanism to offer my opponent a draw, but includes a move. It is now his turn to move, his clock is running, he has a draw offer that is valid until he accepts or makes a move of his own.

5. programs must _not_ be able to tell the GUI the game outcome. Or, at the very least, they can only claim they lost. They can not claim draws or wins. Causes too many problems for non-debugged programs.

When winboard/xboard can do all of that correctly, it will be worth something. As it is, I rely on ICC which does know how to handle draw offers/claims, and which _never_ lets an opponent tell it how the game ended...

What it "should do" vs what it "actually does" might well be two different things,
This will not be two different things, as I personally will see to that.
but the result string from the engine to the GUI will certainly terminate the game. Right, wrong or indifferent.
A result string has to terminate the game, as engines are in general not prepared to play on after sending one. But you can be sure that many existing and future GUIs will end the game right, i.e. forfeit it for the engine sending an improper result string. Such as a rep-draw claim before the move realizing it.
This makes no sense to me. Why allow an engine to send a result string if the GUI might or might not use it? In real chess I can't just say "I win" and the game ends. That is not something either player should be able to do. If the GUI handles the clocks, the players should not have to call "flag" either, as there is a timing hole there. The one that should call the flag is the one watching the flag. Since the program might not know that the flag dropped when the opponent took .6 seconds too long, but then the clock increment took it back to positive, the program is unsuited to handling this. Makes no sense for the GUI to tell the program "his clock went below 0.0" either... Because who would not claim the win, making the tit-for-tat messages redundant...



Does that mean programs can't claim a win draw or loss themselves?
They can claim it, but if the claim has no basis, it will be treated as a 'resign', and the game will be forfeited.
Any chance this can follow the _real_ rules of chess and not use some ad-hoc policy? If I claim a draw in a tournament, I don't lose if the draw is not upheld. I am forced to play on, sometimes with a time penalty, sometimes not. But I certainly do not lose.





and what is "winboard_f"???
The latest version of WinBoard, of course.
"Of Course"??? I've worked with Tim Mann for many years adding features to the winboard/xboard protocol. "winboard_f" certainly doesn't conjure up ideas of "the latest winboard version" to me. Does this carry to xboard as well???





I don't like artificial implementations like "move in comment field". For a repetition, I have to actually choose and make a move to justify the claim. And the move has to become part of the PGN game record as in "33. Rh1+ {Draw by repetition}. The only caveat is that if a programmer is sloppy, he has to be prepared for his 3-fold repetition claim to be denied due to a timing issue, so a program can't claim a draw and stop playing, else it might well lose on time (as happened to me back when this problem first came to light).
Well, then we will settle on the draw-offer solution.

That means that making the claim before the move will result in a forfeit under any conditions, even when you specify a move in the COMMENT. Due to a protocol violation, rather than a FIDE rules violation.
Again, that is wrong. If this doesn't follow the FIDE rules of chess, nobody is going to use/accept it. I can _always_ claim a draw without making a move. If you make a move, hit your clock, and I notice that both sides have made at least 50 or more moves without a capture or pawn push, I can claim a draw "right now" without having to make a move or anything. Ditto for repetitions. If the position before my move has occurred twice previously with me on move, I can claim a draw without having to make any move at all, and it ends the game once verified.
So engine programmers, beware!

I'm not going to beware, I am not going to use it if it doesn't actually match the official FIDE rules. And assuming the "feature" command doesn't get broken, I would probably modify Crafty to refuse to play if the GUI behaves as you are describing, since it is broken.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Engine authors, beware of false-draw-claim forfeits!

Post by bob »

Matthias Gemuh wrote:
hgm wrote: That means that making the claim before the move will result in a forfeit under any conditions, even when you specify a move in the COMMENT. Due to a protocol violation, rather than a FIDE rules violation.

So engine programmers, beware!


Absolutely same in ChessGUI !

Matthias.

Then you, too, have a broken system. I can claim a draw without making a move of any kind. At least in _real_ chess...
User avatar
Matthias Gemuh
Posts: 3245
Joined: Thu Mar 09, 2006 9:10 am

Re: Engine authors, beware of false-draw-claim forfeits!

Post by Matthias Gemuh »

bob wrote:
Matthias Gemuh wrote:
hgm wrote: That means that making the claim before the move will result in a forfeit under any conditions, even when you specify a move in the COMMENT. Due to a protocol violation, rather than a FIDE rules violation.

So engine programmers, beware!


Absolutely same in ChessGUI !

Matthias.

Then you, too, have a broken system. I can claim a draw without making a move of any kind. At least in _real_ chess...


No, my system is not broken !! If some FIDE guys can't define GUI-meaningful rules for lack of programming insight, the GUI programmers must set the standards. It is _real_ chess, not FIDE theory.
My engine was quite strong till I added knowledge to it.
http://www.chess.hylogic.de
Guetti

Re: Engine authors, beware of false-draw-claim forfeits!

Post by Guetti »

Hm. Should I care about Winboard_F? I don't run Windows. I run Unix.
I use xboard. Latest version is 4.2.7.

By the way, from where comes this "false draw claim = forfeit" rule? FIDE? The last time I claimed a draw wrongly in a OTB game I was just IGNORED and I was told to play on. Shouldn't a GUI do the same?