Engine authors, beware of false-draw-claim forfeits!

Discussion of chess software programming and technical issues.

Moderator: Ras

Uri Blass
Posts: 10792
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Engine authors, beware of false-draw-claim forfeits!

Post by Uri Blass »

hgm wrote:
Graham Banks wrote:
bob wrote:
Graham Banks wrote:
Guetti wrote:The last time I claimed a draw wrongly in a OTB game I was just IGNORED and I was told to play on. Shouldn't a GUI do the same?
Ideally, yes, but it would probably cause one of the engines to crash? 8-)
Then the engine that crashes should lose on time, correct? If it can't play by the official rules, that's the preferred outcome, the one that doesn't follow the rules pays the piper...
Agreed.
But I can hardly imagine that you, as a tester, would prefer having to wait upto 40 minutes before forfeiting the game for the offending engine, when it is abundantly clear that it is not going to make a move anymore...

WinBoard_F immediately ends games with the proper result when one of the engines is no longer alive and the pipes to it are broken (i.e. it forfeits the game for that engine.) It does not wait for the flag.

In practice engines won't crash, though. They would just sit idle, not playing any moves anymore in reaction to moves you feed them. So you would have to engage in a needless wait for the flag, while they already unambiguously let you know they are not going to move anymmore.
I think that things may be dependent on the programmer.
I can imagine programs that are programmed to continue after they get a move even if they claimed a draw claim earlier.

I think that the ideal solution is simply that engines that do it will tell winboard in the beginning of the game that they plan to do it.

I can also imagine engines that try to continue if they get a message that they played an illegal move by trying to play another move with the same piece(I doubt if there are engines with this type of behaviour but it is possible to have engines with this type of behaviour).

Again the ideal solution is that engines with this behaviour tell winboard in the beginning of the game that they plan to do it.

Uri
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Engine authors, beware of false-draw-claim forfeits!

Post by hgm »

What would be the purpose? Basically you want the engines to be able to redefine the protocol, by means of an even more complicated protocol. If engines have the capability to come up with a legal move after their illegal move is rejected, wouldn't it be simpler to require them to send that legal move in the first place? If they want to play on, in stead of ending the game, wouldn't it be simpler to require them not to send the message that they have ended the game, rather than first have them send something that this message should not be taken seriously?
Uri Blass
Posts: 10792
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Engine authors, beware of false-draw-claim forfeits!

Post by Uri Blass »

hgm wrote:What would be the purpose? Basically you want the engines to be able to redefine the protocol, by means of an even more complicated protocol. If engines have the capability to come up with a legal move after their illegal move is rejected, wouldn't it be simpler to require them to send that legal move in the first place? If they want to play on, in stead of ending the game, wouldn't it be simpler to require them not to send the message that they have ended the game, rather than first have them send something that this message should not be taken seriously?
1)The engine may have a list of moves that include an illegal move because of a bug.

In case of getting the information that this move is illegal the engine may be programmed to delete the last move from the list of moves and look for a different move.

I do not know about a programmer who does it and I think that the simplest way to solve the problem is to fix the bug but I can imagine the possibility that some programmer decides to be creative.

2)In case of incorrect draw claim programming that allows to continue the game after a move is simple and there are advantages of it because the rules when to draw by insufficient material are not simple and the interface may not understand them.

Here is an example that I believe that winboard_F does not understand

[d]7k/8/8/8/6pp/4p2P/4P1PK/5BR1 b - - 0 1

Based on the fide rules black can play g3+ and claim a draw based on 9.6 of the fide rules

http://www.fide.com/official/handbook.asp?level=EE101

"The game is drawn when a position is reached from which a checkmate cannot occur by any possible series of legal moves, even with the most unskilled play. This immediately ends the game, provided that the move producing this position was legal."

The engine claims a draw in order to save time and finish the game earlier but the engine may be ready for the situation that the interface is not smart enough to understand that it is a draw so the engine is ready to continue the game if the draw is not accepted.

Uri
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Engine authors, beware of false-draw-claim forfeits!

Post by hgm »

Uri Blass wrote:The engine claims a draw in order to save time and finish the game earlier but the engine may be ready for the situation that the interface is not smart enough to understand that it is a draw so the engine is ready to continue the game if the draw is not accepted.
But WB protocol already defines a command that means exactly that: "offer draw". What could be more simple than using that? A very smart GUI could then indeed grant the draw immediately. A less smart GUI (most likely every GUI in the World...) would not grant the draw, and pass the draw request (and the move) to the opponent, through the "draw" command. A smart opponent would then grant the draw. A less smart opponent would ignore the draw request, play Kh1, and send a 1/2-1/2 { statlemate } message.

No need to add new commands to redefine the meaning of existing commands...
Uri Blass
Posts: 10792
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Engine authors, beware of false-draw-claim forfeits!

Post by Uri Blass »

hgm wrote:
Uri Blass wrote:The engine claims a draw in order to save time and finish the game earlier but the engine may be ready for the situation that the interface is not smart enough to understand that it is a draw so the engine is ready to continue the game if the draw is not accepted.
But WB protocol already defines a command that means exactly that: "offer draw". What could be more simple than using that? A very smart GUI could then indeed grant the draw immediately. A less smart GUI (most likely every GUI in the World...) would not grant the draw, and pass the draw request (and the move) to the opponent, through the "draw" command. A smart opponent would then grant the draw. A less smart opponent would ignore the draw request, play Kh1, and send a 1/2-1/2 { statlemate } message.

No need to add new commands to redefine the meaning of existing commands...
You are right.
I think that usually engines claim a draw and not offer it when they consider the position to be drawn and I guess that the interface is going to accept claiming draw for the same reason in KB vs K or KN vs K or KB vs KB when the bishops are of the same color.

I know that Leo accept claiming draw even in KB vs KB when the bishops are of different colours but this is not according to the fide rules.
Practically the only problem that I see is to decide what to do in KB or KN vs KB or KN when one engine claims a draw.

It is not a draw by the fide rules except the case when the bishops are of the same color but it is a draw by Leo's rules of WBEC

Uri
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Engine authors, beware of false-draw-claim forfeits!

Post by hgm »

Well, if the GUI cannot possibly refuse the draw, because it is draw by rule, obvious enough for even a 'stupid' GUI to see (because the repeat is currently on the board, or 50 reversible moves have already been played), it is safe for the engine to claim the draw by a hard claim like the RESULT message, in stead of the softer draw offer (which considers that the draw might be refused, and expresses willingness to go on after such a refusal).

WinBoard_F solves the KBKN problem you mention in the following way:

The engines only have to assume FIDE rules apply. The tournament director decides if he wants to adjudicate positions like KBKN, through enabling or disabling the option 'adjudicate trivial draws' of the GUI. If he enables that option, the GUI declares draw in KBKN even if none of the engines would claim it. Easier for the engine programmers, easier for Leo... :wink:
User avatar
Matthias Gemuh
Posts: 3245
Joined: Thu Mar 09, 2006 9:10 am

Re: Engine authors, beware of false-draw-claim forfeits!

Post by Matthias Gemuh »

bob wrote:
Matthias Gemuh wrote:
No, my system is not broken !! If some FIDE guys can't define GUI-meaningful rules for lack of programming insight, the GUI programmers must set the standards. It is _real_ chess, not FIDE theory.
Sorry, but the FIDE rules make perfect sense, and work just fine the way they are written. And when a GUI acts like a _real_ arbiter, there are no problems. ICC works perfectly for both computer and human (FIDE) events in fact, if the engine programmers take the time to make it work.

Can you please explain how a GUI is to determine whether a draw claim is legitimate without knowing the move the engine would draw with ?
Should the GUI assume some random legal move for engine ?

Matthias.
My engine was quite strong till I added knowledge to it.
http://www.chess.hylogic.de
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Engine authors, beware of false-draw-claim forfeits!

Post by bob »

Matthias Gemuh wrote:
bob wrote:
Matthias Gemuh wrote:
No, my system is not broken !! If some FIDE guys can't define GUI-meaningful rules for lack of programming insight, the GUI programmers must set the standards. It is _real_ chess, not FIDE theory.
Sorry, but the FIDE rules make perfect sense, and work just fine the way they are written. And when a GUI acts like a _real_ arbiter, there are no problems. ICC works perfectly for both computer and human (FIDE) events in fact, if the engine programmers take the time to make it work.

Can you please explain how a GUI is to determine whether a draw claim is legitimate without knowing the move the engine would draw with ?
Should the GUI assume some random legal move for engine ?

Matthias.
I have already given two alternatives.

1. Send the move, then send the draw claim. The GUI can certainly figure out that even if the opponent makes a move after receiving the move just played, that the draw claim should be enforced, and do so.

2. the move and draw offer/claim can be combined into one string if desired.

ICC handles this with absolutely no problems.

Basic idea:

move Re1+
draw

and if the position is a 3-fold or 50 move draw, the draw is enforced. How hard is that???
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Engine authors, beware of false-draw-claim forfeits!

Post by hgm »

Well, if you think this is acceptable, and if I already said this is what I will implement to accomodate that vast majority of engines that do not claim draws in a safe way, (i.e. by preceeding the move with a draw _offer_), why are you insisting that it should also be considered valid if the engine sends the draw claim _before_ the move, when there was no repeat or 50-moves yet???????????

Are you arguing against everything other write just out of habit? :roll:
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Engine authors, beware of false-draw-claim forfeits!

Post by bob »

hgm wrote:
bob wrote: OK, in that same tone, "HELLO..." Who gives a damn about some non-standard version of winboard that drastically changes the way things work from the 100 previous releases?
Organizers of engine-engine tournaments, and (rating) testers give a damn. It is at their request that I am building these features in WinBoard. They want a GUI that takes a firm stance against faulty engines, rather than one that lets such engines call the shots (as WinBoard 4.2.7 and earlier does).

You yourself just stated that WinBoard was unacceptable to you, and you had to write your own software for engine-engine tplay. So it must be easy to see even for you how likely it is that this new WinBoard will become the new 'industry standard' in most tournaments that are now played under WinBoard.

My posting was just a friendly warning. Give as many damns as you like. But if you want to be stubborn, be prepared to see Crafty forfeit games, loose a few Elo points and drop a few places on the rating lists. :lol:
Won't happen unless you adopt something contrary to FIDE rules. Then I won't care because I am not going to support crappy standards (such as UCI) that differ from the rules I have to use in real tournaments... Current crafty handles draw offers, accepting draws, draw claims, perfectly. I don't see that changing
We have discussed now already two work-arounds for this: the draw offer before the move, and giving the move in the comment field. In fact there even is a third: the GUI can judge the validity of a claim both in the position before and after the last move of the opponent, if the the side making the claim is on move, and accept the claim if it would be valid in either of these positions.
that's how ICC does it and it works perfectly...

And I explained why that won't work in the general case where a _program_ is allowed to claim an actual result. You can't claim a draw before you repeat the position for the third time. The GUI has no business holding up the game for what has to be an indeterminate amount of time waiting to see if the engine is also going to send a move after the draw offer. And, in fact, the idea is broken. Does that mean while it is my move, I can't simply offer my opponent a draw and see how he response, as the rules of chess specify?
Oh man... Why don't you for once READ what others write, in stead of immediately starting fumigating against your own silly misconceptions? NONE of the three methods did involve waiting of the GUI for what the engine might send. I am sorry if you don't like the tone, but it seems the only way to get through to your imaginary world, and pull you back to reality... :?
The _right_ way to do this would be the following:

1. A specific string to claim a draw by repetition or by 50-move rule, where the claim is valid _right now_ without my having to make a move. In real chess, I can claim a forced draw before or after making my move.

2. A specific string to claim a draw by repetition or by 50 move rule, where the claim is valid _after_ I make a move, and the move has to be sent with the claim (but not as a comment, since it is a real move and needs to be entered in the PGN correctly).

3. A mechanism to offer draws to my opponent, and accept draw offers from my opponent without having to make a move.

4. A mechanism to offer my opponent a draw, but includes a move. It is now his turn to move, his clock is running, he has a draw offer that is valid until he accepts or makes a move of his own.

5. programs must _not_ be able to tell the GUI the game outcome. Or, at the very least, they can only claim they lost. They can not claim draws or wins. Causes too many problems for non-debugged programs.

When winboard/xboard can do all of that correctly, it will be worth something. As it is, I rely on ICC which does know how to handle draw offers/claims, and which _never_ lets an opponent tell it how the game ended...
Well, WinBoard protocol does support ALL that correctly, so why are you arguing against my implementation of stricter enforcement of the protocol? Do you just want to be difficult?
Back to reality. Winboard does _NOT_ support that correctly. An engine can claim any result it wants. If you will simply re-read what I wrote above, you won't be off into never-never land. The "result" is not used by winboard when playing on ICC. But in match mode, it is, and it is a bad idea.

1. This string is the RESULT command: "1/2-1/2 { 3-fold repetition }" (or whatever the reason: 50-move rule, stalemate).
What part of "that is bad" don't you understand? If you send the 1/2-1/2 string today, the game ends. Even if you are losing

2. This string is "offer draw". Such an offer can be refused if there is no legal draw condition, but it is never penalized to make it.
Eh? I can offer a draw any time I choose, so long as my clock is running when I make the offer (see FIDE rules of chess for clarification). Such an offer can _never_ be refused, unless it comes _after_ I make a move, because when I make a move, that implicitly stops my clock and starts my opponent's clock.

Better solution would be a "hit the clock" command so that I can send anything I want, a move, a draw offer/claim, etc, and my "move" is not completed (by the FIDE rules of chess) until I press the clock button.

3. Again, "offer draw" does this, in absence of a legal draw condition. If the GUI cannot grant the draw, because there is no legal basis to claim one, it sends "draw" to the opponent to inform it of the offer.
There is a significant difference between "offering a draw" and "claiming a draw". I see no sense in overloading the "offer draw" to behave as above.

4. Same as 3. The engine can send "offer draw" at any time, before its move or after its move. It is a separate command.
It should not be able to send anything while the opponent is on move. We have already had interesting problems with this. I make a move, then while it is not my move I flood my opponent with draw offers to slow him down. In human games, it is against the rules to offer a draw while he is on move.

5. Exactly. This is why WinBoard_F makes them forfeit the game when the claim a draw through the WinBoard RESULT command in a position that is not a legal draw. Remember that WinBoard protocol defines the RESULT command as what the engine sends when it is not possible or prepared under any circumstances to continue with the game. So when an engine sends it, the GUI has no option but to terminate the game; refusing the RESULT command and waiting for the engine to reconsider its decision, will just hang the game and the tournament.

So what exactly is your problem????
I want to see it done correctly, according to the rules of chess. I want two options. I want to be able to offer a draw, only when it is my move, otherwise the offer is simply ignored; I want to be able to claim a draw, only when it is my move, and the claim is accepted or rejected, but without causing me to lose the game. That is the way FIDE works. If a program can't handle a rejected draw claim, then let it lose on time since it is not playing legal chess.

I prefer one of the following, syntactically:

offer draw Re1+

{I am offering a draw, because I think the position is either eventually drawn or it is dead even at the moment. I am also playing the move Re1 whether he accepts the offer or not.}

Re1+ draw

{I am playing the move Re1+ and I claim it is a draw (it is up to the GUI to figure out if it is a 50 move draw, or a repetition, or insufficient material, or a double-flag, or whatever. Again I have to be prepared to play on if the GUI disagrees for whatever reason}

Re1+ draw stop clock or
draw Re1+ stop clock

{I give my move, a draw claim, and tell the GUI my move is complete by giving the stop clock. The GUI doesn't inform my opponent of the move until I "stop the clock" as in FIDE human games.}

To offer a draw, this would become:

re1+ offer draw stop clock

We also need the special case:

draw stop clock

as it is possible that the position before I even move is a repetition or 50-move rule draw and I don't have to make a move to make the claim (again by the FIDE rules).
This makes no sense to me. Why allow an engine to send a result string if the GUI might or might not use it? In real chess I can't just say "I win" and the game ends. That is not something either player should be able to do. If the GUI handles the clocks, the players should not have to call "flag" either, as there is a timing hole there. The one that should call the flag is the one watching the flag. Since the program might not know that the flag dropped when the opponent took .6 seconds too long, but then the clock increment took it back to positive, the program is unsuited to handling this. Makes no sense for the GUI to tell the program "his clock went below 0.0" either... Because who would not claim the win, making the tit-for-tat messages redundant...
Should I read this as an admission that it starts to dawn on you why WinBoard_F is superior to the WinBoard you prefer, which does allow the engines to claim any result at any time?

ONLY if your fix is better than what we have. So far it does not seem to be so since it is violating FIDE rules in the way it behaves. For example, in FIDE rules, claiming a draw does not cause me to lose if the arbiter disallows the claim (perhaps I have an incomplete score sheet, or I incorrectly ignored enpassant on the first repetition or whatever... I just have to keep playing.
Any chance this can follow the _real_ rules of chess and not use some ad-hoc policy? If I claim a draw in a tournament, I don't lose if the draw is not upheld. I am forced to play on, sometimes with a time penalty, sometimes not. But I certainly do not lose.
Unfortunately Chess engines are not reasonably entities, that you can argue with. They are stupid state machines, and when the get into the state "game finished", no amount of input given to them is going to convince them that this game is not finished. WinBoard protocol does not specify any mechanism to tell an engine to restart a game after a RESULT command has been received; it defines it as the ultimate refusal to play on. If a Human would refuse to play on in the case you describe above, wouldn't you think that the _real_ rules of Chess would in the end declare the game forfeited? Seems to me you have no case...
Apples and oranges. I have already said that the "result" operation should not be allowed. Claiming a draw is a different matter. Crafty will play on after a 3-fold repetition if you want, as it depends on the GUI to enforce the draw, but doesn't quit...
"Of Course"??? I've worked with Tim Mann for many years adding features to the winboard/xboard protocol. "winboard_f" certainly doesn't conjure up ideas of "the latest winboard version" to me. Does this carry to xboard as well???
Well, WinBoard is open source, and Tim Mann currently doesn't seem to be a very active developer. If you only look at Tim Mann, you will be likely to find yourself in the stone age soon.
That means that making the claim before the move will result in a forfeit under any conditions, even when you specify a move in the COMMENT. Due to a protocol violation, rather than a FIDE rules violation.
Again, that is wrong. If this doesn't follow the FIDE rules of chess, nobody is going to use/accept it. I can _always_ claim a draw without making a move. If you make a move, hit your clock, and I notice that both sides have made at least 50 or more moves without a capture or pawn push, I can claim a draw "right now" without having to make a move or anything. Ditto for repetitions. If the position before my move has occurred twice previously with me on move, I can claim a draw without having to make any move at all, and it ends the game once verified.
We were talking here about the case where no draw condition exists before the move. So the examples you give here are irrelevant. Draw claims in these situations are accepted by WinBoard_F without any problems. The whole issue is what the GUI should do if an engine unconditionally refuses to play on (by sending the 1/2-1/2 command) in a position 3 moves after a capture, that has never been on the board before, and isn't a stalemate. If you would deem such a claim to "follow FIDE rules", I think it is you who are wrong!
Thinking doesn't make it so, however. If an engine refuses to play on, let it lose on time, as it would in a FIDE event. Only step in what you are saying that I would advocate is to ELIMINATE the 1/2-1/2, 1-0 and 0-1 strings from the engine to the GUI. If the engine sends 'em, ignore 'em. By the rules of chess, one player can not end the game on his own. I can't end a game on a 50-move claim without someone agreeing. So we do it like that in the engines, or we lose games until we do...

I'm not going to beware, I am not going to use it if it doesn't actually match the official FIDE rules. And assuming the "feature" command doesn't get broken, I would probably modify Crafty to refuse to play if the GUI behaves as you are describing, since it is broken.
Well, as I am an engine programmer next to being a GUI programmer, I always enjoy seeing competitors drop in rating... :lol: :lol: :lol:

Nice attitude. You won't see Crafty drop in rating. You just won't see it running under your GUI, unless you do it reasonably. If you think you can dictate a new and flawed standard, go for it. I won't follow, and I'll bet many others won't either. Do it right and most will be happy to have an improved protocol. But bury your head in the sand, and do it "your way or no way" and it ain't going to happen.