where FEN is not consistent

[pijl]

Both end in identical positions, but FEN demands for different FEN strings. XFEN instead decides to not use any e.p. flag in both cases. Thus XFEN reduces such divergences without creating any problem.

This is a non-issue. Due to compatibility reasons you'll need to check for valid EP while parsing anyway, and when writing FEN there is no compatibility issue when you do not print a field in the ep square either.
Richard.

[pijl]

When I mentioned the "neighbor Pawn is pinned" case I forgot the possibility of the double-stepping pawn giving a discovered check. Thanks for pointing that out.

So a "perfect" system would test for pins and also test whether the moving side is in check but not by the double-stepping pawn. This is not cheap, of course, which may be the major reason for engines not caring about that special case.

There are zillions of cases where the flags are not relevant due to legality of moves. Even castling flags, like in:
[d]k7/8/8/q7/8/8/8/4K2R w K -
as all legal descendants of this position will be the same with and without castling rights.
Richard.

[Sven]

When I mentioned the "neighbor Pawn is pinned" case I forgot the possibility of the double-stepping pawn giving a discovered check. Thanks for pointing that out.

So a "perfect" system would test for pins and also test whether the moving side is in check but not by the double-stepping pawn. This is not cheap, of course, which may be the major reason for engines not caring about that special case.

There are zillions of cases where the flags are not relevant due to legality of moves. Even castling flags, like in:
[d]k7/8/8/q7/8/8/8/4K2R w K -
as all legal descendants of this position will be the same with and without castling rights.
Richard.

I do not get your point.

All legal descendants of this position have zero castling rights since the wKing must move.

If White does *not* have the castling right in the beginning then a repetition is possible (e.g. Ke2 Qb5+ Ke1 Qa5+).

The given position *with* the castling right is not repeatable since according to the rules the original position is different from the position after Ke2 Qb5+ Ke1 Qa5+. IMO it does not matter that White has no other legal response to the check than moving the king, the castling availability in the first position makes the position different from the second occurrence (after returning). Castling may be "temporarily unavailable" due to blocking pieces or being in check, and in case of check I think it is not relevant that moving the king is the only choice.

So based on my rule interpretation there is no reasonable way for a program to miss a repetition due to setting its internal castling flags wrongly. This makes the difference to the ep square case that has been discussed here.

But I may be wrong regarding the rules. When acting as an OTB chess arbiter, which is what I sometimes do, I would decide as described above in case of a draw claim based on threefold repetition. When this is proven to be wrong, it would be necessary to add the "castle rights intact but king must move out of check" to the list of special cases where more dynamic checks would be "in theory" necessary to get internal move availability information correct.

Once again: this is clearly not only about conversion from/to FEN, it would be relevant at each node during search where such a special case occurs. Having an ep target square or castling flags set or not set clearly affects the hash key and therefore repetition detection.

Sven

[bob]

Both end in identical positions, but FEN demands for different FEN strings. XFEN instead decides to not use any e.p. flag in both cases. Thus XFEN reduces such divergences without creating any problem.

This is a non-issue. Due to compatibility reasons you'll need to check for valid EP while parsing anyway, and when writing FEN there is no compatibility issue when you do not print a field in the ep square either.
Richard.

It's an issue if you process the FEN as text. For example, in unix, doing this:

sort FEN-file | uniq | wc -l

gives a wrong answer since the same position with and without EP target will be counted separately.

[michiguel]

Both end in identical positions, but FEN demands for different FEN strings. XFEN instead decides to not use any e.p. flag in both cases. Thus XFEN reduces such divergences without creating any problem.

This is a non-issue. Due to compatibility reasons you'll need to check for valid EP while parsing anyway, and when writing FEN there is no compatibility issue when you do not print a field in the ep square either.
Richard.

It's an issue if you process the FEN as text. For example, in unix, doing this:

sort FEN-file | uniq | wc -l

gives a wrong answer since the same position with and without EP target will be counted separately.

I agree that the ep should have been designed differently, but your are not supposed to trust FENs 100% for that purpose anyway. There are positions that are virtually identical with a different FEN, and different positions with the same FEN. Same positions with different counters (very common in certain openings), or different positions because of 3-fold rep, etc. In other words, processing this is an approximation no matter what you do, because history is not contained in the positions. But I agree, the ep in the current situation makes things way much more difficult.

Miguel

[bob]

Both end in identical positions, but FEN demands for different FEN strings. XFEN instead decides to not use any e.p. flag in both cases. Thus XFEN reduces such divergences without creating any problem.

This is a non-issue. Due to compatibility reasons you'll need to check for valid EP while parsing anyway, and when writing FEN there is no compatibility issue when you do not print a field in the ep square either.
Richard.

It's an issue if you process the FEN as text. For example, in unix, doing this:

sort FEN-file | uniq | wc -l

gives a wrong answer since the same position with and without EP target will be counted separately.

I agree that the ep should have been designed differently, but your are not supposed to trust FENs 100% for that purpose anyway. There are positions that are virtually identical with a different FEN, and different positions with the same FEN. Same positions with different counters (very common in certain openings), or different positions because of 3-fold rep, etc. In other words, processing this is an approximation no matter what you do, because history is not contained in the positions. But I agree, the ep in the current situation makes things way much more difficult.

Miguel

The funny thing is that the SAN rules were well-defined so that producing SAN is safe and never produces ambiguous moves. And if you follow the PGN SAN standard, you will get the same ASCII each and every time the same move is played, assuming nothing is different enough to change things (disambiguation for example, where if you have two pieces that can reach the same square, vs a position where only one can reach that square).

When I first discovered this "bug" I assumed that the FEN was wrong. Because if I set an EP target square in Crafty, my program will break hashing because it will reset this non-existant target square in the hash signature. I don't think Crafty of today will break, although in years past it would actually crash. But this will cause hashing issues. I do carefully check the EP field and if it is bogus, I ignore it when setting up a position. But checking the position should not be necessary with a well-defined standard.

[Don]

Both end in identical positions, but FEN demands for different FEN strings. XFEN instead decides to not use any e.p. flag in both cases. Thus XFEN reduces such divergences without creating any problem.

This is a non-issue. Due to compatibility reasons you'll need to check for valid EP while parsing anyway, and when writing FEN there is no compatibility issue when you do not print a field in the ep square either.
Richard.

It's an issue if you process the FEN as text. For example, in unix, doing this:

sort FEN-file | uniq | wc -l

gives a wrong answer since the same position with and without EP target will be counted separately.

I agree with Bob on this.

That is exactly why I like the en-passant target to make sense. I have done that exact thing when processing positions in unix. I did not want what is essentially the same positions to be represented in different ways. To me it's just insane that you cannot take a unique set of fen strings and expect them to correspond to the same number of unique positions. (Of course I realize that technically they are not unique if the move ordering is different, but for all practical purposes and for my purposes it's ok to process them as if they are irreversible.)

[jhaglund]

while that is true, the following is also true. The following snippets are all legal and identical, except that a character string match will fail:

8/4n3/...
8/22n3/...
8/1111n111/...

etc.

The idea is to provide something like SAN, which is pretty well defined, to produce the shortest (but consistent) move representation,

rather than the ugly long algebraic stuff like e2e4 and such.

..but i like my coordinate notation.

An EP target when no EP is possible makes little sense to me.

Yes, agreed. I don't like the idea of a EP target move in a FEN. Rather it should be a special character within the FEN to determine if the advanced pawn is able to en passant.

* astricks look like a simple solution to all the problems.

without the astrisk there would be no EP move possible.

Just add a check for '*' in your FEN reader/writer and you'd be fine.

Just add '*' if last side to move was a pawn move advanced 2 spaces, passing any enemy pawns left/right. This doesn't have to be anything different then say a check (+) symbol would be done after a move. Like in displaying the PV lines...

example:

rnbqkbnr/p1p1pppp/1p6/3pP3/8/8/PPPP1PPP/RNBQKBNR w KQkq d6 0 3

would be written:

rnbqkbnr/p1p1pppp/1p6/3pP*3/8/8/PPPP1PPP/RNBQKBNR w KQkq 0 3

This would tell the chess program that e5* has the opportunity to en passant on the next move, so make sure to look for en passant in your search.

Pretty simple.

[hgm]

I considered this idea too, but it would lead to some problems with Berolina Chess, where Pawns capture straight ahead, and non-capture diagonally. In that case indicating only the square to which the capture can go can be ambiguous. With a separate double-push field, you could simply write the pawn move there (disambiguated as the need arises, because a Pawn on e4 could have come from c2 or g2). I guess this could be handled by putting asterisks on both the e.p. square and the fromSquare of the pushed Pawn.

This system would also help in 10x10 variants where a Pawn can have a triple push from the 2nd rank, and a double push from the 3rd rank. If needed all squares passed by the Pawn could be marked.

[Don]

Both end in identical positions, but FEN demands for different FEN strings. XFEN instead decides to not use any e.p. flag in both cases. Thus XFEN reduces such divergences without creating any problem.

This is a non-issue. Due to compatibility reasons you'll need to check for valid EP while parsing anyway, and when writing FEN there is no compatibility issue when you do not print a field in the ep square either.
Richard.

You are looking at this from only one narrow point of view. What you say is all well and good if all you care about is loading a position into your program, but not good if you are processing a set of these positions and want to avoid duplicates for example.

Of course you can work around this by doing some extra programming, such as loading them up into a program and then tracking them by their positional hash or doing something like that. But having to do this exposes the weakness of that approach.

fen is just fine for reading and writing by itself, there is no issue with bugs or loading the wrong position or anything like that. But have you ever had a bunch of PGN games that were non-standard? A good parser might be forgiving and not care whether you say "o-o", "O-O", "0-0" or "Kg1" but if you follow the standard there is only 1 way to represent any given move from a given position. That is how it should be.