Excuses for Wishy Washy FEN/EPD/PGN implementation

[chrisw]

So what? They may have exactly the same possible moves because they block the R from moving or whatever, but their descendants will be different because of differing castle status flags still holding true. The FEN position data may be the same, but the positions are not.

[d]r1b1k1nr/pp1n1ppp/2pb4/3p4/3PpPPq/2N1P2P/PPP3B1/R1BQK1NR w KQkq -
Did you at least read my message before replying?

Ah! Well, yes and no, I was looking at the immobility of the black rook on h8, but actually you hadn't altered it's castling rights.
Are you implying that an EPD checker should test for incheck with forced move of the king and back project the loss of castling rights to the original EPD? Possible, but a lot of work (which further deters engine programmers btw), we'ld presumably need to do a legal move generation and determine all of them were of the king, and for what gain? It's a practical problem we're trying to solve, adding that sort of complexity to as EPD handler is asking for bugs and what would be the gain in doing it? Looks like all downside for no gain.

Please post an example of this immaginary different descendants.

[Ras]

Can't the function that converts the FEN to the internal representation simply take care of the EP square?

Sanitising the EP square is what I do anyway. The point is that each engine needs to do this because the standard is defective. I also sanitise castling rights, i.e. if they are given when the associated king or rook aren't on their original positons, I just discard the castling rights instead of rejecting the position.

[chrisw]

Can't the function that converts the FEN to the internal representation simply take care of the EP square?

Sanitising the EP square is what I do anyway. The point is that each engine needs to do this because the standard is defective. I also sanitise castling rights, i.e. if they are given when the associated king or rook aren't on their original positons, I just discard the castling rights instead of rejecting the position.

For any engine programmers who need free-to-use verification source, that's all covered in the GitHub code ...

https://github.com/ChrisWhittington/Che ... /README.md

[Fulvio]

The point is that each engine needs to do this because the standard is defective.

I still do not understand.
The minimum for an engine is to be able to generate moves, including the en-passant.
Even if the FEN notation is never used, it must still keep track of the EP square after a double pawn push.
For example Stockfish:
https://github.com/official-stockfish/S ... n.cpp#L795
Note also that Stockfish sets the EP even if the move is not legal and that it checks if the pawn is pinned only afterwards:
https://github.com/official-stockfish/S ... n.cpp#L505

[chrisw]

The point is that each engine needs to do this because the standard is defective.

I still do not understand.
The minimum for an engine is to be able to generate moves, including the en-passant.
Even if the FEN notation is never used, it must still keep track of the EP square after a double pawn push.
For example Stockfish:
https://github.com/official-stockfish/S ... n.cpp#L795
Note also that Stockfish sets the EP even if the move is not legal and that it checks if the pawn is pinned only afterwards:
https://github.com/official-stockfish/S ... n.cpp#L505

At the risk of misunderstanding you (again), there is no standard as to how individual engine programmers generate moves, and all are free to check for legality, ep status and so on when they generate the moves(s), or later.
It’s simply good practice when receiving an EPD/FEN from external source to carry out an integrity check on it and fix anything that can be fixed before passing the data on into the engine. If SF move routines can stand weirdness in the ep field, then I guess SF doesn’t need check off the EPD that way. Me, I prefer not to risk that a future developer modifying the code didn’t realise and just assumes ep field is good, then bug appears, hard to track down, wastes time. verification checking equates to defensive programming.

[Terje]

Removing an ep flag due to inability of opponent to make use of it, or castle status due to inevitably losing it next move will affect 3fold repetitions. Does a repetition of such a position, but without the flag, not count as a new position under normal chess rules?

Obviously pointless for analysis as going in circles is hardly useful, but while playing it could become relevant.

[chrisw]

Removing an ep flag due to inability of opponent to make use of it, or castle status due to inevitably losing it next move will affect 3fold repetitions. Does a repetition of such a position, but without the flag, not count as a new position under normal chess rules?

Obviously pointless for analysis as going in circles is hardly useful, but while playing it could become relevant.

Only if you want to give yourself a headache

[Fulvio]

Looking at this position, how often do you think it would have happened in a real game?

It is much more frequent than pinned pawns which cannot capture en-passant.
Out of curiosity I checked the latest PGN (116GB uncompressed, 55+ million games) from lichess:
https://database.lichess.org/

There are 7.43 million games with al least a rook on A1 or H1(or A8 and H8 for black) and the king on E1 (or E8 for black) which had to move after a check. For example in this position you may expect black to still have the right to castle long:

Code: Select all

DrNykterstein  (3156)   --   penguingim1  (3055)
Rated Bullet game (-)  https://lichess.org/7W5vjLtt
2020.03.25  1-0  C02
Position: r2qk3/pp1brnQR/2n1p3/3pPpb1/3P4/2NBB1P1/PP3PK1/R7 w - - 8 24

24.Qg8# 

In this position instead the long castle flag was set:

Code: Select all

penguingim1  (3105)   --   Jasugi99  (2908)
Rated Bullet game (-)  https://lichess.org/ZLgeYQEu
2020.03.18  0-1  A41
Position: 2kr3b/pppb4/3p2p1/3P2q1/1QP2p2/2N2P2/PP2B3/R3K3 b Q - 0 20

20...Qg3+ 

On the other hand there are only 18.8k games with pseudo-legal en-passant which are non legal because the pawn is pinned (17454)

Code: Select all

DrNykterstein  (3160)   --   penguingim1  (3042)
Rated Bullet game (-)  https://lichess.org/T3ZeKwLv
2020.03.24  0-1  D37
Position: 2b3k1/2q1rp2/7Q/3pP3/1p1P3P/1P1BP3/6PK/q7 b - - 0 31

31...f5 

or due to a discovered check(1330).

Code: Select all

penguingim1  (3130)   --   catask  (2953)
Rated Bullet game (-)  https://lichess.org/SYBZIif5
2020.03.18  0-1  D35
Position: 8/r5pK/7p/7P/6Q1/8/5p1k/2R5 b - - 3 56

56...g5+ 

[Fulvio]

At the risk of misunderstanding you (again), there is no standard as to how individual engine programmers generate moves

This thread started with:

>And what about the en passant square?
Oh yeah we definitely set that, but only if there's an adjacent pawn of the opposite color."

Setting a needless e.p. target square should be an imprisonable offense.
I suggest this, because I doubt the death penalty would be acceptable to the squeamish.

Which doesn't seem such a big deal to me, because it is exactly what Stockfish does internally without being forced by anyone.
Then I replyed to this:

The point is that each engine needs to do this because the standard is defective.

He is referring to the FEN standard which in his opinion is defective and force every engine to check if an en-passant move is legal. I pointed out that checking if an en-passant move is legal is part of moves generation, a component which every engine should possess. So there is no extra burden caused by the FEN standard and I still don't understand what exactly the problem is.

[Fulvio]

Removing an ep flag due to inability of opponent to make use of it, or castle status due to inevitably losing it next move will affect 3fold repetitions. Does a repetition of such a position, but without the flag, not count as a new position under normal chess rules?

Interesting, I don't know. I googled the FIDE handbook:
"Positions are considered the same if and only if the same player has the move, pieces of the same kind and colour occupy the same squares and the possible moves of all the pieces of both players are the same. Thus positions are not the same if:
9.2.2.1 at the start of the sequence a pawn could have been captured en passant
9.2.2.2 a king had castling rights with a rook that has not been moved, but forfeited these after moving. The castling rights are lost only after the king or rook is moved."
"possible moves" is ambiguous.
If it means legal moves than a pinned pawn cannot capture en-passant. However in this case there are many situation when a side has a castling right, but castling is not legal.