Excuses for Wishy Washy FEN/EPD/PGN implementation

[mar]

the point is that a position with spurious ep will hash to something else. engines won't choke on that, sure.
it should be responsibility of whoever outputs FENs/EPDs to validate such things, ideally (which is not the engines, obviously).
but then one could always filter EPD sets through external tools to clean up the FENs, for example.

this thread seems like too much hassle for nothing, because invalid FENs/EPDs will happen regardless of what the standard says

[chrisw]

At the risk of misunderstanding you (again), there is no standard as to how individual engine programmers generate moves

This thread started with:

>And what about the en passant square?
Oh yeah we definitely set that, but only if there's an adjacent pawn of the opposite color."

Setting a needless e.p. target square should be an imprisonable offense.
I suggest this, because I doubt the death penalty would be acceptable to the squeamish.

Which doesn't seem such a big deal to me, because it is exactly what Stockfish does internally without being forced by anyone.
Then I replyed to this:

The point is that each engine needs to do this because the standard is defective.

He is referring to the FEN standard which in his opinion is defective and force every engine to check if an en-passant move is legal. I pointed out that checking if an en-passant move is legal is part of moves generation,

Nope. There is no standard set for engine programmers to check for legality at move generation time, nor should there be. Hence the concept of the pseudo-legal move generator. It's entirely up to the engine designer to decide where he makes the legality check. Delaying the check until it really has to be done (the engine actually decides to try to 'play' that move in the tree) saves much time. For example, you don't need to waste time checking legality of a bunch of moves that are going to be cut anyway.

a component which every engine should possess. So there is no extra burden caused by the FEN standard and I still don't understand what exactly the problem is.

[Deberger]

it should be responsibility of whoever outputs FENs/EPDs to validate such things, ideally (which is not the engines, obviously).

It IS responsibility of whoever outputs FENs/EPDs to validate such things.

In the other forum, I urged users to complain, this month, to correct any such errors they encounter in chess software.

This month because some developers are more productive than ever, thanks to skipping various face-to-face meetings.

And the goal is interoperability.

Your users should be able to copy something from your application, and paste into another application without introducing an error.

Recently I've seen a horrible trend where internationalization has a higher priority than interoperability.

User: "The FEN string is wrong here."

Developer: "That's not really bug, but I'll grudgingly make it a feature request."

User: "This how to say Engine Settings in Esperanto"

Developer: "OMG, I'll check that in right away!"

[mar]

It IS responsibility of whoever outputs FENs/EPDs to validate such things.

In the other forum, I urged users to complain, this month, to correct any such errors they encounter in chess software.

This month because some developers are more productive than ever, thanks to skipping various face-to-face meetings.

And the goal is interoperability.

Your users should be able to copy something from your application, and paste into another application without introducing an error.

Recently I've seen a horrible trend where internationalization has a higher priority than interoperability.

User: "The FEN string is wrong here."

Developer: "That's not really bug, but I'll grudgingly make it a feature request."

User: "This how to say Engine Settings in Esperanto"

Developer: "OMG, I'll check that in right away!"

Cool story.
I agree that a program should work in the first place, i.e. it should not crash or freeze and should do what it's supposed to do - and these should be fixed in the first place (Visual Studio IDE instead introduces new bugs with each new update, which makes me sad and I'm considering switching to something less powerful but more stable).

Translations, well... everyone should learn English and there would be no need for any translations, ever.

In fact, I hate Czech translations of software, for example, as I find them both bad and hard to parse (translating technical terms is always a problem) => so I always switch to English anyway.

There are many problems with localization, you need to find someone to translate your program and incrementaly translate new things as you add new features. Also some texts can be simple and make sense in say English, but in language B you may actually need different texts where you just need one in English - this creates problems for translators and results in confused users.

There are UI alignment problems, say someone translates x so that it now takes twice as many characters. Also, some languages write right to left
and you have to re-align all UI elements from right to left, for exotic scripts you need fonts that support them and so on. Some scripts use different characters based on position in the word etc. etc.

Unless you want to make money with your program (and target China), I don't see translation being worth it except being a giant paint in the ass.

I don't see how EPD processing software would benefit from translations either.

[hgm]

I don't encounter many of those types of positions, but bad e.p. pawns are in the millions in data that I have collected,

That is because the FEN standard prescribes it, and most people stick to the standard.

The problem is that the standard is not like you would like it to be. The 'e.p.' field doesn't indicate whether e.p. capture is possible and legal, it is in fact a double-push field that tells you a certain Pawn was doubly pushed on the preceding ply. Often this is redundant information as far as the game state is concerned. And it forces the user of the FEN to derive whether there actually are e.p. rights in a non-trivial way.

[Deberger]

I don't encounter many of those types of positions, but bad e.p. pawns are in the millions in data that I have collected,

That is because the FEN standard prescribes it, and most people stick to the standard.

Most software is sticking to some hybrid standard, which is removing the bad ep flags unless it's too much trouble to calculate.

And concerning the standard, I recently learned it was corrected and updated in this very forum!

http://talkchess.com/forum3/viewtopic.p ... 25#p588125

The original FEN specification is incorrect concerning the en passant target square.

The correct version is: The en passant target square which appears as the fourth field of a FEN record shall be "-" (nil) if no en passant capture is available. If an en passant capture is available, then the en passant target square will be the name of the destination square of an en passant capture. Note that for a non-nil en passant target square in a FEN string there will be at least one en passant capture in the position described by the FEN string; there may be two.

(en passant target square == "-") if and only if (no en passant move available)

(en passant target square != "-") if and only if (at least one en passant move available)

Simple, really. If a program is to generate a FEN record, then the program needs to know how to generate moves. There is no need for complex retro-analysis at all.

Yes, the twenty year old version of the documentation is out of date. But the above is the only change from the original, so if you've read the above then you're up-to-date.

[hgm]

Well, apparently these updated specs were never able to make it to the places where it matters. What is said here on the forum carries little weight. And when you change a standard, there will of course always remain lots of material around that conforms to the old standard. Can't be done, really.

[Deberger]

What is said here on the forum carries little weight.

That's to be expected.

However, in this case we are talking about FEN,

Which is the Forsyth-Edwards Notation,

And the 1995 specs were written by Steven J. Edwards.

Yes that's exactly the same man who posted the update.

The problem is not that the forum lacks gravitas.

The problem is developers who are arrogant and lazy.

Well, apparently these updated specs were never able to make it to the places where it matters.

That is unfortunate.

[chrisw]

What is said here on the forum carries little weight.

That's to be expected.

However, in this case we are talking about FEN,

Which is the Forsyth-Edwards Notation,

And the 1995 specs were written by Steven J. Edwards.

Yes that's exactly the same man who posted the update.

The problem is not that the forum lacks gravitas.

The problem is developers who are arrogant and lazy.

Well, apparently these updated specs were never able to make it to the places where it matters.

That is unfortunate.

Notably, the developers who have done the most for interoperability have been commercial chess programmers. RS232 (remember that?) was free software plus specs from Chrilly Donninger. UCI was an open spec designed by Stefan Meyer Kahlen. Both enabled direct linking of one engine to another. Both were heavily criticised (as crap) by academic chess programmers. If it wasn’t for those two we’ld quite probably still be typing in moves. Commercial programmers are way more user-oriented, they have to be.

[abulmo2]

Delaying the check until it really has to be done (the engine actually decides to try to 'play' that move in the tree) saves much time. For example, you don't need to waste time checking legality of a bunch of moves that are going to be cut anyway.

A legal move generator do not check moves for legality, it just avoids generating illegal moves. Personally I think it is a waste of time to generate a move that is illegal and another waste of time to verify the legality of a move that is legal. I do not think there is a significative speed difference between both methods. Even staged move generation, which tries to play moves from the hashtable, killer move list, etc. without generating them is not a speed winner by a huge margin. All this is more a matter a taste, in my opinion, than an argument of obvious technical superiority.