Position Causes Stockfish and Komodo To Crash

[syzygy]

So if UCI leaves it to the GUI, which it does according to its creator, it should also specify the requirements of a "valid" position (since the FIDE definition of a legal position is way too cumbersome to implement correctly).

I wonder what happens if we introduce the following two concessions:

• We do not care about the full-move and half-move clocks; they just need to be integers in some defined range.
• We assume that pawns were allowed to move backwards (but not to the base rank), up to but not including the last move.

It seems to me this reduces the problem from a near-impossible one to merely a difficult one. The former fixes a lot of parity issues and such, mainly related to the openings. The latter makes sure we don't need to go through great contortions to invalidate positions such as “starting position but with the rook on a1 moved to a3” (now possible since the pawn could have moved to a4, the rook moved out and then the pawn back), which are illegal under the strict FIDE definition but irrelevant for almost any kind of engine algorithm.

This still leaves some tricky issues about checks; there's the double-pawn-check as mentioned earlier (and similar situations, such as double-bishop-checks), and also some funny issues about X-ray checks: K1qq3k/8/8/8/8/6b1/8/8 (white to move) is legal but K1qq3k/8/8/8/8/7b/8/8 is not!

It may be easier to directly specify conditions on the position itself, which is probably equivalent to what you propose:
- restrictions on the numbers of white pieces of each type, and same for black pieces (consistent with these numbers being reachable by captures and promotions);
- no pawns on 1st or 8th rank;
- castling rights consistent with the board position;
- the side to move cannot directly capture the enemy king;
- if the side to move is in check, the situation should be "reachable".

The last condition is indeed tricky, but it will be just a finite list of situations to check.

Why is
[d]K1qq3k/8/8/8/8/7b/8/8 w
not reachable?
[d]K1Rq3k/8/8/8/8/7b/8/2q5 b - - 0 1

[Michel]

So if UCI leaves it to the GUI, which it does according to its creator, it should also specify the requirements of a "valid" position (since the FIDE definition of a legal position is way too cumbersome to implement correctly).

I wonder what happens if we introduce the following two concessions:

• We do not care about the full-move and half-move clocks; they just need to be integers in some defined range.
• We assume that pawns were allowed to move backwards (but not to the base rank) and sideways, up to but not including the last move.

It seems to me this reduces the problem from a near-impossible one to merely a difficult one. The former fixes a lot of parity issues and such, mainly related to the openings. The latter makes sure we don't need to go through great contortions to invalidate positions such as “starting position but with the rook on a1 moved to a3” (now possible since the pawn could have moved to a4, the rook moved out and then the pawn back), which are illegal under the strict FIDE definition but irrelevant for almost any kind of engine algorithm.

This still leaves some tricky issues about checks; there's the double-pawn-check as mentioned earlier (and similar situations, such as double-bishop-checks), and also some funny issues about X-ray checks: K1qq3k/8/8/8/8/6b1/8/8 (white to move) is legal but K1qq3k/8/8/8/8/7b/8/8 is not!

Personally I do not see the point of requiring that a legal position can be reached from the starting position, or even that it has a parent. This should be irrelevant for engine search.

If one drops this condition then the number of things to check is not so large.

I don't actually know how SF fares if you feed it exotic positions (e.g. with 32 queens).

[Fulvio]

Note that is entirely the same as the Stockfish/GUI case. ld is supposed to process valid .o files (produced by a c-compiler or perhaps assembler). So it would be "within specs" if it does just that and crashes on everything else.

Not even close.
Nobody ever claimed that it's ok for a program like Word to crash with invalid files, just because it is a binary format created by other programs.
What has been repeated over and over and over it's that it depends on how much you can trust your input.
Can you trust a file that can be created by other machines, maybe on a network drive or a faulty usb key?
When I wrote the code that allows to use a network engine in SCID, did I just forward the UCI commands hoping for the best?
Obviously not.
If you want a more fitting example you can think about libc.
It's separated from your program, usually stored as executable code in a different file, but it requires its input to be valid.
Let's pause for a second and breathe (yes, I used the word input and just in a second i will use the c* word, another deep breath).
What's the input of a function like fgets? Its parameters.
So, if I send the wrong input, like a wrong buffer size, it may crash. And that's ok.

[Fulvio]

- the situation should be "reachable".

Just to avoid further confusion, do we all agree that GUIs like Chessbase have been running smoothly for decades without worrying about that?
Before starting to solve the "problem", it wouldn't be better to find at least one position of that type that crash the engine?

[Sesse]

Why is
[d]K1qq3k/8/8/8/8/7b/8/8 w
not reachable?
[d]K1Rq3k/8/8/8/8/7b/8/2q5 b - - 0 1

You're right, I didn't think far enough I was too locked into the “strange situations must be discovered checks” mindset.

[Dann Corbit]

When humans write software there always will be bugs. Imperfection must be accepted.

"position fen ..." is not input for humans. It's meant for other programs.

That being said: Stockfish shouldn't crash on invalid input. (I'm too lazy to write a "patch" anyway.).

Mayhem 2.5 (hopefully) doesn't crash. I haven't tested it super well. Maybe it does.

So you believe that all those EPD positions that show up in the TalkChess general forum are computer generated?

[Sesse]

It seems you don't even need a double-check to make an unreachable position. For instance, how did this happen?

[d]k1R5/2p4K/8/pp1ppppp/rnbq1bnr/8/PPPPPPPP/RNBQ1BN1

OTOH, I guess it should be fairly cheap to backwards-generate all moves, captures and promotions and see if any of them are without check on the “wrong” side.

[Michel]

[d]RNBKQBNR/PPPPPPPP/8/8/8/8/pppppppp/rnbkqbnr w - - 0 1
Wild/5 (upside down chess) is quite popular on FICS. It is of course a position not reachable from the starting position but most engines can play it anyway. The only thing that is important is to cap the depth of quiescence search for otherwise search will grind to a halt after a few queen promotions.

[syzygy]

- the situation should be "reachable".

Just to avoid further confusion, do we all agree that GUIs like Chessbase have been running smoothly for decades without worrying about that?
Before starting to solve the "problem", it wouldn't be better to find at least one position of that type that crash the engine?

Stockfish used to crash with too many queens on the board. The removal of piece lists seems to have fixed that, but there may still be some array boundary violations in the evaluation function, for example. I'm sure there are other engines that still do crash on positions with 20 white queens.

[syzygy]

It seems you don't even need a double-check to make an unreachable position. For instance, how did this happen?

[d]k1R5/2p4K/8/pp1ppppp/rnbq1bnr/8/PPPPPPPP/RNBQ1BN1

This is a very interesting position
In my browser the diagram in your post has extra squares i1, j1, k1, l1 with a black bishop on j1 and a black rook on k1. But in this reply those extra squares seem to disappear...