Position Causes Stockfish and Komodo To Crash

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
syzygy
Posts: 5000
Joined: Tue Feb 28, 2012 10:56 pm

Re: Position Causes Stockfish and Komodo To Crash

Post by syzygy » Sat Dec 19, 2020 11:43 pm

Sesse wrote:
Sat Dec 19, 2020 11:25 pm
syzygy wrote:
Sat Dec 19, 2020 10:37 pm
So if UCI leaves it to the GUI, which it does according to its creator, it should also specify the requirements of a "valid" position (since the FIDE definition of a legal position is way too cumbersome to implement correctly).
I wonder what happens if we introduce the following two concessions:
  • We do not care about the full-move and half-move clocks; they just need to be integers in some defined range.
  • We assume that pawns were allowed to move backwards (but not to the base rank), up to but not including the last move.
It seems to me this reduces the problem from a near-impossible one to merely a difficult one. The former fixes a lot of parity issues and such, mainly related to the openings. The latter makes sure we don't need to go through great contortions to invalidate positions such as “starting position but with the rook on a1 moved to a3” (now possible since the pawn could have moved to a4, the rook moved out and then the pawn back), which are illegal under the strict FIDE definition but irrelevant for almost any kind of engine algorithm.

This still leaves some tricky issues about checks; there's the double-pawn-check as mentioned earlier (and similar situations, such as double-bishop-checks), and also some funny issues about X-ray checks: K1qq3k/8/8/8/8/6b1/8/8 (white to move) is legal but K1qq3k/8/8/8/8/7b/8/8 is not!
It may be easier to directly specify conditions on the position itself, which is probably equivalent to what you propose:
- restrictions on the numbers of white pieces of each type, and same for black pieces (consistent with these numbers being reachable by captures and promotions);
- no pawns on 1st or 8th rank;
- castling rights consistent with the board position;
- the side to move cannot directly capture the enemy king;
- if the side to move is in check, the situation should be "reachable".

The last condition is indeed tricky, but it will be just a finite list of situations to check.

Why is
[D]K1qq3k/8/8/8/8/7b/8/8 w
not reachable?
[D]K1Rq3k/8/8/8/8/7b/8/2q5 b - - 0 1

Michel
Posts: 2219
Joined: Sun Sep 28, 2008 11:50 pm

Re: Position Causes Stockfish and Komodo To Crash

Post by Michel » Sat Dec 19, 2020 11:46 pm

Sesse wrote:
Sat Dec 19, 2020 11:25 pm
syzygy wrote:
Sat Dec 19, 2020 10:37 pm
So if UCI leaves it to the GUI, which it does according to its creator, it should also specify the requirements of a "valid" position (since the FIDE definition of a legal position is way too cumbersome to implement correctly).
I wonder what happens if we introduce the following two concessions:
  • We do not care about the full-move and half-move clocks; they just need to be integers in some defined range.
  • We assume that pawns were allowed to move backwards (but not to the base rank) and sideways, up to but not including the last move.
It seems to me this reduces the problem from a near-impossible one to merely a difficult one. The former fixes a lot of parity issues and such, mainly related to the openings. The latter makes sure we don't need to go through great contortions to invalidate positions such as “starting position but with the rook on a1 moved to a3” (now possible since the pawn could have moved to a4, the rook moved out and then the pawn back), which are illegal under the strict FIDE definition but irrelevant for almost any kind of engine algorithm.

This still leaves some tricky issues about checks; there's the double-pawn-check as mentioned earlier (and similar situations, such as double-bishop-checks), and also some funny issues about X-ray checks: K1qq3k/8/8/8/8/6b1/8/8 (white to move) is legal but K1qq3k/8/8/8/8/7b/8/8 is not!
Personally I do not see the point of requiring that a legal position can be reached from the starting position, or even that it has a parent. This should be irrelevant for engine search.

If one drops this condition then the number of things to check is not so large.

I don't actually know how SF fares if you feed it exotic positions (e.g. with 32 queens).
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

Fulvio
Posts: 244
Joined: Fri Aug 12, 2016 6:43 pm

Re: Position Causes Stockfish and Komodo To Crash

Post by Fulvio » Sun Dec 20, 2020 8:41 am

Michel wrote:
Sat Dec 19, 2020 11:04 pm
Note that is entirely the same as the Stockfish/GUI case. ld is supposed to process valid .o files (produced by a c-compiler or perhaps assembler). So it would be "within specs" if it does just that and crashes on everything else.
Not even close.
Nobody ever claimed that it's ok for a program like Word to crash with invalid files, just because it is a binary format created by other programs.
What has been repeated over and over and over it's that it depends on how much you can trust your input.
Can you trust a file that can be created by other machines, maybe on a network drive or a faulty usb key?
When I wrote the code that allows to use a network engine in SCID, did I just forward the UCI commands hoping for the best?
Obviously not.
If you want a more fitting example you can think about libc.
It's separated from your program, usually stored as executable code in a different file, but it requires its input to be valid.
Let's pause for a second and breathe (yes, I used the word input and just in a second i will use the c* word, another deep breath).
What's the input of a function like fgets? Its parameters.
So, if I send the wrong input, like a wrong buffer size, it may crash. And that's ok.
Last edited by Fulvio on Sun Dec 20, 2020 8:51 am, edited 1 time in total.

Fulvio
Posts: 244
Joined: Fri Aug 12, 2016 6:43 pm

Re: Position Causes Stockfish and Komodo To Crash

Post by Fulvio » Sun Dec 20, 2020 8:50 am

syzygy wrote:
Sat Dec 19, 2020 11:43 pm
- the situation should be "reachable".
Just to avoid further confusion, do we all agree that GUIs like Chessbase have been running smoothly for decades without worrying about that?
Before starting to solve the "problem", it wouldn't be better to find at least one position of that type that crash the engine?

Sesse
Posts: 284
Joined: Mon Apr 30, 2018 9:51 pm
Contact:

Re: Position Causes Stockfish and Komodo To Crash

Post by Sesse » Sun Dec 20, 2020 9:24 am

syzygy wrote:
Sat Dec 19, 2020 11:43 pm
Why is
[D]K1qq3k/8/8/8/8/7b/8/8 w
not reachable?
[D]K1Rq3k/8/8/8/8/7b/8/2q5 b - - 0 1
You're right, I didn't think far enough :-) I was too locked into the “strange situations must be discovered checks” mindset.

Dann Corbit
Posts: 12149
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Position Causes Stockfish and Komodo To Crash

Post by Dann Corbit » Sun Dec 20, 2020 11:05 am

JohnWoe wrote:
Fri Dec 18, 2020 8:28 pm
When humans write software there always will be bugs. Imperfection must be accepted.

"position fen ..." is not input for humans. It's meant for other programs.

That being said: Stockfish shouldn't crash on invalid input. (I'm too lazy to write a "patch" anyway.).

Mayhem 2.5 (hopefully) doesn't crash. I haven't tested it super well. Maybe it does. :lol:
So you believe that all those EPD positions that show up in the TalkChess general forum are computer generated?
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Sesse
Posts: 284
Joined: Mon Apr 30, 2018 9:51 pm
Contact:

Re: Position Causes Stockfish and Komodo To Crash

Post by Sesse » Sun Dec 20, 2020 11:12 am

It seems you don't even need a double-check to make an unreachable position. For instance, how did this happen?

[D]k1R5/2p4K/8/pp1ppppp/rnbq1bnr/8/PPPPPPPP/RNBQ1BN1

OTOH, I guess it should be fairly cheap to backwards-generate all moves, captures and promotions and see if any of them are without check on the “wrong” side.

Michel
Posts: 2219
Joined: Sun Sep 28, 2008 11:50 pm

Re: Position Causes Stockfish and Komodo To Crash

Post by Michel » Sun Dec 20, 2020 11:53 am


Wild/5 (upside down chess) is quite popular on FICS. It is of course a position not reachable from the starting position but most engines can play it anyway. The only thing that is important is to cap the depth of quiescence search for otherwise search will grind to a halt after a few queen promotions.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

syzygy
Posts: 5000
Joined: Tue Feb 28, 2012 10:56 pm

Re: Position Causes Stockfish and Komodo To Crash

Post by syzygy » Sun Dec 20, 2020 12:44 pm

Fulvio wrote:
Sun Dec 20, 2020 8:50 am
syzygy wrote:
Sat Dec 19, 2020 11:43 pm
- the situation should be "reachable".
Just to avoid further confusion, do we all agree that GUIs like Chessbase have been running smoothly for decades without worrying about that?
Before starting to solve the "problem", it wouldn't be better to find at least one position of that type that crash the engine?
Stockfish used to crash with too many queens on the board. The removal of piece lists seems to have fixed that, but there may still be some array boundary violations in the evaluation function, for example. I'm sure there are other engines that still do crash on positions with 20 white queens.

syzygy
Posts: 5000
Joined: Tue Feb 28, 2012 10:56 pm

Re: Position Causes Stockfish and Komodo To Crash

Post by syzygy » Sun Dec 20, 2020 1:08 pm

Sesse wrote:
Sun Dec 20, 2020 11:12 am
It seems you don't even need a double-check to make an unreachable position. For instance, how did this happen?

[D]k1R5/2p4K/8/pp1ppppp/rnbq1bnr/8/PPPPPPPP/RNBQ1BN1
This is a very interesting position :mrgreen:
In my browser the diagram in your post has extra squares i1, j1, k1, l1 with a black bishop on j1 and a black rook on k1. But in this reply those extra squares seem to disappear...

Post Reply