Position Causes Stockfish and Komodo To Crash

[RubiChess]

What do you think about this kind of position?

KQQQQQQQ/QQQQQQQQ/QQQQQQQQ/QQQqqqqq/QQQqqqqq/QQQqqqqq/QQQqqqqq/QQQqqqqk b - - 1 1

GUIs might (and do) accept this as a "puzzle" position although it is not legal in terms of reachable from the start position. Which is totally okay in my opinion.
Stockfish crashes on it.

Regards, Andreas

[Ras]

I think that crashing on invalid input is never OK.

My engine would reject this position with "info string error (illegal position: too many pieces)". A subsequent "go" command would be answered with "info string error (illegal position)" and "bestmove 0000".

[mvanthoor]

Was doing some analysis when I came across a position that causes the latest stockfish to crash, also tried with komodo 14.1 and they both segfault.

uci
ucinewgame
position fen 1B1b1k2/7p/1ppN3R/5Ppb/8/8/PP2rKPP/8 # komodo crashes here
go depth 10 # stockfish crashes here

Looking at the board white is in check, so it makes sense if it tries to analyze for black, so i added w - - to the epd and it worked fine.

Not sure if this is useful, I would hope it would send back invalid position or something along those lines rather than a segfault.

The FEN is incomplete. Both engines seem to set it up as far as it goes, then miss some information, and crash when searching.

My own engine refuses to set up the position:

Code: Select all

$ ./target/release/rustic.exe --fen "1B1b1k2/7p/1ppN3R/5Ppb/8/8/PP2rKPP/8"

d888888b                      dP   oo
88     88                     88
88oooo88  88    88  d8888b  d8888P dP d88888b
88    88  88    88  8ooooo    88   88 88
88     88 88    88       88   88   88 88
88     88  88888P  888888P    dP   dP 888888P
ooooooooooooooooooooooooooooooooooooooooooooo

Engine: Rustic Alpha 1
Author: Marcel Vanthoor
EMail: mail@marcelvanthoor.nl
Website: https://rustic-chess.org/
Protocol: uci
Threads: 1

Error code 0: FEN: Must have six parts

$

[mvanthoor]

What do you think about this kind of position?

KQQQQQQQ/QQQQQQQQ/QQQQQQQQ/QQQqqqqq/QQQqqqqq/QQQqqqqq/QQQqqqqq/QQQqqqqk b - - 1 1

GUIs might (and do) accept this as a "puzzle" position although it is not legal in terms of reachable from the start position. Which is totally okay in my opinion.
Stockfish crashes on it.

Regards, Andreas

Rustic doesn't crash, and the search runs, but even after 2 minutes, there's no output for depth 1. I assume that it chokes on a qsearch explosion.

[gbtami]

There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.

what a lame excuse - crashing on a bad FEN is pathetic (especially when a fix is trivial)

Everyone who is a little bit followed Stockfish development over the years knows that this is their design decision. If you didn't no it, now you know.
See https://github.com/official-stockfish/S ... issues/905

[RubiChess]

There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.

what a lame excuse - crashing on a bad FEN is pathetic (especially when a fix is trivial)

Everyone who is a little bit followed Stockfish development over the years knows that this is their design decision. If you didn't no it, now you know.
See https://github.com/official-stockfish/S ... issues/905

But there is a difference between
1. Syntactically wrong input (like the fen in the issue you linked)
2. Input that obviously violates chess rules like positions with side not to move being checked
3. Positions that may be impossible to reach from start positions but are not falling into 1. and 2. and can be setup by the user of a chess GUI

While positions that fall into 1. and 2. can be filtered by the GUI quite easily it may be very hard or almost impossible for a GUI to filter out 3.
My position in http://talkchess.com/forum3/posting.php ... f#pr876023 falls into 3. and would be easy to detect even by the GUI. But lets assume the user sets up some more tricky position with e.g. some pawns on the same file and engine X has some optimization and crashes on this position knowing that this pawn constellation can never be reached from starting position. Is it the GUI's fault passing this position to the engine? How should the GUI check if this position is "valid"? Whatever valid means. Should the GUI prevent the user to setup an arbitrary position on the board only because some engine may crash on it?

So in my opinion: At least for positions falling into 3. the engine must not crash.

Regards, Andreas

[Dann Corbit]

Someone recently wasted their time writing up 20 paragraphs showing how they could "exploit" Stockfish into crashing....

If you send a chess engine garbage, you should expect garbage. If you don't know it is garbage, then maybe a GUI should be doing it for you.

The above has been affirmed many times in Stockfish PRs, where users come saying they have found a "bug"

Is that how you would write code for a customer? Unexpected input causes a program crash?
If a programmer is too lazy even to think hard, they can still use try catch (yes, I know, a 1% performance penalty).
It boggles my mind how chess programmers will spend 10,000 hours writing their programs but spend zero minutes checking the input for correctness.

[jshriver]

Sorry guys didn't mean to start a fight.

[Nordlandia]

Try same position, just with rooks

[gbtami]

There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.

what a lame excuse - crashing on a bad FEN is pathetic (especially when a fix is trivial)

Everyone who is a little bit followed Stockfish development over the years knows that this is their design decision. If you didn't no it, now you know.
See https://github.com/official-stockfish/S ... issues/905

But there is a difference between
1. Syntactically wrong input (like the fen in the issue you linked)
2. Input that obviously violates chess rules like positions with side not to move being checked
3. Positions that may be impossible to reach from start positions but are not falling into 1. and 2. and can be setup by the user of a chess GUI

While positions that fall into 1. and 2. can be filtered by the GUI quite easily it may be very hard or almost impossible for a GUI to filter out 3.
My position in http://talkchess.com/forum3/posting.php ... f#pr876023 falls into 3. and would be easy to detect even by the GUI. But lets assume the user sets up some more tricky position with e.g. some pawns on the same file and engine X has some optimization and crashes on this position knowing that this pawn constellation can never be reached from starting position. Is it the GUI's fault passing this position to the engine? How should the GUI check if this position is "valid"? Whatever valid means. Should the GUI prevent the user to setup an arbitrary position on the board only because some engine may crash on it?

So in my opinion: At least for positions falling into 3. the engine must not crash.

Regards, Andreas

Sure, you can have your opinion. But at the end of the day Marco Costalba decides