Position Causes Stockfish and Komodo To Crash

[jshriver]

Was doing some analysis when I came across a position that causes the latest stockfish to crash, also tried with komodo 14.1 and they both segfault.

uci
ucinewgame
position fen 1B1b1k2/7p/1ppN3R/5Ppb/8/8/PP2rKPP/8 # komodo crashes here
go depth 10 # stockfish crashes here

Looking at the board white is in check, so it makes sense if it tries to analyze for black, so i added w - - to the epd and it worked fine.

Not sure if this is useful, I would hope it would send back invalid position or something along those lines rather than a segfault.

[Peperoni]

Hum on Lichess, it doesn't crash.

[Dann Corbit]

Was doing some analysis when I came across a position that causes the latest stockfish to crash, also tried with komodo 14.1 and they both segfault.

uci
ucinewgame
position fen 1B1b1k2/7p/1ppN3R/5Ppb/8/8/PP2rKPP/8 # komodo crashes here
go depth 10 # stockfish crashes here

Looking at the board white is in check, so it makes sense if it tries to analyze for black, so i added w - - to the epd and it worked fine.

Not sure if this is useful, I would hope it would send back invalid position or something along those lines rather than a segfault.

There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.

[Ajedrecista]

Hello:

I am not a programmer, but I have found the following things at position.cpp file since SF is open source:

Code: Select all

/// Position::set() initializes the position object with the given FEN string.
/// This function is not very robust - make sure that input FENs are correct,
/// this is assumed to be the responsibility of the GUI.

Code: Select all

  if (   (sideToMove != WHITE && sideToMove != BLACK)
      || piece_on(square<KING>(WHITE)) != W_KING
      || piece_on(square<KING>(BLACK)) != B_KING
      || (   ep_square() != SQ_NONE
          && relative_rank(sideToMove, ep_square()) != RANK_6))
      assert(0 && "pos_is_ok: Default");

Which might explain SF behaviour. Not being a programmer is a real handicap in order to conclude it or not once seen the code.

OTOH, looking at FEN standards thanks to the link provided by the excellent CPW:

======================================================================
TABLE OF CONTENTS
======================================================================
======================================================================

[...]

16.1: FEN
16.1.1: History
16.1.2: Uses for a position notation
16.1.3: Data fields
16.1.3.1: Piece placement data
16.1.3.2: Active color
16.1.3.3: Castling availability
16.1.3.4: En passant target square
16.1.3.5: Halfmove clock
16.1.3.6: Fullmove number
16.1.4: Examples

[...]

16.1: FEN

FEN is "Forsyth-Edwards Notation"; it is a standard for describing chess positions using the ASCII character set.

A single FEN record uses one text line of variable length composed of six data fields. The first four fields of the FEN specification are the same as the first four fields of the EPD specification.

A text file composed exclusively of FEN data records should have a file name with the suffix ".fen".

[...]

16.1.3: Data fields

FEN specifies the piece placement, the active color, the castling availability, the en passant target square, the halfmove clock, and the fullmove number. These can all fit on a single text line in an easily read format. The length of a FEN position description varies somewhat according to the position. In some cases, the description could be eighty or more characters in length and so may not fit conveniently on some displays. However, these positions aren't too common.

A FEN description has six fields. Each field is composed only of non-blank printing ASCII characters. Adjacent fields are separated by a single ASCII space character.

[...]

16.1.3.2: Active color

The second field represents the active color. A lower case "w" is used if White is to move; a lower case "b" is used if Black is the active player.

[...]

Bold was added by me. Despite black can not be the side to move in this particular position because the other side is in check, I see no reason to omit the side to move field in FEN because I understand that this is a compulsory field and regardless of the position.

Regards from Spain.

Ajedrecista.

[Uri Blass]

Was doing some analysis when I came across a position that causes the latest stockfish to crash, also tried with komodo 14.1 and they both segfault.

uci
ucinewgame
position fen 1B1b1k2/7p/1ppN3R/5Ppb/8/8/PP2rKPP/8 # komodo crashes here
go depth 10 # stockfish crashes here

Looking at the board white is in check, so it makes sense if it tries to analyze for black, so i added w - - to the epd and it worked fine.

Not sure if this is useful, I would hope it would send back invalid position or something along those lines rather than a segfault.

There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.

When you collect positions from the internet it is enough to use one function to test if the positions are bad and get rid of them.
I see no reasons that every engine's author has to write a function to check if the position is a bad position and the simplest solution is to have a tool that test if the position is good before giving it to an engine.

[Dann Corbit]

Was doing some analysis when I came across a position that causes the latest stockfish to crash, also tried with komodo 14.1 and they both segfault.

uci
ucinewgame
position fen 1B1b1k2/7p/1ppN3R/5Ppb/8/8/PP2rKPP/8 # komodo crashes here
go depth 10 # stockfish crashes here

Looking at the board white is in check, so it makes sense if it tries to analyze for black, so i added w - - to the epd and it worked fine.

Not sure if this is useful, I would hope it would send back invalid position or something along those lines rather than a segfault.

There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.

When you collect positions from the internet it is enough to use one function to test if the positions are bad and get rid of them.
I see no reasons that every engine's author has to write a function to check if the position is a bad position and the simplest solution is to have a tool that test if the position is good before giving it to an engine.

Programs which do not perform sensible checks on input and allow undefined behavior are incompetent.
This exact thing was the cause of the first ever computer virus, the internet worm in 1988:
https://en.wikipedia.org/wiki/Morris_worm
If you know of a bug in your program which causes potentially extremely harmful undefined behavior and do not correct it, I think that is a negligent action. Of course, chess hobby programmers do not care if their sloppy program causes damage.

I wrote some of my own routines to check data, and I also use tools written by others.
I have written, for instance, a simple routine which removes all castle flags when the given castle move is not possible.
That was one source of crashes.
I have many other legality checks, and my friend Les Fernandez wrote a comprehensive EPD legality checker that I use all the time.
About 97% of all e.p. flags (other than '-') in the wild are spurious.

There are tons of EPD records with the wrong dash character or no intervening space.
There are thousands of positions with no side to move, castle rights, or e.p. status (w - - is assumed).
Yes, the original problem writers should have written them correctly. But they are humans, so I don't *expect* perfect data from them.

[syzygy]

If you know of a bug in your program which causes potentially extremely harmful undefined behavior and do not correct it, I think that is a negligent action. Of course, chess hobby programmers do not care if their sloppy program causes damage.

The "rm" command can do huge damage to your (Unix) system too, and you don't even need to give it illegal input.

If you blindly download stuff from the internet and feed it to a UCI engine (which could not even report an error to the GUI), then your action is negligent.

[mar]

There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.

what a lame excuse - crashing on a bad FEN is pathetic (especially when a fix is trivial)

[Dann Corbit]

If you know of a bug in your program which causes potentially extremely harmful undefined behavior and do not correct it, I think that is a negligent action. Of course, chess hobby programmers do not care if their sloppy program causes damage.

The "rm" command can do huge damage to your (Unix) system too, and you don't even need to give it illegal input.

If you blindly download stuff from the internet and feed it to a UCI engine (which could not even report an error to the GUI), then your action is negligent.

Yes, of course. it is the tool user's fault.
Same goes for all those people who used finger and sendmail when Morris wrote the worm, because the inputs were not correct.
And passing along bad data to an application totally excuses the authors of the application, as everyone knows.
Why, yes, this is the true genesis of all quality software. Assume that all inputs are perfect and that nothing can go wrong.

[AndrewGrant]

Someone recently wasted their time writing up 20 paragraphs showing how they could "exploit" Stockfish into crashing....

If you send a chess engine garbage, you should expect garbage. If you don't know it is garbage, then maybe a GUI should be doing it for you.

The above has been affirmed many times in Stockfish PRs, where users come saying they have found a "bug"