Position Causes Stockfish and Komodo To Crash

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
jshriver
Posts: 1242
Joined: Wed Mar 08, 2006 8:41 pm
Location: Morgantown, WV, USA
Contact:

Position Causes Stockfish and Komodo To Crash

Post by jshriver » Wed Dec 09, 2020 7:16 am

Was doing some analysis when I came across a position that causes the latest stockfish to crash, also tried with komodo 14.1 and they both segfault.

uci
ucinewgame
position fen 1B1b1k2/7p/1ppN3R/5Ppb/8/8/PP2rKPP/8 # komodo crashes here
go depth 10 # stockfish crashes here

Looking at the board white is in check, so it makes sense if it tries to analyze for black, so i added w - - to the epd and it worked fine.

Not sure if this is useful, I would hope it would send back invalid position or something along those lines rather than a segfault.

Peperoni
Posts: 45
Joined: Sun Nov 01, 2020 4:27 pm
Full name: Richard Porti

Re: Position Causes Stockfish and Komodo To Crash

Post by Peperoni » Thu Dec 10, 2020 5:13 pm

Hum on Lichess, it doesn't crash.

Dann Corbit
Posts: 12038
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Position Causes Stockfish and Komodo To Crash

Post by Dann Corbit » Thu Dec 10, 2020 7:40 pm

jshriver wrote:
Wed Dec 09, 2020 7:16 am
Was doing some analysis when I came across a position that causes the latest stockfish to crash, also tried with komodo 14.1 and they both segfault.

uci
ucinewgame
position fen 1B1b1k2/7p/1ppN3R/5Ppb/8/8/PP2rKPP/8 # komodo crashes here
go depth 10 # stockfish crashes here

Looking at the board white is in check, so it makes sense if it tries to analyze for black, so i added w - - to the epd and it worked fine.

Not sure if this is useful, I would hope it would send back invalid position or something along those lines rather than a segfault.
There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

User avatar
Ajedrecista
Posts: 1538
Joined: Wed Jul 13, 2011 7:04 pm
Location: Madrid, Spain.
Contact:

Re: Position causes Stockfish and Komodo to crash.

Post by Ajedrecista » Thu Dec 10, 2020 8:09 pm

Hello:

I am not a programmer, but I have found the following things at position.cpp file since SF is open source:

Code: Select all

/// Position::set() initializes the position object with the given FEN string.
/// This function is not very robust - make sure that input FENs are correct,
/// this is assumed to be the responsibility of the GUI.

Code: Select all

  if (   (sideToMove != WHITE && sideToMove != BLACK)
      || piece_on(square<KING>(WHITE)) != W_KING
      || piece_on(square<KING>(BLACK)) != B_KING
      || (   ep_square() != SQ_NONE
          && relative_rank(sideToMove, ep_square()) != RANK_6))
      assert(0 && "pos_is_ok: Default");
Which might explain SF behaviour. Not being a programmer is a real handicap in order to conclude it or not once seen the code.

OTOH, looking at FEN standards thanks to the link provided by the excellent CPW:
======================================================================
TABLE OF CONTENTS
======================================================================
======================================================================

[...]

16.1: FEN
16.1.1: History
16.1.2: Uses for a position notation
16.1.3: Data fields
16.1.3.1: Piece placement data
16.1.3.2: Active color
16.1.3.3: Castling availability
16.1.3.4: En passant target square
16.1.3.5: Halfmove clock
16.1.3.6: Fullmove number
16.1.4: Examples

[...]

16.1: FEN

FEN is "Forsyth-Edwards Notation"; it is a standard for describing chess positions using the ASCII character set.

A single FEN record uses one text line of variable length composed of six data fields. The first four fields of the FEN specification are the same as the first four fields of the EPD specification.

A text file composed exclusively of FEN data records should have a file name with the suffix ".fen".

[...]

16.1.3: Data fields

FEN specifies the piece placement, the active color, the castling availability, the en passant target square, the halfmove clock, and the fullmove number. These can all fit on a single text line in an easily read format. The length of a FEN position description varies somewhat according to the position. In some cases, the description could be eighty or more characters in length and so may not fit conveniently on some displays. However, these positions aren't too common.

A FEN description has six fields. Each field is composed only of non-blank printing ASCII characters. Adjacent fields are separated by a single ASCII space character.

[...]

16.1.3.2: Active color

The second field represents the active color. A lower case "w" is used if White is to move; a lower case "b" is used if Black is the active player.


[...]
Bold was added by me. Despite black can not be the side to move in this particular position because the other side is in check, I see no reason to omit the side to move field in FEN because I understand that this is a compulsory field and regardless of the position.

Regards from Spain.

Ajedrecista.

Uri Blass
Posts: 8948
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Position Causes Stockfish and Komodo To Crash

Post by Uri Blass » Thu Dec 10, 2020 11:03 pm

Dann Corbit wrote:
Thu Dec 10, 2020 7:40 pm
jshriver wrote:
Wed Dec 09, 2020 7:16 am
Was doing some analysis when I came across a position that causes the latest stockfish to crash, also tried with komodo 14.1 and they both segfault.

uci
ucinewgame
position fen 1B1b1k2/7p/1ppN3R/5Ppb/8/8/PP2rKPP/8 # komodo crashes here
go depth 10 # stockfish crashes here

Looking at the board white is in check, so it makes sense if it tries to analyze for black, so i added w - - to the epd and it worked fine.

Not sure if this is useful, I would hope it would send back invalid position or something along those lines rather than a segfault.
There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.
When you collect positions from the internet it is enough to use one function to test if the positions are bad and get rid of them.
I see no reasons that every engine's author has to write a function to check if the position is a bad position and the simplest solution is to have a tool that test if the position is good before giving it to an engine.

Dann Corbit
Posts: 12038
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Position Causes Stockfish and Komodo To Crash

Post by Dann Corbit » Fri Dec 11, 2020 12:22 am

Uri Blass wrote:
Thu Dec 10, 2020 11:03 pm
Dann Corbit wrote:
Thu Dec 10, 2020 7:40 pm
jshriver wrote:
Wed Dec 09, 2020 7:16 am
Was doing some analysis when I came across a position that causes the latest stockfish to crash, also tried with komodo 14.1 and they both segfault.

uci
ucinewgame
position fen 1B1b1k2/7p/1ppN3R/5Ppb/8/8/PP2rKPP/8 # komodo crashes here
go depth 10 # stockfish crashes here

Looking at the board white is in check, so it makes sense if it tries to analyze for black, so i added w - - to the epd and it worked fine.

Not sure if this is useful, I would hope it would send back invalid position or something along those lines rather than a segfault.
There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.
When you collect positions from the internet it is enough to use one function to test if the positions are bad and get rid of them.
I see no reasons that every engine's author has to write a function to check if the position is a bad position and the simplest solution is to have a tool that test if the position is good before giving it to an engine.
Programs which do not perform sensible checks on input and allow undefined behavior are incompetent.
This exact thing was the cause of the first ever computer virus, the internet worm in 1988:
https://en.wikipedia.org/wiki/Morris_worm
If you know of a bug in your program which causes potentially extremely harmful undefined behavior and do not correct it, I think that is a negligent action. Of course, chess hobby programmers do not care if their sloppy program causes damage.

I wrote some of my own routines to check data, and I also use tools written by others.
I have written, for instance, a simple routine which removes all castle flags when the given castle move is not possible.
That was one source of crashes.
I have many other legality checks, and my friend Les Fernandez wrote a comprehensive EPD legality checker that I use all the time.
About 97% of all e.p. flags (other than '-') in the wild are spurious.

There are tons of EPD records with the wrong dash character or no intervening space.
There are thousands of positions with no side to move, castle rights, or e.p. status (w - - is assumed).
Yes, the original problem writers should have written them correctly. But they are humans, so I don't *expect* perfect data from them.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

syzygy
Posts: 4999
Joined: Tue Feb 28, 2012 10:56 pm

Re: Position Causes Stockfish and Komodo To Crash

Post by syzygy » Fri Dec 11, 2020 12:35 am

Dann Corbit wrote:
Fri Dec 11, 2020 12:22 am
If you know of a bug in your program which causes potentially extremely harmful undefined behavior and do not correct it, I think that is a negligent action. Of course, chess hobby programmers do not care if their sloppy program causes damage.
The "rm" command can do huge damage to your (Unix) system too, and you don't even need to give it illegal input.

If you blindly download stuff from the internet and feed it to a UCI engine (which could not even report an error to the GUI), then your action is negligent.

mar
Posts: 2325
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Position Causes Stockfish and Komodo To Crash

Post by mar » Fri Dec 11, 2020 4:30 am

Dann Corbit wrote:
Thu Dec 10, 2020 7:40 pm
There are lots of things that will make SF coredump if the position has an error in it.
I reported some to fishcooking and the response was:
"Don't input bad positions."
Of course, when you collect positions from the internet, you will collect thousands of bad positions.
Nothing like finding 3.5 TB of core files in your crashdump folder in the morning.
what a lame excuse - crashing on a bad FEN is pathetic (especially when a fix is trivial)
Martin Sedlak

Dann Corbit
Posts: 12038
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Position Causes Stockfish and Komodo To Crash

Post by Dann Corbit » Fri Dec 11, 2020 5:43 am

syzygy wrote:
Fri Dec 11, 2020 12:35 am
Dann Corbit wrote:
Fri Dec 11, 2020 12:22 am
If you know of a bug in your program which causes potentially extremely harmful undefined behavior and do not correct it, I think that is a negligent action. Of course, chess hobby programmers do not care if their sloppy program causes damage.
The "rm" command can do huge damage to your (Unix) system too, and you don't even need to give it illegal input.

If you blindly download stuff from the internet and feed it to a UCI engine (which could not even report an error to the GUI), then your action is negligent.
Yes, of course. it is the tool user's fault.
Same goes for all those people who used finger and sendmail when Morris wrote the worm, because the inputs were not correct.
And passing along bad data to an application totally excuses the authors of the application, as everyone knows.
Why, yes, this is the true genesis of all quality software. Assume that all inputs are perfect and that nothing can go wrong.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

AndrewGrant
Posts: 1047
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: Position Causes Stockfish and Komodo To Crash

Post by AndrewGrant » Fri Dec 11, 2020 7:14 am

Someone recently wasted their time writing up 20 paragraphs showing how they could "exploit" Stockfish into crashing....

If you send a chess engine garbage, you should expect garbage. If you don't know it is garbage, then maybe a GUI should be doing it for you.

The above has been affirmed many times in Stockfish PRs, where users come saying they have found a "bug"

Post Reply