Position Causes Stockfish and Komodo To Crash

[syzygy]

What do you think about this kind of position?

KQQQQQQQ/QQQQQQQQ/QQQQQQQQ/QQQqqqqq/QQQqqqqq/QQQqqqqq/QQQqqqqq/QQQqqqqk b - - 1 1

GUIs might (and do) accept this as a "puzzle" position although it is not legal in terms of reachable from the start position. Which is totally okay in my opinion.
Stockfish crashes on it.

The recent removal of piece lists might have fixed that (it seems to work for me).

[Ras]

a UCI engine (which could not even report an error to the GUI)

Of course it can. My engine does this.

[AndrewGrant]

Someone recently wasted their time writing up 20 paragraphs showing how they could "exploit" Stockfish into crashing....

If you send a chess engine garbage, you should expect garbage. If you don't know it is garbage, then maybe a GUI should be doing it for you.

The above has been affirmed many times in Stockfish PRs, where users come saying they have found a "bug"

Is that how you would write code for a customer? Unexpected input causes a program crash?
If a programmer is too lazy even to think hard, they can still use try catch (yes, I know, a 1% performance penalty).
It boggles my mind how chess programmers will spend 10,000 hours writing their programs but spend zero minutes checking the input for correctness.

You are lazy for not giving me good inputs? Its not lazy, its simply not important.
Some engines don't even parse the entire FEN. Now that is lazy.

[syzygy]

a UCI engine (which could not even report an error to the GUI)

Of course it can. My engine does this.

Your engine reports it to the user, not to the GUI. The GUI will likely get terribly confused if it does not crash.

[syzygy]

I think that crashing on invalid input is never OK.

My engine would reject this position with "info string error (illegal position: too many pieces)". A subsequent "go" command would be answered with "info string error (illegal position)" and "bestmove 0000".

Does your engine also refuse to crash or freeze if you change the TT size or the number of search threads halfway through a search?

The whole point of the UCI protocol is that it allows an engine not to bother with a whole lot of stuff that is better left to the GUI.

[syzygy]

Someone recently wasted their time writing up 20 paragraphs showing how they could "exploit" Stockfish into crashing....

If you send a chess engine garbage, you should expect garbage. If you don't know it is garbage, then maybe a GUI should be doing it for you.

The above has been affirmed many times in Stockfish PRs, where users come saying they have found a "bug"

Is that how you would write code for a customer? Unexpected input causes a program crash?
If a programmer is too lazy even to think hard, they can still use try catch (yes, I know, a 1% performance penalty).
It boggles my mind how chess programmers will spend 10,000 hours writing their programs but spend zero minutes checking the input for correctness.

Not doing more than what the spec requires is the right form of laziness. A customer should not need to pay for fancy and unnecessary stuff going beyond the spec.

The UCI spec requires the GUI to give the engine sane inputs. Therefore the engine should expect sane inputs.

(Admittedly the UCI spec is very mistaken where it requires an engine to ignore unrecognised tokens but to do its best to process the rest of the input line. I'm not convinced that many engines will "correctly" process "gobbledygook go infinite".)

[syzygy]

https://pubs.opengroup.org/onlinepubs/0 ... /free.html

The free() function shall cause the space pointed to by ptr to be deallocated; that is, made available for further allocation. If ptr is a null pointer, no action shall occur. Otherwise, if the argument does not match a pointer earlier returned by the calloc(), malloc(), posix_memalign(), realloc(), or strdup() function, or if the space has been deallocated by a call to free() or realloc(), the behavior is undefined.

Any use of a pointer that refers to freed space results in undefined behavior.

How many here take the view that a C library that claims to be POSIX-compliant but that crashes when free() is called twice on the same pointer is "pathetic"/"never OK"/etc?

[mar]

Does your engine also refuse to crash or freeze if you change the TT size or the number of search threads halfway through a search?

is crashing or freezing the right thing to do though? my engine simply aborts the search in this case, I really don't like crashes in general.

as for double free - a dangling pointer is the responsibility of the programmer, not the C library.
I don't see how this analogy is relevant here.
I certainly wouldn't like my text editor to crash on bad regex pattern, which is a better example I think.

[syzygy]

Does your engine also refuse to crash or freeze if you change the TT size or the number of search threads halfway through a search?

is crashing or freezing the right thing to do though? my engine simply aborts the search in this case, I really don't like crashes in general.

Des your engine catch all such cases? Is your GUI thread aware of all the possible invalid input sequences that a malicious GUI could send it? It seems a lot of work to get 100% right.

as for double free - a dangling pointer is the responsibility of the programmer, not the C library.

Exactly.

I don't see how this analogy is relevant here.

It is the GUI author's responsibility to give the UCI engine valid input.

I certainly wouldn't like my text editor to crash on bad regex pattern, which is a better example I think.

I certainly wouldn't like my chess GUI to crash on an illegal position set up by the user.

[mar]

Des your engine catch all such cases? Is your GUI thread aware of all the possible invalid input sequences that a malicious GUI could send it? It seems a lot of work to get 100% right.

most likely not, it's hard to cover 100% of the cases and all code paths. but at least I try to handle the obvious ones.

It is the GUI author's responsibility to give the UCI engine valid input.

I agree, but some people run engines from the command line. I certainly use command line when debugging my engine.