Position Causes Stockfish and Komodo To Crash

[Fulvio]

There are various levels for interfaces, it is not a particularly complicated concept, I am amazed that so many who have written in this thread cannot understand it.
High level, for example a car dashboard. It is not acceptable to press wrong commands and have the engine explode.
Low level, I open the hood, take apart some pieces, put my special fuel in it and the engine explodes. That's ok.
High level, unprivileged code should not be able to bring down the kernel.
Low level, buggy driver that runs with full privilege and crash the system? That's ok.
High level, it is not acceptable for a program intended for end-users to crash due to an incorrect FEN.
Low level, chess engine meant to be used by other programs that crash with invalid inputs? That's ok.
Stockfish choose to be low level and crash on invalid inputs? That's ok, there are billions of tools that can check if FENs are valid: just use those before sending the position to Stockfish.

[Ras]

Stockfish choose to be low level and crash on invalid inputs? That's ok

It's not OK for low level code to be shoddy. That's not how you get robust systems. Even inside a car, each controller has to deal with possibly invalid inputs from other controllers because you don't want e.g. a glitchy tyre sensor to bring down your brake system. It's also about fault containment.

With chess engines, it has nothing to do with such a botched up definition of low level. It's that some devs either don't care about basics of proper software engineering, or maybe don't even know about them.

[Fulvio]

Stockfish choose to be low level and crash on invalid inputs? That's ok

It's not OK for low level code to be shoddy. That's not how you get robust systems. Even inside a car, each controller has to deal with possibly invalid inputs from other controllers because you don't want e.g. a glitchy tyre sensor to bring down your brake system. It's also about fault containment.

With chess engines, it has nothing to do with such a botched up definition of low level. It's that some devs either don't care about basics of proper software engineering, or maybe don't even know about them.

If you input petrol to a diesel engine it will crash. Doesn't seem so strange to me.
Let me google the word you used, system: "A set of things working together as parts of a mechanism".
So, if you abandon your crusade about the only proper and right way, you shouldn't have too much trouble understanding that they designed it to be a component, a part. Just add another thing (a GUI, a command line tool, whatever) and then you get your perfectly secure and reliable system.

[syzygy]

A lot of software is designed to be invoked through wrapper programs which present a friendly interface to the user (e.g. a compiler toolchain, or the kernel of a computer algebra system). But no one would accept the underlying "worker" programs to crash when presented with invalid input.

Imagine the Linux kernel crashing after a system call with invalid arguments...

The Linux kernel has to treat all user space programs as enemies. A typical UCI engine treats the GUI as a friend.

I doubt that the Linux kernel exhaustively checks data received from kernel modules. Kernel modules are friends.

The C library and many other libraries treat the program invoking it as a friend and will happily crash or wipe out any data accessible to the program when given illegal input.

Where do the security boundaries lie? That is where you need to check.

[Michel]

A lot of software is designed to be invoked through wrapper programs which present a friendly interface to the user (e.g. a compiler toolchain, or the kernel of a computer algebra system). But no one would accept the underlying "worker" programs to crash when presented with invalid input.

Imagine the Linux kernel crashing after a system call with invalid arguments...

The Linux kernel has to treat all user space programs as enemies. A typical UCI engine treats the GUI as a friend.

I doubt that the Linux kernel exhaustively checks data received from kernel modules. Kernel modules are friends.

The C library and many other libraries treat the program invoking it as a friend and will happily crash or wipe out any data accessible to the program when given illegal input.

Where do the security boundaries lie? That is where you need to check.

The fact that C libraries are allowed to unceremoniously segfault is (a) for performance reasons (b) they cannot verify if a pointer they receive is valid or not.

However Python of Java libraries will typically handle illegal input nicely with an appropriate exception.

[Ras]

So, if you abandon your crusade about the only proper and right way,

There is a right way and a wrong way here - which is why input validation has been standard practice in software engineering for decades. Except apparently some chess engine programmers, people have gotten that message.

Just add another thing (a GUI, a command line tool, whatever) and then you get your perfectly secure and reliable system.

If you regard crashing stuff (see thread title) as "reliable", you have a very low expectation of reliability. And no, it's not the GUI's fault because the GUI can't know whether a specific engine can deal with positions that have e.g. more than nine queens.

[gaard]

So, if you abandon your crusade about the only proper and right way,

There is a right way and a wrong way here - which is why input validation has been standard practice in software engineering for decades. Except apparently some chess engine programmers, people have gotten that message.

Just add another thing (a GUI, a command line tool, whatever) and then you get your perfectly secure and reliable system.

If you regard crashing stuff (see thread title) as "reliable", you have a very low expectation of reliability. And no, it's not the GUI's fault because the GUI can't know whether a specific engine can deal with positions that have e.g. more than nine queens.

Maybe I am wrong, but I think if someone introduced a patch that caught bogus FENs and it showed it to have a negligible impact on performance, it would be implemented in master. However, I myself am too lazy to implement and test such a patch so it's hard to argue that anyone else should.

[Michel]

I would also like to comment on the issue of kernel modules which has been raised several times.

I consider kernel modules to be part of the kernel. They run in the kernel's address space so of course the kernel cannot defend itself against malicious kernel modules. So there is no point in trying.

But it is not so that this is universally regarded as a desirable situation. There have been many attempts in the past at creating OS'es that run device drivers in user space so that they can handle misbehaving device drivers gracefully. Such "microkernel OS'es" are generally less performant and that is the reason they have not caught on.

So this is to reinforce my point. When a software entity is allowed to crash on invalid input it is (a) because of performance reasons or (b) it has no way of verifying the input anyway.

In the case of the kernel's handling of modules both (a) and (b) apply.

[Michel]

So this is to reinforce my point. When a software entity is allowed to crash on invalid input it is (a) because of performance reasons or (b) it has no way of verifying the input anyway.

I realize that this is not a black or white issue (few issues are). So I would be happy to learn about other examples where people accept crashing on invalid input, not falling under (a) or (b).

[JohnWoe]

When humans write software there always will be bugs. Imperfection must be accepted.

"position fen ..." is not input for humans. It's meant for other programs.

That being said: Stockfish shouldn't crash on invalid input. (I'm too lazy to write a "patch" anyway.).

Mayhem 2.5 (hopefully) doesn't crash. I haven't tested it super well. Maybe it does.