UCI fuzzing / stress testing

[benvining]

I'm aware of tournament manager tools like fastchess/cutechess, but I'm wondering if there are any tools that are designed to stress test an engine's conformance to UCI. Perhaps by doing things like sending commands with arbitrary amounts of whitespace between tokens, sending unknown commands to make sure they're just ignored, etc.

[ngocnhungplaza]

Good question. I’ve done some informal fuzzing myself using custom scripts that push malformed or unexpected UCI commands — things like extra whitespace, rapid position/go sequences, or even unknown keywords. Some engines handle this gracefully, others don’t. I don’t know of a dedicated UCI fuzzing tool, but writing a basic Python script around a socket interface can get you pretty far if you're looking to test edge cases.

[benvining]

I see. Coming from a background in audio plugin development, we have https://github.com/Tracktion/pluginval, so I was hoping there would be a similar tool for chess engines. Perhaps I'll start working on one, maybe others would be interested in contributing.

[flok]

I'm aware of tournament manager tools like fastchess/cutechess, but I'm wondering if there are any tools that are designed to stress test an engine's conformance to UCI. Perhaps by doing things like sending commands with arbitrary amounts of whitespace between tokens, sending unknown commands to make sure they're just ignored, etc.

https://github.com/Disservin/fastchess does that