Python engine test scripts

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Terje
Posts: 347
Joined: Tue Nov 19, 2019 4:34 am
Location: https://github.com/TerjeKir/weiss
Full name: Terje Kirstihagen

Python engine test scripts

Post by Terje »

I've written a few python scripts to test perft, mate finding and eval symmetry in Weiss. They aren't too polished, and I'm open to suggestions for improvements to the code or functionality, or for other tests to add.

https://github.com/TerjeKir/EngineTests
sedicla
Posts: 178
Joined: Sat Jan 08, 2011 12:51 am
Location: USA
Full name: Alcides Schulz

Re: Python engine test scripts

Post by sedicla »

Thanks for making this available.
I have a couple of tests but all implemented in the engine. I always wanted to make external so I could compare results to other engines, to see how my engine was doing :)
Terje
Posts: 347
Joined: Tue Nov 19, 2019 4:34 am
Location: https://github.com/TerjeKir/weiss
Full name: Terje Kirstihagen

Re: Python engine test scripts

Post by Terje »

sedicla wrote: Fri Nov 06, 2020 6:27 pm Thanks for making this available.
I have a couple of tests but all implemented in the engine. I always wanted to make external so I could compare results to other engines, to see how my engine was doing :)
You're welcome :)

Weiss also had these internally (probably inherited some of them from VICE), but I decided scripts would be a better solution and removed them. Only got around to writing the scripts now, quite a bit later tho :P
Terje
Posts: 347
Joined: Tue Nov 19, 2019 4:34 am
Location: https://github.com/TerjeKir/weiss
Full name: Terje Kirstihagen

Re: Python engine test scripts

Post by Terje »

Made some improvements:
  • Engine class sets up more properly, sending uci and isready and consuming all replies on startup.
  • Tests now run on each engine placed in the engines/ folder so no need to edit scripts to test different engines.
  • Mate test now uses all the .epds from depth 1 to 8.
  • Fixed missing ./ in front of binary name on linux.
Also tested on various other engines. Eval symmetry test revealed a couple of minor bugs in FabChess, Mate test revealed a bug in SF12 (reports false shorter mates before finding the real ones, seems to have been fixed in SF Dev).