Game analysis for search debugging

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
niel5946
Posts: 30
Joined: Thu Nov 26, 2020 9:06 am
Full name: Niels Abildskov

Game analysis for search debugging

Post by niel5946 » Thu Feb 18, 2021 11:56 am

Hi.

My engine has just played a couple of games against itself, and since it is not even near Stockfish's rating, I thought I would use that to analyse the blunders/mistake my engine made. Now I have the analysis, but I don't know what to do with it.

How does one debug an engine's search/eval by analyzing the games it has played? Is there some consensus on a specific methodology to use?

Thanks in advance! :D

User avatar
hgm
Posts: 25861
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Game analysis for search debugging

Post by hgm » Thu Feb 18, 2021 1:26 pm

I am not sure that you can. Poor play is not necessarily a bug. The engine can blunder pieces away simply because it did not search deep enough to see the tactics that caused it. And it could not have searched deep enough because it put more emphasis on searching other tree branches than the one that contained the fatal tactics. There is a nice discussion about a '5-mover' in the general section that illustrates that: it took Stockfish 11 minutes to see the mate-in-5; Fairy-Max found it in 13 sec. So Stockfish would have blundered in a blitz game, by overlooking the mate, as analyzing the game with Fairy-Max would later reveal. Doesn't mean Stockfish is buggy. Doesn't even mean Stockfish is weaker. It is in fact about 1500 Elo stronger.

But some of the blunders could be moves that you had expected the engine to see, and a blunder check with Stockfish at least helps you to find the blunders, so that you can judge them to see if they look suspect. If you have an inexplicable blunder, and it is reproducible, then you can start debugging your search (by examining the tree) to see what caused this move to be played, and the good alternative to be rejected.

Post Reply