My engine has just played a couple of games against itself, and since it is not even near Stockfish's rating, I thought I would use that to analyse the blunders/mistake my engine made. Now I have the analysis, but I don't know what to do with it.
How does one debug an engine's search/eval by analyzing the games it has played? Is there some consensus on a specific methodology to use?
I am not sure that you can. Poor play is not necessarily a bug. The engine can blunder pieces away simply because it did not search deep enough to see the tactics that caused it. And it could not have searched deep enough because it put more emphasis on searching other tree branches than the one that contained the fatal tactics. There is a nice discussion about a '5-mover' in the general section that illustrates that: it took Stockfish 11 minutes to see the mate-in-5; Fairy-Max found it in 13 sec. So Stockfish would have blundered in a blitz game, by overlooking the mate, as analyzing the game with Fairy-Max would later reveal. Doesn't mean Stockfish is buggy. Doesn't even mean Stockfish is weaker. It is in fact about 1500 Elo stronger.
But some of the blunders could be moves that you had expected the engine to see, and a blunder check with Stockfish at least helps you to find the blunders, so that you can judge them to see if they look suspect. If you have an inexplicable blunder, and it is reproducible, then you can start debugging your search (by examining the tree) to see what caused this move to be played, and the good alternative to be rejected.
