Test suites (e.g. BT2450) and evaluation correctness

[GrahamA]

I've only recently made the move from 0x88 to bitboards. I used perft to get correct movegen / make / unmake "bug free" and then moved on to BT2450 to rate my search + eval functionality.

I've found that for the first test in BT2450,

FEN "rq2r1k1/5pp1/p7/4bNP1/1p2P2P/5Q2/PP4K1/5R1R w - - 0 1"

for which Nxg7 is the correct response. With the same evaluation function my 0x88 search found this quiet early, and then moved to a different line. With bitboards allowing deeper searches in the allotted time my search now finds the move early, moves off, and then back on before running out of time.

Two questions:
a) what would cause this behavior?
b) even knowing the correct 1st move how am I to know that my engine found the correct line? Is there a full line solution for this test suite rather than just the first "correct" move?

I wrote the 0x88 one based on the Mediocre blog starting in Jan this year. It was 30 years ago I wrote a (basic) chess engine in Pascal. The cpw, and forums like this are an amazing asset.

Thanks for any feedback on these questions,
Graham....

[zullil]

b) even knowing the correct 1st move how am I to know that my engine found the correct line? Is there a full line solution for this test suite rather than just the first "correct" move?

Thanks for any feedback on these questions,
Graham....

Your engine doesn't need to find the correct line. All it needs to do is find a best move for the position at hand.

That said, here is Stockfish's PV for the position, after a long search. If you really want to check for a best line, you'll need to search deeply after each move is played on the board.

Code: Select all

+6.41 1. Nxg7 Rf8 2. Nf5 Qe8 3. Nh6+ Kg7 4. h5 Rc8 5. Rf2 Qe6 6. Rh4 b3 7. axb3 Rb8 8. Rh3 Rbd8 9. Nf5+ Kg8 10. Qg4 Rd6 11. g6 fxg6 12. hxg6 Rxf5 13. exf5 Qd5+ 14. Qf3 Qxf3+ 15. Kxf3 Rb6 16. Ke4 Bg7 17. Rh7 Rb4+ 18. Kd5 Rb6 19. Kc5 Rf6 20. Rh4 Rf8 21. Ra4 Rf6 22. Re4 Kf8 23. Re6 a5 24. Kd5 Bh8 25. Ke4 Rxe6+ 26. fxe6+ Ke7 27. Rf7+ Kxe6 28. g7 Bxg7 29. Rxg7 (depth 46, 1:34:34)