Testing openning-phase eval

[Kempelen]

Hello
I would like to improve opening evaluation in my engine, but I doubt what would be better, testing against engines with use opening book, or testing only with engine without them?. What do you think would be the prefered method and why?
Fermin

[rbarreira]

IMHO, perhaps starting out by testing against engines that are using a book, and only switching to the other method if this results in too many losses.

My reasoning is that most engines probably have not tuned their eval to play well without a book, so it's not very relevant to remove their book and use them as test opponents that way. By testing against engines with a book you will know that the opponent is playing the opening well.

[Steve Maughan]

I think Chess960 may offer a great way of testing opening evaluation. Just play a bazillion 5 second games and you should be able to spot weaknesses and improvements in the engine. This is the approach I'll take if I ever get round to doing more work on Monarch.

Seve

[Evert]

I think it matters little. The reason is that if your engine plays without a book it will probably play a move that brings the other engine out of book as well.

[Ferdy]

Hello
I would like to improve opening evaluation in my engine, but I doubt what would be better, testing against engines with use opening book, or testing only with engine without them?. What do you think would be the prefered method and why?
Fermin

To improve the opening evaluation is to know a good estimate value of those openings. Example e2e4 c7c5, if your engine evaluates to +100 then there's a problem with your engine. Check with strong engines if you are close to their evaluations.
To do this automatically, collect as many opening positions then run with strong engines at say 30sec per position and store its value. Then run your engine and compare with the results, if the score difference is way too high or too low say a margin of +30/-30 then store and evaluate those positions again, possibly increasing the thinking time or check with your code at this time.