Opening Book Question

[Milton]

I have a question about the evaluation of the strength of chess program opening books. The procedure for comparing the strength of chess engines is pretty straightforward - run matches with them under controlled conditions and compare the results -- as done by SSDF, etc.

Is there some testing methodology or procedure for objectively evaluating the strength of opening books so that one could make the claim that program x has a stronger (or better) opening book than program y? Thanks.

Milton

[Ron Murawski]

I have a question about the evaluation of the strength of chess program opening books. The procedure for comparing the strength of chess engines is pretty straightforward - run matches with them under controlled conditions and compare the results -- as done by SSDF, etc.

Is there some testing methodology or procedure for objectively evaluating the strength of opening books so that one could make the claim that program x has a stronger (or better) opening book than program y? Thanks.

Milton

A quick way to determine opening book strength is to look for short games. If an engine loses many short games against almost-equal opponents it's an indication that its book has weaknesses. (But it could indicate a weakness in engine evaluation instead.)

The long way is first testing using an engine's own book and then using a so-called "standard book". You will need to play many, many games to get results you can trust. But even that may not work because many of the books, including "standard books", lead amateur engines into situations they do not understand (e.g., opening traps) and cause needless losses.

Ron

[Milton]

Thanks!
Milton