Book testing

[Andres Valverde]

Hi all,

What is the best way in your opinion to test books for a given engine?

I'm currently running blitz selftests, but I'n not sure about the reliability of the results.

PS : Sorry if I repeat the topic but my couldnt find it in my searchs

[Bill Rogers]

I would suggest playing a series of games against different opponents using no books at say one minute per move. After that play the same opponents using your books and see if it made a difference. As a matter of fact I would pick three or four opponents that were maybe just a little stronger than me. That way you get hard fought games and although you may lose some you can see the real strengths and then if using you books you beat the same men then you know your books are good.
Bill

[Ovyron]

Actually book-less matches don't make much sense, because you need a lot of games to be certain of something, and without book you could get the same game over and over, exploring only a bit of the chess space. If an engine without book reaches positions it doesn't know how to play, any book is going to get overrated.

I suggest a minimal book match, like a 2 moves book like suggested by Larry Kaufman.

[Andres Valverde]

Thanks for your suggestions.

The idea is not to test without book, but select the best book for a given engine at a given TC.

My conclusions i got after my first post is that *selftests" are not good. At least with the set of books / TCs I have tried so far. I will post the data when my tests are finished

[Tony Thomas]

Thanks for your suggestions.

The idea is not to test without book, but select the best book for a given engine at a given TC.

My conclusions i got after my first post is that *selftests" are not good. At least with the set of books / TCs I have tried so far. I will post the data when my tests are finished

By selftests, are you talking about the same version of engine playing with different books? In my opinion they are more reliable than an engine playing vs 6 different opponents. I have only seen one book test with large number of games in recent times.. Sedat tested all the books with same version of Rybka..

[Andres Valverde]

Thanks for your suggestions.

The idea is not to test without book, but select the best book for a given engine at a given TC.

My conclusions i got after my first post is that *selftests" are not good. At least with the set of books / TCs I have tried so far. I will post the data when my tests are finished

By selftests, are you talking about the same version of engine playing with different books? In my opinion they are more reliable than an engine playing vs 6 different opponents. I have only seen one book test with large number of games in recent times.. Sedat tested all the books with same version of Rybka..

Yes I meant that. But my results so far are contradictory. Let's wait for more games.. Good to know Sedat does it