The influence of books on test results.

[lkaufman]

If my testing method possibly negates something you are trying to do with Houdini, then you are free to ban me from receiving a copy of the next version or from publishing any results. I would not complain, nor would I have hard feelings about you.

Although neither Robert nor Komodo has any intention of taking you up on this, I think it is a bad policy to allow an engine to ban one particular tester of a group. In theory we could ban any tester who uses a book we deem unfavorable to our program. An engine could ask the CCRL not to rate it at all, but if it is to be rated, all the testers should be free to test it.

[Adam Hair]

If my testing method possibly negates something you are trying to do with Houdini, then you are free to ban me from receiving a copy of the next version or from publishing any results. I would not complain, nor would I have hard feelings about you.

Although neither Robert nor Komodo has any intention of taking you up on this, I think it is a bad policy to allow an engine to ban one particular tester of a group. In theory we could ban any tester who uses a book we deem unfavorable to our program. An engine could ask the CCRL not to rate it at all, but if it is to be rated, all the testers should be free to test it.

It would be bad policy.

After some reflection, I believe that I should not have imposed that situation upon Robert. I should have written to him in private. By doing so in public, he had no choice but to back away from any criticism.

[geots]

If my testing method possibly negates something you are trying to do with Houdini, then you are free to ban me from receiving a copy of the next version or from publishing any results. I would not complain, nor would I have hard feelings about you.

Although neither Robert nor Komodo has any intention of taking you up on this, I think it is a bad policy to allow an engine to ban one particular tester of a group. In theory we could ban any tester who uses a book we deem unfavorable to our program. An engine could ask the CCRL not to rate it at all, but if it is to be rated, all the testers should be free to test it.

It would be bad policy.

After some reflection, I believe that I should not have imposed that situation upon Robert. I should have written to him in private. By doing so in public, he had no choice but to back away from any criticism.

Adam, I have listened to this horseshit about as long as I can. You are not at fault in this. Hard for me, because I like all involved in this discussion- however, it is what it is. If a programmer has anything to discuss with any rating group or individual concerning ANYTHING ABOUT ANYTHING having to do with the testing of his engine, IT SHOULD BE IN PRIVATE. It should NEVER be discussed on an open forum. Since you are not a programmer whose engine is being tested by CCRL- you are NOT the one who is out of order.

Just a bit of friendly advice- abandon this discussion. It was a train wreck from the get-go.


Best,

george

[velmarin]

The theme was very nice,
And we could draw conclusions about how to make future books.

That variety, how many plys, ect.ect.

There are more engines than those named, and users also make their own tests.

The issue led to a cock fight and forgot chess.

These numbers are impossible to follow.

We lost the thread.
Hurt.

[Mike S.]

Book depths are an issue, but there is more. - Maybe variety has an effect, too? If you'd like to try a test book which is based on quality games, but provides a bigger variety - IOW the fashion variations have smaller probabilities than usual - I can provide my "Balanced" books for free (except for commercial distribution).

http://members.aon.at/computerschach/li ... #downloads

See the readme file for details. Intended are Fritz book options normal, min. games = 3.

Depth up to 16 full moves (14 in Arena). As you most probably know, you can limit a book to any depth you like in Fritz or Arena, to make it a short book if desired.

(I have become sceptical about very short books because the early opening is probably not a typical area where chess players use engines.)

The trick was that I compiled the database these books are based on, step by step in a way that I controlled the share each (group of) openings had regarding the number of games, each. The result is not at a all a mostly "miscellaneous" opening play, it's still rather classical but just with the typical fashion openings being less frequent, and other "normal" openings appearing a bit more often.

[Graham Banks]

Book depths are an issue, but there is more. - Maybe variety has an effect, too? If you'd like to try a test book which is based on quality games, but provides a bigger variety - IOW the fashion variations have smaller probabilities than usual - I can provide my "Balanced" books for free (except for commercial distribution).

http://members.aon.at/computerschach/li ... #downloads

See the readme file for details. Intended are Fritz book options normal, min. games = 3.

Depth up to 16 full moves (14 in Arena). As you most probably know, you can limit a book to any depth you like in Fritz or Arena, to make it a short book if desired.

(I have become sceptical about very short books because the early opening is probably not a typical area where chess players use engines.)

The trick was that I compiled the database these books are based on, step by step in a way that I controlled the share each (group of) openings had regarding the number of games, each. The result is not at a all a mostly "miscellaneous" opening play, it's still rather classical but just with the typical fashion openings being less frequent, and other "normal" openings appearing a bit more often.

I recommend your books and enjoy using them.

Balanced-16.ctg as you describe.
Xmas2640-12.ctg - good for tournaments, but not recommended for gauntlets due to its small size.
History107.ctg - openings played by champions and near champions from the past.

Must be due for a new one Mike!

[Mike S.]

Thank you Graham, your appreciation was always very encouraging for me. Unfortunately, I lost the "bookbase" database for the Balanced books somehow, due to several difficult circumstances. Nevertheless, I am of course capable to recreate if from scratch, even based on more up-to-date data. My Fritz 13 database has ~2.6 million games up to 2011.

But I am getting old and tired, and I was always kind of lazy. Maybe the Balanced-16 remains the best what I could contribute, to computer chess testing...

[marijan]

Sure there is influence, if book has bad lines... That s why I m using reversible starting positions in my tests... Think the score is more accurate....

[velmarin]

Sometimes the stiffness to go to the play 8,10,12 ect,
can leave the ideas of a certain opening unfinished

A book looking for that point would be well balanced.
Independent the plys.

Anyway divided between evaluation and OPEN,END of most engines may leave the opening moves a bit neglected.

In engine BOUQUET I am experiencing dividing the evaluation into seven phases, the first is just opening, I hope that gives you another personality should play without a book or a short book.

[Graham Banks]

Btw,since several years (more than 10 years) i spent many efforts over my Perfect book series,
and unfortunately still there are some lines in my latest book,where some engines performance suffer...
My main goal is that, each opening line/positition should be with winning percentage around 55% Whites and 45 % Blacks

For more details:
http://www.sedatcanbaz.com/chess/downlo ... 011-books/

Best Wishes,
Sedat

Hi Sedat,

here's a line to remove from Perfect 2012a:

[Event "Round Robin"]
[Site "ChessGUI"]
[Date "2012.08.06"]
[Round "8.5"]
[White "Chiron 1.1a 64-bit"]
[Black "Strelka 5.5 64-bit"]
[Result "0-1"]
[ECO "A15"]
[PlyCount "24"]
[EventDate "2012.??.??"]
[TimeControl "40/1500:40/1500:40/1500"]

{Unknown Hardware} 1. c4 {[%eval 0,1] [%emt 0:00:00]} Nf6 {
[%eval 0,1] [%emt 0:00:00]} 2. d4 {[%eval 0,1] [%emt 0:00:00]} e6 {
[%eval 0,1] [%emt 0:00:00]} 3. Nc3 {[%eval 0,1] [%emt 0:00:00]} d5 {
[%eval 0,1] [%emt 0:00:00]} 4. cxd5 {[%eval 0,1] [%emt 0:00:00]} exd5 {
[%eval 0,1] [%emt 0:00:00]} 5. Bg5 {[%eval 0,1] [%emt 0:00:00]} Bb4 {
[%eval 0,1] [%emt 0:00:00]} 6. Nf3 {[%eval 0,1] [%emt 0:00:00]} Nbd7 {
[%eval 0,1] [%emt 0:00:00]} 7. e3 {[%eval 0,1] [%emt 0:00:00]} c5 {
[%eval 0,1] [%emt 0:00:00]} 8. b3 {[%eval 0,1] [%emt 0:00:00]} Bxc3+ {
[%eval 955,8] [%emt 0:00:00]} 9. Nd2 {(Nd2) [%eval -526,23] [%emt 0:00:56]} h6
{(Qa5) [%eval 1232,21] [%emt 0:02:02]} 10. Bxf6 {
(Bxf6) [%eval -515,24] [%emt 0:00:56]} Qxf6 {
(Nxf6) [%eval 1230,19] [%emt 0:00:00]} 11. Rc1 {
(Rc1) [%eval -545,21] [%emt 0:00:48]} cxd4 {
(cxd4) [%eval 1247,20] [%emt 0:01:25]} 12. exd4 {
(exd4) [%eval -545,23] [%emt 0:00:48]} Qxd4 {
(Bxd4) [%eval 1245,19] [%emt 0:00:00]} 0-1