testing & duplicate games

[flok]

Hi,

Currently I'm only testing with a single threaded version of my program.
In a test it played 112642 games against other programs.
Using pgn-extract -D I checked how many unique games were played and found that only 31% of the games were unique, the other 69% were games with exact the same moves as others. So effectively it played only 35262 games.
My question now is: how do you guys handle this? Just live with it? Or add some randomness? And how?

[brtzsnr]

I use 2moves_v1.pgn opening book which was developed for fishtest (Stockfish's testing framework). This provides one with a very diverse set of balanced opening positions. However, Kai was suggesting in another thread that balanced positions need more games in general.

Search for it on the fishcooking group. Here is the 2nd version.
https://groups.google.com/forum/#!searc ... yDyrlcbi4J

[Laskos]

I use 2moves_v1.pgn opening book which was developed for fishtest (Stockfish's testing framework). This provides one with a very diverse set of balanced opening positions. However, Kai was suggesting in another thread that balanced positions need more games in general.

Search for it on the fishcooking group. Here is the 2nd version.
https://groups.google.com/forum/#!searc ... yDyrlcbi4J

I actually discovered to my surprise that 2moves_v1.epd (I think the same positions as the ends of PGN games) is pretty varied, I had to filter it for balanced and unbalanced. Some positions in it are evaluated as 90-100cp by SF (250ms). But the number of balanced positions is much larger.

[Dirt]

The openings.epd looks good to me. It's at Crafty. The down side is you don't get complete games since you are missing the start, but I never cared.

[Daniel Anulliero]

I use an openning book and I play opennings twice ( reversed color)
I test against 10 engines + the last best version of Isa
I play 60 games match , with 30 opennings and reversed colors
Total 660 games , not very much but I test at 1'+ 0.25'' AND at 5' + 0.25 '' witch take much CPU time

[Luis Babboni]

Could be this the reason in the TCEC they don´t allow programs to choice the openings by itselfs?

[Frank Quisinsky]

Hi,

indeed, biggest problems with all the books we have. The reason I create my own.

Easy ...
Two years I am using a 6 move database by best correspondence / gm games. After each test-run I added the new moves between 6-10 in the book. With the results the book have more variabilty, but it need time.

One advantage more ...
It will be a mix between computer chess and human chess. In my book are a lot of lines, never can be found in GM-databases ... found by the best engines we have.

Now I do the same and added the moves 10-12 from the games I have. Again, the book have more variabilty. Bad lines with "f" never played again and fast draw games with "e" ... never played again. I do that last week and like the results in Hakkapeliitta test-run.

Download for an example one of the last 2.800 test-runs I created. Now, give all 57 players the same name. Truncate with the tool by Norm Pollock undo move 10. And you can see ... around 6% double lines and with around ~ 450-470 different ECO codes are played (but all can be played, only the rarely lines are rarely in my book too), no bad lines in database.

6% is around 11% better as the best commercial book I know. Normely ... around 30% double lines in available opening books and not many of the 500 ECO codes played. In most of cases not more as 300.

Again, very important topic for a rating list. Equal lines, 500 ECO codes and a good variability without many double lines is the big secret and a lot of work ... working years on it, daily 1-2 hours on the own book.

Available for Shredder Classic GUI ...
In my opinion the event on my site I am working a long time.

I have 6% but I am sure 3-4% is possible ...
Maybe I have that result in 1-2 years.

Best
Frank