creating an opening book

[mhull]

How is that different from purchasing the assistance of a grandmaster? Is that also unfair?

It is fine if the GM is using SF 6 for example or it uses its knowledge and experience alone in creating the opening book.

This is not logical. The GM is better than any program (commercial or free) at opening theory.

That was before, but now there are equally non GM that are also good in book preparations this is because of the engine help and some software/databases that are available - Aquarium, chessbase and others that are not commercial.

You miss the point that a GM is still a commercial asset, just like a commercial program.

And what if a commercial programmer donates his program to help make a book for a free program? Ok? Not ok?

What if the commercial program is 5 years old? OK or not OK?

Your reasoning is not logical.

[Ferdy]

How is that different from purchasing the assistance of a grandmaster? Is that also unfair?

It is fine if the GM is using SF 6 for example or it uses its knowledge and experience alone in creating the opening book.

This is not logical. The GM is better than any program (commercial or free) at opening theory.

That was before, but now there are equally non GM that are also good in book preparations this is because of the engine help and some software/databases that are available - Aquarium, chessbase and others that are not commercial.

You miss the point that a GM is still a commercial asset, just like a commercial program.

The point is that the ability of the author of the engine will be tested in tournaments. During book creation he is allowed to use his engine to analyze the opening lines or positions or he can use the free strong program SF to improve the book. SF is free so we expect the other participants to use it unless the engine of the participant is stronger than SF which in that case he may not use SF.
He may hire a GM to help create a book, it is fine provided that GM does not use an engine that is stronger than SF assuming that SF is the strongest free engine on that time.

And what if a commercial programmer donates his program to help make a book for a free program? Ok? Not ok?

If his program is weaker than the free SF then that is fine. If the program he is donating is stronger than
the available free engine (say SF), it is still fine if he donates it to all participants. That's the idea of even level competition.

What if the commercial program is 5 years old? OK or not OK?

It depends on the free strongest engine available. If the commercial program is weaker then all is fine. It is not about the age, but strength of the free engine.

[flok]

Status update:

- book is now for 7,5% filled (percentage of all moves of 6 ply deep)
- book fill is paused because of the heat in the Netherlands (22:00 and here inhouse it is around 33 degrees celsius)
- a look-up of any position (via a FEN string) takes +/- 9ms
- the book is stored in a mysql database running in a virtual machine on a core i7 with an SSD
- the book is accessed from a raspberry pi model 2 over an 100Mb lan
- earlier this evening a look-up took 70ms so did some nice improvements
- java sucks because SELECT ... FROM ... WHERE column IN (...) cannot be executed with a PreparedStatement if the number of elements for "IN (...)" is variable

[Ferdy]

Status update:
- book is now for 7,5% filled (percentage of all moves of 6 ply deep)

At 7.5% filled how many positions is that?
So you are also storing positions that are bad, and will remove those later after the analysis?

[flok]

Status update:
- book is now for 7,5% filled (percentage of all moves of 6 ply deep)

At 7.5% filled how many positions is that?
So you are also storing positions that are bad, and will remove those later after the analysis?

Hmm don't know where I got that 7,5% from but it is actually 6.8587%.
Total stored nodes: 1350311, # done: 92614.

[flok]

Note that all 1.3M positions are in the database but for 94% of them eval is still null. So I don't expect the query time to go up a lot.

[Ferdy]

Note that all 1.3M positions are in the database but for 94% of them eval is still null. So I don't expect the query time to go up a lot.

I am experimenting using sqlite3, saving unique fen + move, (as first pass) by browsing thru the games from Frank's tournament pgn file of 51500 games (also interesting is to add games from TCEC). I am only saving up to 13 moves. After this I will update the eval, depth, W, L, D stats still based on that database on a second pass. The format I am initially going to use is.

Code: Select all

fen, move, eval, depth, W , L , D

The eval and depth are extracted from the comment (if there is) in the game from the engine playing in actual game. This eval will be subjected to longer analysis time later when attempting to improve the book.

I still don't have a concrete idea on how to select moves when there are more than 1 move in a given position. Perhaps some formula based on eval, depth, and results stats.