Polyglot extension tool

[Rebel]

Poly version 1.0

The polyglot book format is the most widespread format in use by chess engines. The format contains 4 unused bytes. Have a look at the nice things you can do using those bytes.

http://rebel13.nl/download/books.html

[hgm]

Beware that Polyglot uses these bytes for book learning, and that they have quite different meaning in that case. So if you are using a book of the described format, be sure book learning stays switched off, or the extra info will be corrupted.

In Polyglot books produced by WinBoard in Monte-Carlo book mode these bytes are used to store WDL counts (which are not deducible from the book weights alone), so that you can load an existing book and add continue the MC play.

[Rebel]

I thought about the same but did not like the maximum of 1023 (10 bits) of the WLD counters. Or were you able to increase the maximum?

[hgm]

It appears I actually cheat here, by only keeping W+D and L+D, each in 16 bit. I originally implemented the Monte-Carlo book building in WinBoard for making a mini-Shogi book. And draws are not possible in mini-Shogi, so I added these as an afterthought.

Note, however, that the weight field in a Polyglot book normally contains 2*W+D. So the three fields together make it possible to construct the full WDL info: W = weight - learnPoints; D = learnPoints - W; L = learnCount - D;

[Ozymandias]

Poly version 1.0

The polyglot book format is the most widespread format in use by chess engines. The format contains 4 unused bytes. Have a look at the nice things you can do using those bytes.

http://rebel13.nl/download/books.html

Are those bytes 16bit?
What are the value boundaries for Score and Depth?
What's the role of Score and Depth, in the move decision?
How does the learn byte keep track of losses? I'm guessing you use a different method, to the way weights are computed (see above).
What use is the learn value if several moves exceed 125?

[hgm]

Bytes are 8 bit, so 4 bytes is 32 bit. In the Polyglot book code they are used to store two 16-bit counters, learnCount and learnPoints. (Presumably the latter is actually half-points). Weight and move are also 16-bit quantities; together with 8 bytes for the position key this makes a 16-byte entry. Each move has its own entry, even if it is from the same position.

[Ozymandias]

Bytes are 8 bit, so 4 bytes is 32 bit. In the Polyglot book code they are used to store two 16-bit counters, learnCount and learnPoints. (Presumably the latter is actually half-points). Weight and move are also 16-bit quantities; together with 8 bytes for the position key this makes a 16-byte entry. Each move has its own entry, even if it is from the same position.

Thanks, where do the "unused" bytes come from? From learnCount or learnPoints?

[hgm]

Yes. So 'unused' is a relative concept.

[Jonathan003]

Hi I use Lucas Chess to build opening repertoires for over the board games.

I was thinking if I use a bin book as base for building a repertoire. It would be interesting to change the weight of the book moves according engine analysis.

So If I import only best move or best moves from a bin book, the moves will probably be good moves.

I would do it this way: change the results of the games to drawn games before building the bin books. (I use games from Grandmasters, engine games are to artificial for a repertoire for over the board games I think). Than in Lucas Chess choose to import all best moves for one collar from these bins. (So only the moves that where most played are imported, regardless of the results of the games)

Then export to pgn, Build new bin books from these pgn's and analyze these bin book with stockfish 10.

I the weight of the books where updated according the evaluation of the chess engine. I would maken new opening lines in Lucas Chess and choose to only import the best moves from the bin book.

I was searching on the web for a tool to just do that.

I came across poly1.0

http://rebel13.nl/download/books.html

I tried it but I don't see any changes in the weight of the moves before and after if I open the bin books in SCID

Do I do something wrong or isn't that the purpose of poly1.0
If not I would like it if someone could write a program that can do just that.

[Rebel]

Answered that on Facebook

Hi there,

The util doesn't change the weight, or anything, it just adds extra information to the book. At the end of the page it states programmers who support the Polyglot format need to update their code to make use of the new features.

But your comment about weights made me think. I will make 2 extra functions that will set the weight on 100% of the best Stockfish move or the highest learn value so that people are no longer dependent on programmers.

Just keep an eye on the website, an update will take not so long, one week max.