Poly version 1.0
The polyglot book format is the most widespread format in use by chess engines. The format contains 4 unused bytes. Have a look at the nice things you can do using those bytes.
http://rebel13.nl/download/books.html
Polyglot extension tool
Moderators: hgm, Rebel, chrisw
-
- Posts: 6997
- Joined: Thu Aug 18, 2011 12:04 pm
- Full name: Ed Schröder
Polyglot extension tool
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 27811
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Polyglot extension tool
Beware that Polyglot uses these bytes for book learning, and that they have quite different meaning in that case. So if you are using a book of the described format, be sure book learning stays switched off, or the extra info will be corrupted.
In Polyglot books produced by WinBoard in Monte-Carlo book mode these bytes are used to store WDL counts (which are not deducible from the book weights alone), so that you can load an existing book and add continue the MC play.
In Polyglot books produced by WinBoard in Monte-Carlo book mode these bytes are used to store WDL counts (which are not deducible from the book weights alone), so that you can load an existing book and add continue the MC play.
-
- Posts: 6997
- Joined: Thu Aug 18, 2011 12:04 pm
- Full name: Ed Schröder
Re: Polyglot extension tool
I thought about the same but did not like the maximum of 1023 (10 bits) of the WLD counters. Or were you able to increase the maximum?
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 27811
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Polyglot extension tool
It appears I actually cheat here, by only keeping W+D and L+D, each in 16 bit. I originally implemented the Monte-Carlo book building in WinBoard for making a mini-Shogi book. And draws are not possible in mini-Shogi, so I added these as an afterthought.
Note, however, that the weight field in a Polyglot book normally contains 2*W+D. So the three fields together make it possible to construct the full WDL info: W = weight - learnPoints; D = learnPoints - W; L = learnCount - D;
Note, however, that the weight field in a Polyglot book normally contains 2*W+D. So the three fields together make it possible to construct the full WDL info: W = weight - learnPoints; D = learnPoints - W; L = learnCount - D;
-
- Posts: 1535
- Joined: Sun Oct 25, 2009 2:30 am
Re: Polyglot extension tool
Are those bytes 16bit?Rebel wrote: ↑Sun Jan 06, 2019 4:18 pm Poly version 1.0
The polyglot book format is the most widespread format in use by chess engines. The format contains 4 unused bytes. Have a look at the nice things you can do using those bytes.
http://rebel13.nl/download/books.html
What are the value boundaries for Score and Depth?
What's the role of Score and Depth, in the move decision?
How does the learn byte keep track of losses? I'm guessing you use a different method, to the way weights are computed (see above).
What use is the learn value if several moves exceed 125?
-
- Posts: 27811
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Polyglot extension tool
Bytes are 8 bit, so 4 bytes is 32 bit. In the Polyglot book code they are used to store two 16-bit counters, learnCount and learnPoints. (Presumably the latter is actually half-points). Weight and move are also 16-bit quantities; together with 8 bytes for the position key this makes a 16-byte entry. Each move has its own entry, even if it is from the same position.
-
- Posts: 1535
- Joined: Sun Oct 25, 2009 2:30 am
Re: Polyglot extension tool
Thanks, where do the "unused" bytes come from? From learnCount or learnPoints?hgm wrote: ↑Mon Jan 07, 2019 1:18 pm Bytes are 8 bit, so 4 bytes is 32 bit. In the Polyglot book code they are used to store two 16-bit counters, learnCount and learnPoints. (Presumably the latter is actually half-points). Weight and move are also 16-bit quantities; together with 8 bytes for the position key this makes a 16-byte entry. Each move has its own entry, even if it is from the same position.
-
- Posts: 27811
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Polyglot extension tool
Yes. So 'unused' is a relative concept.
-
- Posts: 239
- Joined: Fri Jul 06, 2018 4:23 pm
- Full name: Jonathan Cremers
Re: Polyglot extension tool
Hi I use Lucas Chess to build opening repertoires for over the board games.
I was thinking if I use a bin book as base for building a repertoire. It would be interesting to change the weight of the book moves according engine analysis.
So If I import only best move or best moves from a bin book, the moves will probably be good moves.
I would do it this way: change the results of the games to drawn games before building the bin books. (I use games from Grandmasters, engine games are to artificial for a repertoire for over the board games I think). Than in Lucas Chess choose to import all best moves for one collar from these bins. (So only the moves that where most played are imported, regardless of the results of the games)
Then export to pgn, Build new bin books from these pgn's and analyze these bin book with stockfish 10.
I the weight of the books where updated according the evaluation of the chess engine. I would maken new opening lines in Lucas Chess and choose to only import the best moves from the bin book.
I was searching on the web for a tool to just do that.
I came across poly1.0
http://rebel13.nl/download/books.html
I tried it but I don't see any changes in the weight of the moves before and after if I open the bin books in SCID
Do I do something wrong or isn't that the purpose of poly1.0
If not I would like it if someone could write a program that can do just that.
I was thinking if I use a bin book as base for building a repertoire. It would be interesting to change the weight of the book moves according engine analysis.
So If I import only best move or best moves from a bin book, the moves will probably be good moves.
I would do it this way: change the results of the games to drawn games before building the bin books. (I use games from Grandmasters, engine games are to artificial for a repertoire for over the board games I think). Than in Lucas Chess choose to import all best moves for one collar from these bins. (So only the moves that where most played are imported, regardless of the results of the games)
Then export to pgn, Build new bin books from these pgn's and analyze these bin book with stockfish 10.
I the weight of the books where updated according the evaluation of the chess engine. I would maken new opening lines in Lucas Chess and choose to only import the best moves from the bin book.
I was searching on the web for a tool to just do that.
I came across poly1.0
http://rebel13.nl/download/books.html
I tried it but I don't see any changes in the weight of the moves before and after if I open the bin books in SCID
Do I do something wrong or isn't that the purpose of poly1.0
If not I would like it if someone could write a program that can do just that.
-
- Posts: 6997
- Joined: Thu Aug 18, 2011 12:04 pm
- Full name: Ed Schröder
Re: Polyglot extension tool
Answered that on Facebook
Hi there,
The util doesn't change the weight, or anything, it just adds extra information to the book. At the end of the page it states programmers who support the Polyglot format need to update their code to make use of the new features.
But your comment about weights made me think. I will make 2 extra functions that will set the weight on 100% of the best Stockfish move or the highest learn value so that people are no longer dependent on programmers.
Just keep an eye on the website, an update will take not so long, one week max.
Hi there,
The util doesn't change the weight, or anything, it just adds extra information to the book. At the end of the page it states programmers who support the Polyglot format need to update their code to make use of the new features.
But your comment about weights made me think. I will make 2 extra functions that will set the weight on 100% of the best Stockfish move or the highest learn value so that people are no longer dependent on programmers.
Just keep an eye on the website, an update will take not so long, one week max.
90% of coding is debugging, the other 10% is writing bugs.