Polyglot extension tool

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Jonathan003
Posts: 46
Joined: Fri Jul 06, 2018 2:23 pm
Full name: Jonathan Cremers

Re: Polyglot extension tool

Post by Jonathan003 » Fri Jan 18, 2019 12:02 am

There is a tool that can convert bin books to pgn called
Cerebellum book CONVERTOR

http://www.chessgod101.com/t3840-cerebe ... -convertor

Of course I don't know if it's done correctly.

I can also import best lines from a bin book (for one color), or al best lines (for one color), to opening lines in Lucas Chess.
And in turn export this opening lines to a pgn file.

In scid it's also possible to load a small bin book and export the bin book to pgn, but that's very limited in size and takes a long time.

Ferdy
Posts: 3935
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Polyglot extension tool

Post by Ferdy » Fri Jan 18, 2019 11:31 am

Rebel wrote:
Sun Jan 06, 2019 3:18 pm
Poly version 1.0

The polyglot book format is the most widespread format in use by chess engines. The format contains 4 unused bytes. Have a look at the nice things you can do using those bytes.

http://rebel13.nl/download/books.html
I have a polyglot book created from pgn file. I also have some epd's analyzed out of this pgn file. Can I use poly to just add/update the score in bin book using epd's with the following format?

Code: Select all

r1bqkbnr/pppp1ppp/2n5/4p3/4P3/5N2/PPPP1PPP/RNBQKB1R w KQkq - ce 50;

User avatar
hgm
Posts: 23296
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Polyglot extension tool

Post by hgm » Fri Jan 18, 2019 12:40 pm

Jonathan003 wrote:
Fri Jan 18, 2019 12:02 am
There is a tool that can convert bin books to pgn called
Cerebellum book CONVERTOR
You would have to be careful with that, as converting Polyglot books to PGN is not generally possible. There is no way to convert the position keys in the book back to positions. So you would be dependent on following the opening lines from the start position to see to which position they lead, and then see if that position occurs in the book. This is fine if all the lines in the book are uninterrupted, so that every position in the book is reachable from the start position by a sequence of moves from the book. But even in a book genererated by Polyglot that is not always the case, as moves with zero weight are not stored. So if a move only scored losses, the position after that move (where all moves scored only wins) will be disconnected from the opening tree, and just following the book moves will miss it. You could work around that by generating all moves from leaf positions, to see if they were omitted this way, or even deeper perfts. But there is no guarantee you would find everything in the book.

Also, if the book weights were tuned, it can be impossible for a PGN collection to correctly reflect the weights, so that converting it back to a book would give you a different book.

User avatar
Rebel
Posts: 4466
Joined: Thu Aug 18, 2011 10:04 am

Re: Polyglot extension tool

Post by Rebel » Fri Jan 18, 2019 4:30 pm

Ferdy wrote:
Fri Jan 18, 2019 11:31 am
Rebel wrote:
Sun Jan 06, 2019 3:18 pm
Poly version 1.0

The polyglot book format is the most widespread format in use by chess engines. The format contains 4 unused bytes. Have a look at the nice things you can do using those bytes.

http://rebel13.nl/download/books.html
I have a polyglot book created from pgn file. I also have some epd's analyzed out of this pgn file. Can I use poly to just add/update the score in bin book using epd's with the following format?

Code: Select all

r1bqkbnr/pppp1ppp/2n5/4p3/4P3/5N2/PPPP1PPP/RNBQKB1R w KQkq - ce 50;
As long as the EPD is in the book, sure.

I need to add 2 more options to the tool, import from PGN and import from EPD.

Not much work, stay tuned.
Everybody is unique, except me.

User avatar
Rebel
Posts: 4466
Joined: Thu Aug 18, 2011 10:04 am

Re: Polyglot extension tool

Post by Rebel » Sat Jan 19, 2019 10:30 pm

Rebel wrote:
Fri Jan 18, 2019 4:30 pm
Ferdy wrote:
Fri Jan 18, 2019 11:31 am
Rebel wrote:
Sun Jan 06, 2019 3:18 pm
Poly version 1.0

The polyglot book format is the most widespread format in use by chess engines. The format contains 4 unused bytes. Have a look at the nice things you can do using those bytes.

http://rebel13.nl/download/books.html
I have a polyglot book created from pgn file. I also have some epd's analyzed out of this pgn file. Can I use poly to just add/update the score in bin book using epd's with the following format?

Code: Select all

r1bqkbnr/pppp1ppp/2n5/4p3/4P3/5N2/PPPP1PPP/RNBQKB1R w KQkq - ce 50;
As long as the EPD is in the book, sure.

I need to add 2 more options to the tool, import from PGN and import from EPD.

Not much work, stay tuned.
That was a bad answer, there has to be a "bm" in the EPD.

Version update tomorrow.
Everybody is unique, except me.

User avatar
Rebel
Posts: 4466
Joined: Thu Aug 18, 2011 10:04 am

Re: Polyglot extension tool

Post by Rebel » Sun Jan 20, 2019 10:13 am

UPDATE version 1.2

http://rebel13.nl/download/books.html

Added 4 essential functions which allows the user to immediately make use of the new features without programmer support.

Weights on score
1. Option 'B' (best move) sets the weight of moves on 100% with the highest Stockfish score. The book will only play those 100% moves. Using this function basically emulates the results in table-1 above.

2. Option 'V' (variation) based on the Stockfish scores will calculate new weights.

Weights on learn
1. Option 'B' (best move) sets the weight of moves on 100% with the highest learn value. The book will only play those 100% moves. Using this function basically emulates the results in table-2 above.

2. Option 'V' (variation) based on the learn scores will calculate new weights.

Import Score and Depth from PGN
Import Score & Depth from an annotated (thus with Score and Depth) ENG-ENG match.

Import Score and Depth from EPD
. Required tags: 'bm' (move) and 'ce' (score).
. Preferable (although not obliged) tag: 'acd' (depth)

The advantage of the two last options is that you can add the analysis (score & depth) of any engine to Polyglot books.

Check the new Polyglot weights with ProDeo 2.9 (book.txt) or with Scid.

______________________________________________________________

If there are new wishes now is a good time.
Everybody is unique, except me.

User avatar
Ozymandias
Posts: 1071
Joined: Sun Oct 25, 2009 12:30 am

Re: Polyglot extension tool

Post by Ozymandias » Sun Jan 20, 2019 12:46 pm

Rebel wrote:
Sun Jan 20, 2019 10:13 am
Weights on learn
1. Option 'B' (best move) sets the weight of moves on 100% with the highest learn value. The book will only play those 100% moves. Using this function basically emulates the results in table-2 above.

2. Option 'V' (variation) based on the learn scores will calculate new weights.
I have to play with this. Thanks.

Jonathan003
Posts: 46
Joined: Fri Jul 06, 2018 2:23 pm
Full name: Jonathan Cremers

Re: Polyglot extension tool

Post by Jonathan003 » Sun Jan 20, 2019 1:38 pm

Hi Ed

I tried the new version of poly 1.2
If I chose for the option 'V' to calculate new weights.
And I openen the new created book in SCID. I can't see the weight of the moves. But just strange characters.

I have included a screenshot in the attachment.

If I try to import this new bin book to opening lines in Lucas Chess. Everything seems to work fine.

I want to be able to see the weights of the moves in SCID to. If I want to change some weights manually.
Attachments
screenshot scid.png
screenshot scid.png (176.99 KiB) Viewed 941 times

User avatar
Rebel
Posts: 4466
Joined: Thu Aug 18, 2011 10:04 am

Re: Polyglot extension tool

Post by Rebel » Sun Jan 20, 2019 4:22 pm

That's odd, everything okay with ProDeo. I will look into it.
Everybody is unique, except me.

User avatar
Rebel
Posts: 4466
Joined: Thu Aug 18, 2011 10:04 am

Re: Polyglot extension tool

Post by Rebel » Sun Jan 20, 2019 11:39 pm

Jonathan003 wrote:
Sun Jan 20, 2019 1:38 pm
Hi Ed

I tried the new version of poly 1.2
If I chose for the option 'V' to calculate new weights.
And I openen the new created book in SCID. I can't see the weight of the moves. But just strange characters.

I have included a screenshot in the attachment.

If I try to import this new bin book to opening lines in Lucas Chess. Everything seems to work fine.

I want to be able to see the weights of the moves in SCID to. If I want to change some weights manually.
Everything is running fine on my end. But I have an idea why things got wrong on your end. The by default created NEW.BIN is meant to keep your orginal unchanged. However if you start with NEW.BIN as source the destination is the same and this result in a conflict, hence the strange output. I will build in a check that source (NEW.BIN) can never be as the same as the destination (NEW.BIN).

So try again.
Everybody is unique, except me.

Post Reply