Export bin book to pgn

[Jonathan003]

Can someone help with this please:

https://sourceforge.net/p/scid/feature-requests/76/

Or if someone knows how this can be done with Scid vs PC (or with any other software tools), that would also be ok for me. But I have seen that they removed the option to export a bin book to a merged game, with the latest version of Scid vs PC.

I have tried multiple ways to export a bin book to pgn, but without success.
I have best results by exporting the book with SCID, but there is a limitation of 3000 moves, and I want the complete bin books to be exported.
SCID creates a huge merged pgn game with the opening lines of the bin book, I would preferer multiple games but a big merged game is ok for me. (I know Chess Position Trainer, and Chess Opening Wizard, don't have problems handling huge merged pgn games).
It is important for me that all lines (and only these lines), that are visible when browsing the bin book in Scid, gets exported.

I also have tried Lucas ChessR, by importing uniform bin books to Opening Lines, and exporting these Opening Lines to pgn. But there is a problem with multiple endless repetition of moves, in the opening lines, when importing moves from uniform bin books. There is also some strange explosion of number of games, when trying to export Opening Lines, created by importing uniform bin books.

Lucas ChessR, is still under heavy development, and I understand the software is not intended to be used like his.
I also have tried to convert bin books with the convert option in Banksia GUI. That also doesn't give good results. I lot of lines that that are visible when browsing the bin book in SCID, doesn't get exported. Also lines that are not visible when browsing bin books in SCID, do get exported. The same problem when I try to export to pgn with Polyglot Tolerant.

I have made a video capture to show how exporting a bin book with SCID works:

[hgm]

I did not even know there were tools that could convert a bin book to PGN. Bin books are not made for supporting that, because the hash keys used to represent the positions are not invertible to a position. So in principle it is impossible to do that. A work-around would be to do a perft-like tree-walk through the opening tree, but only following moves listed in the book. This only works if the book is connected, and there are no positions in it unreachable from the start position. For a white-only or a black-only book that is by definition not the case, but you could extend the work-around to handle that special case of a disconnected book, by always trying all moves of the player that is not in the book. Still, there is no guarantee that would access the entire book.

Another problem is that the task seems ambiguously defined. A bin book will contain a number positions, with a number of moves plus a weight in each position. You could hope that the weights reflect the number of points scored with that move from that position in the set of games used to create the book, but since people can edit the weights, this can be a wrong assumption. And what exactly do you expect to come out? In the simplistic case that I would make a book of a PGN file that contains only 1000 copies of the same (drawn) game, all moves would get weight 1000. Would you expect the 'book dump' to also repeat the game 1000 times? Or just once?

Also, when a position was reached to transposition in a different way, in the original game set, you cannot unambiguously untangle it. If there is a game that goes through positions A - B - C - D - E - F - G and one that goes through A - B' - C' - D - E' - F' - G', it would give the same book as the games A - B - C - D - E' - F' - G' and A - B' - C' - D - E - F - G, so you would never know which of the two you should print.

Referring to you rothe posting about this subject:

Why would you want to make separate white and black books? Is that just to be able to put a different percentage threshold on the two colors? This could already be done by allowing specification of different thresholds for white and black. In fact specifying thresholds can be seen as a continuous version of a white-only or black-only option, as you could suppress all black moves (and thus get a white-only book) by requiring the black percentage is > 101%.

[Jonathan003]

I did not even know there were tools that could convert a bin book to PGN. Bin books are not made for supporting that, because the hash keys used to represent the positions are not invertible to a position. So in principle it is impossible to do that. A work-around would be to do a perft-like tree-walk through the opening tree, but only following moves listed in the book. This only works if the book is connected, and there are no positions in it unreachable from the start position. For a white-only or a black-only book that is by definition not the case, but you could extend the work-around to handle that special case of a disconnected book, by always trying all moves of the player that is not in the book. Still, there is no guarantee that would access the entire book.

Another problem is that the task seems ambiguously defined. A bin book will contain a number positions, with a number of moves plus a weight in each position. You could hope that the weights reflect the number of points scored with that move from that position in the set of games used to create the book, but since people can edit the weights, this can be a wrong assumption. And what exactly do you expect to come out? In the simplistic case that I would make a book of a PGN file that contains only 1000 copies of the same (drawn) game, all moves would get weight 1000. Would you expect the 'book dump' to also repeat the game 1000 times? Or just once?

Also, when a position was reached to transposition in a different way, in the original game set, you cannot unambiguously untangle it. If there is a game that goes through positions A - B - C - D - E - F - G and one that goes through A - B' - C' - D - E' - F' - G', it would give the same book as the games A - B - C - D - E' - F' - G' and A - B' - C' - D - E - F - G, so you would never know which of the two you should print.

This is way to complicated for me because I have no programming knowledge.
I only know it is possible because it works with SCID to export a bin book to pgn, and the result is perfect. There is only for some reason a limitation of 3000 moves when exporting a bin book in SCID. I think with modern hardware there is no need for such a limitation. Maybe someone with good programming skills can undo this limitation for SCID. Scid is open source so everyone can contribute to it's development, at least this is how I understand it.

Here is an example of a small bin book (for my white repertoire), I would like to convert to pgn:

[Jonathan003]

Referring to you rothe posting about this subject:

Why would you want to make separate white and black books? Is that just to be able to put a different percentage threshold on the two colors? This could already be done by allowing specification of different thresholds for white and black. In fact specifying thresholds can be seen as a continuous version of a white-only or black-only option, as you could suppress all black moves (and thus get a white-only book) by requiring the black percentage is > 101%.

I make two bin books, one for my white repertoire and one for my black repertoire.
I start with a only-white bin book for my white repertoire. Because I use poly 1.7a to blunder check the bin books. There is no option in poly 1.7a to blundercheck only the white moves, of a bin book if the book has moves both for white as for black. That's why I start with a only-white book. After the blundercheck, I make a only-black book (for my white repertoire), from the same games where the only-white book is made of, and I merge the books.

[hgm]

So it seems that what you really need is an option to limit blunder checking in Polyglot 1.7a to only one player.

[Fulvio]

There is only for some reason a limitation of 3000 moves

Try if increasing that limit works:
https://sourceforge.net/p/scid/code/ci/ ... ok.tcl#l18

[Jonathan003]

Thanks for the tip. I will try it with Scid 4.7
I have tried this already with Scid vs PC, and that didn't work.

https://sourceforge.net/p/scidvspc/disc ... f2fc4c1e1/

To what value do you recommend me to change it? 300.0000 or 3000.0000

[Jonathan003]

So it seems that what you really need is an option to limit blunder checking in Polyglot 1.7a to only one player.

Poly 1.7a is not created with the purpose of making repertoires for over the board games.
I wonder if it is possible somehow to make a only-white or a only-black bin book, from a bin book that has moves for both collars?
Than I can use obk2bin and use the resulting bin books.

[Jonathan003]

I find an alternative way to convert bin books to pgn.
With polyglot-tolerant there is an option to dump a bin book.
This creates a text file like 'book_black.text'
If you change this extension to pgn, you can open this pgn with the software ChessX.
And in ChessX, you can export this pgn database to a new pgn database to normalize the pgn.
The result is not perfect but it looks pretty good.
So thanks to the developers of ChessX, to make this possible.

Does someone know a better ways to convert a bin book to pgn? I want to do this for creating repertoires for over the board games.

[hgm]

I am not sure what 'Polyglot-tolerant' is. But did the standard Polyglot option 'dump-book' with a -color argument not work for you?