Idea for new option for making polyglot bin books

[Jonathan003]

I would like to have a uniform option, and an option to make a only_white or only_black book. And an option to merge books.
These options are already included in most polyglot book making tools.
I would like to have this new option for building bin books. Next to 'Minimum number or games' and 'Minimum score', I would like to have an option to specify how much percentage of the time a move have to be played.
For my white repertoire than, I would for example make a 'only_white' book, from I collection of high quality games, and choose the option 'min_percentage=20'. Than I make a 'only_black' book, and choose the option 'min_percentage=0.5'
Than I merge white and black books, and I import the book to Opening Lines in Lucas Chess, and I have my repertoire.

[phhnguyen]

I would like to have a uniform option, and an option to make a only_white or only_black book. And an option to merge books.
These options are already included in most polyglot book making tools.
I would like to have this new option for building bin books. Next to 'Minimum number or games' and 'Minimum score', I would like to have an option to specify how much percentage of the time a move have to be played.
For my white repertoire than, I would for example make a 'only_white' book, from I collection of high quality games, and choose the option 'min_percentage=20'. Than I make a 'only_black' book, and choose the option 'min_percentage=0.5'
Than I merge white and black books, and I import the book to Opening Lines in Lucas Chess, and I have my repertoire.

I believe it may work for some cases but create big questions for other cases.

Typically both humans & computers play very fast in the opening period, almost instantly for high-level players. Do you want to ignore all their moves (from high-level players)?

Logically, humans & computers may spend more time on blunder or hard positions, some moves that lead to them must be blunder/low quality, or those players have not studied/awarded those openings at all. Do you want to count their moves to your opening books? They maybe get hit when vs low-level players but are totally useless when vs high-level ones.

[Jonathan003]

Typically both humans & computers play very fast in the opening period, almost instantly for high-level players. Do you want to ignore all their moves (from high-level players)?

No that's why I would use a low percentage for 'min_percentage' for the other color like 0.5% or 0.4%
I hink you don't understand my idea I try to explain.
I would make two bin books, one for my white repertoire with the setting 'min_percentage' for white is 20%, and 'min_percentage' for black is 0.5% for example.
For my black repertoire I would make a bin book with the setting 'min_percentage' for white is 0.5%, and 'min_percentage' for black is 20% for example.
Its just a starting point for making a repertoire, of course I still will have to do some blunder checking, and looking at game statics. But it would be a good way to start so I don't have to do everything manually.

[phhnguyen]

I still missed your points.

What the purpose of that book? Who will use that book (I guess it’s not for chess engines)? If for studying, could a database with opening tree satisfy your needs? From where will you collect games? Your wife’s games only? (Usually engines play instantly the opening period, thus their games almost fail for any time limit). What is the benefit when the thinking time above the limit? What happen if some moves are longer than the limit but some other moves shorter? Discard whole games?

[Jonathan003]

John Merlino made a obk making tool, where you can set different percentages for the book weights (the three dot's). When making a obk book and choose the option weight by frequently, you can now specify how much percentage a move have to be played to get one, two, three or no dot's assigned. Then you can export the obk book to pgn with obk2pgn and remove sidelines (moves with no dot's), by searching for games with a '?' in the annotation and delete them. That's a great approach for making repertoires for over the board games.

The only problem I can see is in positions were all moves where played equally. Or if you set for example a option min-percentage is 20%, and there are position where none of the variations are played at least 20% of the time. But there must be a way to solve this to. Maybe some way to search for these positions, and to set the weight for these positions manually.

I would like to have a similar option for building bin books.
I have put all the tools and files I use together in one folder on my Google drive. I will send you the link in PM. I have also made a detailed how to text file, that explains how I use the tools for building repertoires. So you can better understand what I'm trying to explain.

[hgm]

I also don't quite understand your purpose. What do you want to do with that book? Have an engine play from it? Consult it yourself, for moves in a given position? If the latter, what tool would you use to view the moves? And why would it bother you that there are moves in the book that were played less frequently than 20% (or 0.5%)? If all moves would be listed with their percentages, you could simply ignore the moves that have a lower percentage than your threshold. (These are score percentages rather than actual playing frequency, but it doesn't seem a good idea to also include losses in the count. I mean, if 30% of the players would chosses move A, and this resulted in a loss in every case, would you really want to play that move?)

Books for one color are usually made by selecting the games where that color won, and convert those to a book. Then the other color would never get any moves in the book, as none of his move ever scored a point.

[Jonathan003]

My purpose is to build a repertoire in pgn format. Based on some example player, (it can also be from games you played yourself). And to fill in the gaps of the repertoire from a bigger reference database. The repertoire may not be to big, and the variations may not be to long, because I need to be able to study it. I want the variation to end after main theory ends.
If I have my repertoires in pgn format there are many ways to study it. For example with Chess Position Trainer, or the android app Chess Trainer PRO - Repertoire Builder. https://play.google.com/store/apps/deta ... n_US&gl=US
I want to avoid spending hours and hours selecting lines manually, so I'm looking for some (partly), automated method.
If I could do this directly with the pgn format this would be great but I don't think this can be done. That's why I first build obk books or bin books.
In Lucas Chess R there is an option to import moves from a bin book, into Opening Lines, and these opening lines can be exported to pgn.
I also think about using the poly1.7a tool by Ed Schröder, to blunder check bin books, and automatically remove bad lines, for the color of the repertoire. That's why I need an option to build an only-white or an only-black bin book.
I hope this explains it better what I'm trying to achieve.

[Jonathan003]

(These are score percentages rather than actual playing frequency, but it doesn't seem a good idea to also include losses in the count. I mean, if 30% of the players would chosses move A, and this resulted in a loss in every case, would you really want to play that move?)

I want an additional option for making bin books to use 'min percentage of playing frequency', next to 'min moves' and 'min score percentages'

[Jonathan003]

If you make a obk book from a pgn database with MakeBook_v3.2, and choose the option depth=999
And afterwards you convert back to pgn with obk2pgn
Than you have exact the same games that you imported in the obk book, but with annotations according the book weights, and without the original game statics.
And if you remove the sidelines by searching for games with a '?' in the annotation, and delete these games. Than you have a pgn with only the main lines. Than you can easily search for the original games with these mainlines, by putting the pgn with the mainlines together with the original games, and searching for doubles with exact the same moves in Chessbase.
I think it is much easier to convert obk books back to pgn than bin books.