Polyglot - merge-book oddity

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
Rebel
Posts: 7312
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: Polyglot - merge-book oddity

Post by Rebel »

To clarify -

d4.pgn

Code: Select all

[Event ""]
[Site ""]
[Date "1994.01.01"]
[White "white"]
[Black "black"]
[Result "1/2-1/2"]

1.d4 1/2-1/2
e4.pgn

Code: Select all

[Event ""]
[Site ""]
[Date "1994.01.01"]
[White "white"]
[Black "black"]
[Result "1/2-1/2"]

1.e4 1/2-1/2
polyglot make-book -pgn pgn\d4.pgn -bin books\d4.bin -max-ply 60 -min-game 1
polyglot make-book -pgn pgn\e4.pgn -bin books\e4.bin -max-ply 60 -min-game 1
polyglot merge-book -in1 d4.bin -in2 e4.bin -out both.bin

And both.bin contains only 1 record, 1.d4 , 1.e4 is skipped.

But never mind, I will write my own Polyglot merge-book.
90% of coding is debugging, the other 10% is writing bugs.
jefk
Posts: 956
Joined: Sun Jul 25, 2010 10:07 pm
Location: the Netherlands
Full name: Jef Kaan

Re: Polyglot - merge-book oddity

Post by jefk »

Ed,
Why is that confusing for you?
to be precisely, i wrote that 'it's a bit confusing' (wouldn't some say so?)
suggesting that it can be a bit confusing (for some people),
but it didn't say it's so confusing for *me* (although i do
consider the whole situation a bit unclear) :)

Good if you write a new polyglot with merge-book;
probably polyglot is also listed on your (Rebel)site.

PS as for what i consider to be a bit unclear sometimes in opensource:
sure, forks are possible (although i'm not a github expert); but
anyway this is the general (sub)forum, not the programmers
part; so imho it would be nice for 'ordinary' users to know where
the original stuff is, for whatever computer chess or other
software;maybe it's indicated when a fork is made, but it's not
always clear for me (for SF development versions, although i'm
not an ordinary 'user', i usually look at abrok.eu rather than
to these hundreds of forks, etc).
And in the sporadic situations i do look at github, the first
thing i learned to look at, is 'release'. Yes, i have a compiler
(codeblock) but even then it's sometimes a bit of a puzzle,
and why should 'users' (not even talking about myself) compile
everything for themselves ? In the good old crafty time there
were always 'fast' compiles made by others (eg. jim ablett)

PS2 talking about tool GUI's, there's pgn-extract, which is a bit
outdated, then someone talked about polyglot-tolerant, don't see
a GUI after quick inspection (some bat files, e,g. merge.bat (!))
and i know about the Poly1.7 tool by Rebel (which i in fact used to
...make a gambit book, but as far as i know there's no merge-book
feature in it (because there's polyglot somewhere else, i presume)
:)
Last edited by jefk on Sun Mar 21, 2021 1:36 pm, edited 4 times in total.
Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: Polyglot - merge-book oddity

Post by Guenther »

Rebel wrote: Sun Mar 21, 2021 12:57 pm ...

And both.bin contains only 1 record, 1.d4 , 1.e4 is skipped.

But never mind, I will write my own Polyglot merge-book.
I tested HGMs quick hack and it works out of the box

1.pgn

Code: Select all

[Event "?"]
[Site "?"]
[Date "2013.11.03"]
[Number "1"]
[White "?"]
[Black "?"]
[Result "1/2-1/2"]
[ECO "A01"]
[PlyCount "12"]

1. b3 e5 1/2-1/2
2.pgn

Code: Select all

[Event "?"]
[Site "?"]
[Date "????.??.??"]
[Number "2"]
[White "?"]
[Black "?"]
[Result "1/2-1/2"]
[ECO "A33"]
[PlyCount "12"]

1. c4 c5 1/2-1/2

Code: Select all

C:\ChessTools\Bookmake>polyglot make-book -pgn 1.pgn -min-game 1 -max-ply 12 -bin 1.bin
PolyGlot 2.0.4 by Fabien Letouzey.
inserting games ...
2 game.
2 entries.
filtering entries ...
2 entries.
sorting entries ...
saving entries ...
all done!

C:\ChessTools\Bookmake>polyglot make-book -pgn 2.pgn -min-game 1 -max-ply 12 -bin 2.bin
PolyGlot 2.0.4 by Fabien Letouzey.
inserting games ...
2 game.
2 entries.
filtering entries ...
2 entries.
sorting entries ...
saving entries ...
all done!

C:\ChessTools\Bookmake>polyglot merge-book -in1 1.bin -in2 2.bin -out both.bin
PolyGlot 2.0.4 by Fabien Letouzey.
done!
Checked the result in ScidvsPC - both moves are in (both.bin) with 50% now

(I can upload my fresh polyglot compilation if desired...)
https://rwbc-chess.de

[Trolls n'existent pas...]
Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: Polyglot - merge-book oddity

Post by Guenther »

jefk wrote: Sun Mar 21, 2021 1:18 pm

PS2 talking about tool GUI's, there's pgn-extract, which is a bit
outdated, ...
Why is it outdated? It is still updated (last update a few weeks ago) and it is one of the mightiest pgn tools around since ever and
probably forever ;-) (and a lot of tools are built around it until today)

https://www.cs.kent.ac.uk/people/staff/djb/pgn-extract/
https://rwbc-chess.de

[Trolls n'existent pas...]
jefk
Posts: 956
Joined: Sun Jul 25, 2010 10:07 pm
Location: the Netherlands
Full name: Jef Kaan

Re: Polyglot - merge-book oddity

Post by jefk »

about pgn-extract
Why is it outdated?
ok as for 'outdated' well a better word would be maybe 'incomplete'.
but i'll have a look at the latest version on your site, anyway
As you know the Barnes console program pgn-extract has
lots of features, about a month a go i needed the --splitfiles option
of pgn-extract, and managed to do it with the original (not GUI) pgn-extract.
But then many options are not used so very often i presume.

PS found polyglot(64) included in the (Rebel) Poly17 dir ,
and a poly-merge.bat file, so now i only have to wait for
a little update regarding the merge-book routine
(in the polyglot64.exe) ...
No problem
:)
Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: Polyglot - merge-book oddity

Post by Guenther »

jefk wrote: Sun Mar 21, 2021 1:56 pm about pgn-extract
Why is it outdated?
ok as for 'outdated' well a better word would be maybe 'incomplete'.
but i'll have a look at the latest version on your site, anyway
As you know the Barnes console program pgn-extract has
lots of features, about a month a go i needed the --splitfiles option
of pgn-extract, and managed to do it with the original (not GUI) pgn-extract.
But then many options are not used so very often i presume.

PS found polyglot(64) included in the (Rebel) Poly17 dir ,
and a poly-merge.bat file, so now i only have to wait for
a little update regarding the merge-book routine
(in the polyglot64.exe) ...
No problem
:)
Well, I uploaded the already compiled fixed version from HGM here:
https://drive.google.com/drive/folders/ ... E15NjVyY2M
https://rwbc-chess.de

[Trolls n'existent pas...]
jefk
Posts: 956
Joined: Sun Jul 25, 2010 10:07 pm
Location: the Netherlands
Full name: Jef Kaan

Re: Polyglot - merge-book oddity

Post by jefk »

ok Guenther
thx for your new polyglot :)
(just downloaded)

PS about pgn-extract, there was a slight misunderstanding,
what i meant with 'outdated' was a specific windows GUIS thus
graphical 'interface' , not the program pgn-extract.exe itself

see here (F.Mosca's site)
https://sites.google.com/site/deuterium ... -interface
(this version is from 2011

best,
User avatar
Rebel
Posts: 7312
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: Polyglot - merge-book oddity

Post by Rebel »

jefk wrote: Sun Mar 21, 2021 1:18 pm Ed,
Why is that confusing for you?
I did not wrote that :D
to be precisely, i wrote that 'it's a bit confusing' (wouldn't some say so?)
suggesting that it can be a bit confusing (for some people),
but it didn't say it's so confusing for *me* (although i do
consider the whole situation a bit unclear) :)

Good if you write a new polyglot with merge-book;
probably polyglot is also listed on your (Rebel)site.

PS as for what i consider to be a bit unclear sometimes in opensource:
sure, forks are possible (although i'm not a github expert); but
anyway this is the general (sub)forum, not the programmers
part; so imho it would be nice for 'ordinary' users to know where
the original stuff is, for whatever computer chess or other
software;maybe it's indicated when a fork is made, but it's not
always clear for me (for SF development versions, although i'm
not an ordinary 'user', i usually look at abrok.eu rather than
to these hundreds of forks, etc).
And in the sporadic situations i do look at github, the first
thing i learned to look at, is 'release'. Yes, i have a compiler
(codeblock) but even then it's sometimes a bit of a puzzle,
and why should 'users' (not even talking about myself) compile
everything for themselves ? In the good old crafty time there
were always 'fast' compiles made by others (eg. jim ablett)

PS2 talking about tool GUI's, there's pgn-extract, which is a bit
outdated, then someone talked about polyglot-tolerant, don't see
a GUI after quick inspection (some bat files, e,g. merge.bat (!))
and i know about the Poly1.7 tool by Rebel (which i in fact used to
...make a gambit book, but as far as i know there's no merge-book
feature in it (because there's polyglot somewhere else, i presume)
:)
Merge-Book is available in Poly 1.7 pressing [F4], but better wait for the new one to come.
90% of coding is debugging, the other 10% is writing bugs.
User avatar
hgm
Posts: 28354
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Polyglot - merge-book oddity

Post by hgm »

There is no 'original' Polyglot anymore. The original author has stpped working on it more than a decade ago.

All I can say is that for inclusion as a protocol adapter in WinBoard packages that I use Polyglot 2.x. The source of which is available in the 'learn' branch in my on-line repository at http://hgm.nubati.net/cgi-bin/gitweb.cgi .
User avatar
Rebel
Posts: 7312
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: Polyglot - merge-book oddity

Post by Rebel »

Guenther wrote: Sun Mar 21, 2021 1:58 pm
jefk wrote: Sun Mar 21, 2021 1:56 pm about pgn-extract
Why is it outdated?
ok as for 'outdated' well a better word would be maybe 'incomplete'.
but i'll have a look at the latest version on your site, anyway
As you know the Barnes console program pgn-extract has
lots of features, about a month a go i needed the --splitfiles option
of pgn-extract, and managed to do it with the original (not GUI) pgn-extract.
But then many options are not used so very often i presume.

PS found polyglot(64) included in the (Rebel) Poly17 dir ,
and a poly-merge.bat file, so now i only have to wait for
a little update regarding the merge-book routine
(in the polyglot64.exe) ...
No problem
:)
Well, I uploaded the already compiled fixed version from HGM here:
https://drive.google.com/drive/folders/ ... E15NjVyY2M
It's not good enough, one gets plenty of doubles. Merging the 24 gambit books together nearly doubled the out-book.

Code: Select all

d4	3%	+0.24/20 Stockfish 13
d4	6%	+0.24/20 Stockfish 13
d4	6%	+0.24/20 Stockfish 13
d4	4%	+0.24/20 Stockfish 13
d4	4%	+0.24/20 Stockfish 13
d4	1%	+0.24/20 Stockfish 13
d4	1%	+0.24/20 Stockfish 13
d4	4%	+0.24/20 Stockfish 13
d4	6%	+0.24/20 Stockfish 13
d4	4%	+0.24/20 Stockfish 13
d4	1%	+0.24/20 Stockfish 13
d4	1%	+0.24/20 Stockfish 13
d4	6%	+0.24/20 Stockfish 13
e4	1%	+0.10/20 Stockfish 13
e4	0%	+0.10/20 Stockfish 13
e4	1%	+0.10/20 Stockfish 13
e4	1%	+0.10/20 Stockfish 13
e4	5%	+0.10/20 Stockfish 13
e4	4%	+0.10/20 Stockfish 13
e4	0%	+0.10/20 Stockfish 13
e4	1%	+0.10/20 Stockfish 13
e4	0%	+0.10/20 Stockfish 13
e4	2%	+0.10/20 Stockfish 13
e4	0%	+0.10/20 Stockfish 13
e4	4%	+0.10/20 Stockfish 13
e4	1%	+0.10/20 Stockfish 13
e4	1%	+0.10/20 Stockfish 13
e4	0%	+0.10/20 Stockfish 13
e4	2%	+0.10/20 Stockfish 13
e4	6%	+0.10/20 Stockfish 13
e4	0%	+0.10/20 Stockfish 13
e4	4%	+0.10/20 Stockfish 13
e4	5%	+0.10/20 Stockfish 13
e4	1%	+0.10/20 Stockfish 13
e4	0%	+0.10/20 Stockfish 13
e4	2%	+0.10/20 Stockfish 13
e4	4%	+0.10/20 Stockfish 13
f4	4%	-0.39/20 Stockfish 13
f4	4%	-0.39/20 Stockfish 13
But it's a start, I can weed out the doubles afterwards.
90% of coding is debugging, the other 10% is writing bugs.