Polyglot 64-bit

[Rebel]

Polyglot (as far as I can tell) always has been 32-bit which limited the working space during --make-book to 1Gb. Compiling Polyglot as a 64-bit program takes this obstacle away. I could make a Polyglot book of 1.5Gb in one run while Polyglot workspace increased to 5.5Gb of memory.

Not only is this more comfortable it also has the advantage you don't need --merge-book any longer which enables you to produce better books because --merge-book has the (nasty) habit not to recalculate the weights.

Caution - Now that you can make books as big as your memory allows you take note that when a created Polyglot book exceeds the size of 2Gb it can't be used by engines that access a Polyglot book with traditional 32-bit instructions, some of the Polyglot code needs an update first.

Download at the end of the page - http://rebel13.nl/download/books.html

[Ferdy]

Thanks Ed, successfully created a book from not less than 600k games.

Book 2 in the gui is from polyglot book built from polyglot64.

[Ozymandias]

Polyglot (as far as I can tell) always has been 32-bit which limited the working space during --make-book to 1Gb. Compiling Polyglot as a 64-bit program takes this obstacle away. I could make a Polyglot book of 1.5Gb in one run while Polyglot workspace increased to 5.5Gb of memory.

Not only is this more comfortable it also has the advantage you don't need --merge-book any longer which enables you to produce better books because --merge-book has the (nasty) habit not to recalculate the weights.

Caution - Now that you can make books as big as your memory allows you take note that when a created Polyglot book exceeds the size of 2Gb it can't be used by engines that access a Polyglot book with traditional 32-bit instructions, some of the Polyglot code needs an update first.

Download at the end of the page - http://rebel13.nl/download/books.html

Looking forward to breaking the 2Gb limit. 64 versions already existed, the first one was posted in this forum years ago, and there's also the so-called "tolerant" version. Both failed to address that size limitation.

As for the --merge-book option, I always liked it as is. I will try your Polyglot 64-bit as soon as I upgrade the memory subsystem (prices getting finally good).

[Rebel]

Polyglot (as far as I can tell) always has been 32-bit which limited the working space during --make-book to 1Gb. Compiling Polyglot as a 64-bit program takes this obstacle away. I could make a Polyglot book of 1.5Gb in one run while Polyglot workspace increased to 5.5Gb of memory.

Not only is this more comfortable it also has the advantage you don't need --merge-book any longer which enables you to produce better books because --merge-book has the (nasty) habit not to recalculate the weights.

Caution - Now that you can make books as big as your memory allows you take note that when a created Polyglot book exceeds the size of 2Gb it can't be used by engines that access a Polyglot book with traditional 32-bit instructions, some of the Polyglot code needs an update first.

Download at the end of the page - http://rebel13.nl/download/books.html

Looking forward to breaking the 2Gb limit. 64 versions already existed, the first one was posted in this forum years ago, and there's also the so-called "tolerant" version. Both failed to address that size limitation.

As for the --merge-book option, I always liked it as is. I will try your Polyglot 64-bit as soon as I upgrade the memory subsystem (prices getting finally good).

To bypass the 2Gb limit I made an option in ProDeo to access 4 books but only 1 search is needed.
Book-1 for move 0-5
Book-2 for move 6-10
Book-3 for move 11-15
Book-4 moves >=16

So in theory I can make a book close to 8Gb.

[Ozymandias]

To bypass the 2Gb limit I made an option in ProDeo to access 4 books but only 1 search is needed.
Book-1 for move 0-5
Book-2 for move 6-10
Book-3 for move 11-15
Book-4 moves >=16

So in theory I can make a book close to 8Gb.

Even if your engine of choice offered the chance to address so many books, which they don't, wouldn't books 2-4 need to include the first moves? There's a "-max-ply" opinion, but to do what you suggest, you'd also need a "-min-ply" one.

[Rebel]

To bypass the 2Gb limit I made an option in ProDeo to access 4 books but only 1 search is needed.
Book-1 for move 0-5
Book-2 for move 6-10
Book-3 for move 11-15
Book-4 moves >=16

So in theory I can make a book close to 8Gb.

Even if your engine of choice offered the chance to address so many books, which they don't, wouldn't books 2-4 need to include the first moves? There's a "-max-ply" opinion, but to do what you suggest, you'd also need a "-min-ply" one.

Not sure if you understood what I was trying to say. The 4-book feature is not yet availabe, that's what I understand from the red. Secondly, no, you don't need the first moves in book 2-4.

[Ozymandias]

To bypass the 2Gb limit I made an option in ProDeo to access 4 books but only 1 search is needed.
Book-1 for move 0-5
Book-2 for move 6-10
Book-3 for move 11-15
Book-4 moves >=16

So in theory I can make a book close to 8Gb.

Even if your engine of choice offered the chance to address so many books, which they don't, wouldn't books 2-4 need to include the first moves? There's a "-max-ply" opinion, but to do what you suggest, you'd also need a "-min-ply" one.

Not sure if you understood what I was trying to say. The 4-book feature is not yet availabe, that's what I understand from the red. Secondly, no, you don't need the first moves in book 2-4.

You said "made", which is past tense, so I assumed it was already available; anyway, how do you make a book containing (say) moves 6-10? I tried getting a look at how it works and after executing the program, I get a command line window, with the usual info in the first line, but this message in the second:

tellusererror Polyglot: EngineCommand error: corchess_x64_bmi2.exe

You can't type anything and it closes a short while later.

[Rebel]

To bypass the 2Gb limit I made an option in ProDeo to access 4 books but only 1 search is needed.
Book-1 for move 0-5
Book-2 for move 6-10
Book-3 for move 11-15
Book-4 moves >=16

So in theory I can make a book close to 8Gb.

Even if your engine of choice offered the chance to address so many books, which they don't, wouldn't books 2-4 need to include the first moves? There's a "-max-ply" opinion, but to do what you suggest, you'd also need a "-min-ply" one.

Not sure if you understood what I was trying to say. The 4-book feature is not yet availabe, that's what I understand from the red. Secondly, no, you don't need the first moves in book 2-4.

You said "made", which is past tense, so I assumed it was already available; anyway, how do you make a book containing (say) moves 6-10?

Is also part of what has to be released yet. But it works. See - http://rebel13.nl/download/data.html

It's currently not available from the menu but it's an example that it can be done. A large PGN is split into 3 books. No release because I am not satisfied with the result.

I tried getting a look at how it works and after executing the program, I get a command line window, with the usual info in the first line, but this message in the second:

tellusererror Polyglot: EngineCommand error: corchess_x64_bmi2.exe

You can't type anything and it closes a short while later.

I don't follow. corchess_x64_bmi2.exe ?

[Ozymandias]

Is also part of what has to be released yet. But it works. See - http://rebel13.nl/download/data.html

It's currently not available from the menu but it's an example that it can be done. A large PGN is split into 3 books. No release because I am not satisfied with the result.

I tried getting a look at how it works and after executing the program, I get a command line window, with the usual info in the first line, but this message in the second:

tellusererror Polyglot: EngineCommand error: corchess_x64_bmi2.exe

You can't type anything and it closes a short while later.

I don't follow. corchess_x64_bmi2.exe ?

So we're clear, can we make a book containing just moves 6-10, with the released executable? If so, what's the syntax to be used?

The quoted text is a copy/paste of what the program says upon double clicking it. Your guess is as good as mine.

On another machine, it also fails to work, giving the following message:

Can't open file "polyglot.ini": No such file or directory

If an ini file is required, shouldn't it be zipped with the exe?

[Rebel]

So we're clear, can we make a book containing just moves 6-10, with the released executable?

Yep, in the next POLY version.

On another machine, it also fails to work, giving the following message:

Can't open file "polyglot.ini": No such file or directory

If an ini file is required, shouldn't it be zipped with the exe?

You don't need a polyglot.ini using --make-book.

For all the other options of Polyglot use (copy) one of the previous versions.