Polyglot extension tool

[Rebel]

A possible explanation can be that you haven't cleared the scores (the "Clear Analysis" function) and the "g2-g4" comes from a previous session, although I must say that a score of 0.31 remains suspect.

Maybe you can run it again with "Clear Analysis"?

Tried but still same result, I sent a PM with d.bin and new.epd download link.

Thanks.

When I load d.bin in ProDeo I get the same list except for g2-g4, not 0.31 (according to your list) but -0.69, see below. 31+69=100, maybe some sort of sign issue? -0.69 looks pretty correct compared to +0.31 for 1.g4

Code: Select all

Opening Book : books\d.bin
Positions    : 381.111

Book Percent Score Depth Learn    Move   Weight(%)  Score  Depth  Learn
d2d4  36.14%     3   20     0     d2d4       36.00   0.03     20      0
e2e4  36.14%     1   20     0     e2e4       36.00   0.02     20      0
g1f3  12.05%     0   20     0     g1f3       12.00  -0.00     20      0
c2c4  12.05%    -9   19     0     c2c4       12.00  -0.09     19      0
b2b3   1.20%    -5   19     0     b2b3        1.00  -0.05     19      0
g2g3   1.20%   -14   21     0     g2g3        1.00  -0.14     21      0
f2f4   1.20%   -21   19     0     f2f4        1.00  -0.21     19      0
b1c3   0.00%   -14   19     0     b1c3        0.00  -0.14     19      0
b2b4   0.00%   -45   19     0     b2b4        0.00  -0.45     19      0
e2e3   0.00%     0   21     0     e2e3        0.00   0.00     21      0
d2d3   0.00%   -21   19     0     d2d3        0.00  -0.21     19      0
a2a3   0.00%   -10   18     0     a2a3        0.00  -0.11     18      0
g2g4   0.00%   -69   18     0     g2g4        0.00   0.31     18      0

BTW I tried to import a single epd to d.bin by Import Analysis.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 20; acd 22;
But after checking with Poly statistics, Analyzed Positions is 0. So this was not imported. So what is the requirement of the epd format that can be imported?

I will look into that later, have little time at the moment.

[Rebel]

BTW I tried to import a single epd to d.bin by Import Analysis.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 20; acd 22;
But after checking with Poly statistics, Analyzed Positions is 0. So this was not imported. So what is the requirement of the epd format that can be imported?

I checked, it is imported. When I run Poly stats a second time it also shows the correct counter.

[Ferdy]

A possible explanation can be that you haven't cleared the scores (the "Clear Analysis" function) and the "g2-g4" comes from a previous session, although I must say that a score of 0.31 remains suspect.

Maybe you can run it again with "Clear Analysis"?

Tried but still same result, I sent a PM with d.bin and new.epd download link.

Thanks.

When I load d.bin in ProDeo I get the same list except for g2-g4, not 0.31 (according to your list) but -0.69, see below. 31+69=100, maybe some sort of sign issue? -0.69 looks pretty correct compared to +0.31 for 1.g4

Code: Select all

Opening Book : books\d.bin
Positions    : 381.111

Book Percent Score Depth Learn    Move   Weight(%)  Score  Depth  Learn
d2d4  36.14%     3   20     0     d2d4       36.00   0.03     20      0
e2e4  36.14%     1   20     0     e2e4       36.00   0.02     20      0
g1f3  12.05%     0   20     0     g1f3       12.00  -0.00     20      0
c2c4  12.05%    -9   19     0     c2c4       12.00  -0.09     19      0
b2b3   1.20%    -5   19     0     b2b3        1.00  -0.05     19      0
g2g3   1.20%   -14   21     0     g2g3        1.00  -0.14     21      0
f2f4   1.20%   -21   19     0     f2f4        1.00  -0.21     19      0
b1c3   0.00%   -14   19     0     b1c3        0.00  -0.14     19      0
b2b4   0.00%   -45   19     0     b2b4        0.00  -0.45     19      0
e2e3   0.00%     0   21     0     e2e3        0.00   0.00     21      0
d2d3   0.00%   -21   19     0     d2d3        0.00  -0.21     19      0
a2a3   0.00%   -10   18     0     a2a3        0.00  -0.11     18      0
g2g4   0.00%   -69   18     0     g2g4        0.00   0.31     18      0

Thanks will recheck the formula I used.

[Ferdy]

BTW I tried to import a single epd to d.bin by Import Analysis.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 20; acd 22;
But after checking with Poly statistics, Analyzed Positions is 0. So this was not imported. So what is the requirement of the epd format that can be imported?

I checked, it is imported. When I run Poly stats a second time it also shows the correct counter.

I saved this epd
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 20; acd 22;
to 1.epd and import it. Still it is 0. Second time 0 too. Redownload poly and tried again still 0.

Analyze that 1.epd using poly and it outputs new.epd with the following format.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - id sf10x.exe; bm e2e4; ce 91; acd 20; c1 Threads 1, Hash 128, Time 2000ms;
Import that new.epd to d.bin, check with poly statistics and it is 0, tried again it is still 0.

Perhaps the uploaded poly (the one that I have) is different from the poly that you use.

[Rebel]

BTW I tried to import a single epd to d.bin by Import Analysis.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 20; acd 22;
But after checking with Poly statistics, Analyzed Positions is 0. So this was not imported. So what is the requirement of the epd format that can be imported?

I checked, it is imported. When I run Poly stats a second time it also shows the correct counter.

I saved this epd
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 20; acd 22;
to 1.epd and import it. Still it is 0. Second time 0 too. Redownload poly and tried again still 0.

Analyze that 1.epd using poly and it outputs new.epd with the following format.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - id sf10x.exe; bm e2e4; ce 91; acd 20; c1 Threads 1, Hash 128, Time 2000ms;
Import that new.epd to d.bin, check with poly statistics and it is 0, tried again it is still 0.

Perhaps the uploaded poly (the one that I have) is different from the poly that you use.

How odd, I did exactly the same as you, re-downloaded Poly 1.2, clear analysis d.bin, made new.epd with the red, ran "Import EPD" and I get the attached output, did you get the same one change?

Then I loaded d.bin in ProDeo and the change is there.

Code: Select all

Opening Book : books\d.bin
Positions    : 381.111

Book Percent Score Depth Learn
d2d4  36.14%     0    0     0
e2e4  36.14%    35   20     0
g1f3  12.05%     0    0     0
c2c4  12.05%     0    0     0
b2b3   1.20%     0    0     0
g2g3   1.20%     0    0     0
f2f4   1.20%     0    0     0
b1c3   0.00%     0    0     0
b2b4   0.00%     0    0     0
e2e3   0.00%     0    0     0
d2d3   0.00%     0    0     0
a2a3   0.00%     0    0     0
g2g4   0.00%     0    0     0

[Ferdy]

BTW I tried to import a single epd to d.bin by Import Analysis.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 20; acd 22;
But after checking with Poly statistics, Analyzed Positions is 0. So this was not imported. So what is the requirement of the epd format that can be imported?

I checked, it is imported. When I run Poly stats a second time it also shows the correct counter.

I saved this epd
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 20; acd 22;
to 1.epd and import it. Still it is 0. Second time 0 too. Redownload poly and tried again still 0.

Analyze that 1.epd using poly and it outputs new.epd with the following format.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - id sf10x.exe; bm e2e4; ce 91; acd 20; c1 Threads 1, Hash 128, Time 2000ms;
Import that new.epd to d.bin, check with poly statistics and it is 0, tried again it is still 0.

Perhaps the uploaded poly (the one that I have) is different from the poly that you use.

How odd, I did exactly the same as you, re-downloaded Poly 1.2, clear analysis d.bin, made new.epd with the red, ran "Import EPD" and I get the attached output, did you get the same one change?

Then I loaded d.bin in ProDeo and the change is there.

Code: Select all

Opening Book : books\d.bin
Positions    : 381.111

Book Percent Score Depth Learn
d2d4  36.14%     0    0     0
e2e4  36.14%    35   20     0
g1f3  12.05%     0    0     0
c2c4  12.05%     0    0     0
b2b3   1.20%     0    0     0
g2g3   1.20%     0    0     0
f2f4   1.20%     0    0     0
b1c3   0.00%     0    0     0
b2b4   0.00%     0    0     0
e2e3   0.00%     0    0     0
d2d3   0.00%     0    0     0
a2a3   0.00%     0    0     0
g2g4   0.00%     0    0     0

All right I got the epd imported now when using "Import EPD".

Code: Select all

>> position startpos
string received: go wtime 300000 btime 300000 winc 100 binc 100
>> go wtime 300000 btime 300000 winc 100 binc 100
Move   Weight(%)  Score  Depth  Learn
d2d4       36.00   0.00      0      0
e2e4       36.00   0.36     20      0
g1f3       12.00   0.00      0      0
c2c4       12.00   0.00      0      0
b2b3        1.00   0.00      0      0
g2g3        1.00   0.00      0      0
f2f4        1.00   0.00      0      0
b1c3        0.00   0.00      0      0
b2b4        0.00   0.00      0      0
e2e3        0.00   0.00      0      0
d2d3        0.00   0.00      0      0
a2a3        0.00   0.00      0      0
g2g4        0.00   0.00      0      0
move e2e4 is selected having the best score!!
<< bestmove e2e4

But I had been using "Import Analysis". You may read back I clearly write it. Anyway this is solved now.

So the use of "Import Analysis" is when the epd is generated and analyzed by poly. In the example single epd we tried, it only worked on "Import EPD".

[Rebel]

But I had been using "Import Analysis". You may read back I clearly write it. Anyway this is solved now.

Yep, missed that.

So the use of "Import Analysis" is when the epd is generated and analyzed by poly. In the example single epd we tried, it only worked on "Import EPD".

For a better understanding you might consider to read the web-page again, the 3 functions Prepare, Analyze and Import belong together as a whole, 3 neccessary steps to identify existing book moves in any Polyglot book, then analyze them and last import them. The thing in this process is to remember the offset in the Polyglot book and it is stored with a po tag in the EPD so the import function knows where to store the SF analysis with SEEK_SET. And that's why Import did not work since there is no po Polyglot offset.

Import EPD doesn't have that problem.

[Ferdy]

A possible explanation can be that you haven't cleared the scores (the "Clear Analysis" function) and the "g2-g4" comes from a previous session, although I must say that a score of 0.31 remains suspect.

Maybe you can run it again with "Clear Analysis"?

Tried but still same result, I sent a PM with d.bin and new.epd download link.

Thanks.

When I load d.bin in ProDeo I get the same list except for g2-g4, not 0.31 (according to your list) but -0.69, see below. 31+69=100, maybe some sort of sign issue? -0.69 looks pretty correct compared to +0.31 for 1.g4

Code: Select all

Opening Book : books\d.bin
Positions    : 381.111

Book Percent Score Depth Learn    Move   Weight(%)  Score  Depth  Learn
d2d4  36.14%     3   20     0     d2d4       36.00   0.03     20      0
e2e4  36.14%     1   20     0     e2e4       36.00   0.02     20      0
g1f3  12.05%     0   20     0     g1f3       12.00  -0.00     20      0
c2c4  12.05%    -9   19     0     c2c4       12.00  -0.09     19      0
b2b3   1.20%    -5   19     0     b2b3        1.00  -0.05     19      0
g2g3   1.20%   -14   21     0     g2g3        1.00  -0.14     21      0
f2f4   1.20%   -21   19     0     f2f4        1.00  -0.21     19      0
b1c3   0.00%   -14   19     0     b1c3        0.00  -0.14     19      0
b2b4   0.00%   -45   19     0     b2b4        0.00  -0.45     19      0
e2e3   0.00%     0   21     0     e2e3        0.00   0.00     21      0
d2d3   0.00%   -21   19     0     d2d3        0.00  -0.21     19      0
a2a3   0.00%   -10   18     0     a2a3        0.00  -0.11     18      0
g2g4   0.00%   -69   18     0     g2g4        0.00   0.31     18      0

Thanks will recheck the formula I used.

I revised my formula, to show cp score from epd.
Example epd's that are imported to d.bin using "Import EPD" in poly.
filename: 1.epd

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm d2d4; ce 60; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 55; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm g1f3; ce 50; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm c2c4; ce 45; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm b2b3; ce 40; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm g2g3; ce 10; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm f2f4; ce 5; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm b1c3; ce 0; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm b2b4; ce -5; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e3; ce -75; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm d2d3; ce -80; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm a2a3; ce -100; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm g2g4; ce -120; acd 22;

After the import I get this.

Code: Select all

Move   Weight(%)  Score     CP  Depth  Learn
d2d4       36.14  15360     60     22      0
e2e4       36.14  14080     55     22      0
g1f3       12.05  12800     50     22      0
c2c4       12.05  11520     45     22      0
b2b3        1.20  10240     40     22      0
g2g3        1.20   2560     10     22      0
f2f4        1.20   1280      5     22      0
b1c3        0.00      0      0     22      0
b2b4        0.00  64511     -5     22      0
e2e3        0.00  46591    -75     22      0
d2d3        0.00  45311    -80     22      0
a2a3        0.00  40191   -100     22      0
g2g4        0.00  35071   -120     22      0

The formula for CP is.

Code: Select all

int entryScore = entries[i].score;
int cpScore = entryScore % 256 ? -1 * (1 + (65536 - entryScore)/256) : entryScore / 256;

The limitation of this formula is that it can only handle a maximum cp score of 255 and a minimum score of -256. Beyond those I don't know how to derive the cp from entry.score. So with only this, I will revise the epd with analysis and change the ce values there, if it is more than +255, set it to +255 and if it is less than -256, set it to -256, then import it to bin book.

With prodeo.bin I got.

Code: Select all

Move   Weight(%)  Score     CP  Depth  Learn
d2d4       51.60   4096     16     20     14
e2e4       41.96  12800     50     20     34
c2c4        4.13   3840     15     20     60
g1f3        2.31  64767     -4     20      0

After e2e4

Code: Select all

Move   Weight(%)  Score     CP  Depth  Learn
c7c5       38.85  44543    -83     20    251
e7e5       22.38  44031    -85     20    253
e7e6       15.20  49407    -64     20    237
c7c6        8.63  43263    -88     20      6
d7d6        5.71  41471    -95     20    242
d7d5        5.46  34815   -121     20      0
g8f6        3.77  31487   -134     20      0

[pedrox]

This?:

double s = entries.score >= 30000 ? (double)entries.score / 65536 - 1.0 : (double)entries.score / 65536;
cp = (int)(s*100);

Code: Select all

# Key: 463b96181691fc9c
# Random Weight (0-100): 39.37%
# Move  Weight  Score    CP     Depth   Learn
# ----  ------  -----    --     -----   -----
# d2d4  51.60%   4096       6   20       14
# e2e4  41.96%  12800      19   20       34
# c2c4   4.13%   3840       5   20       60
# g1f3   2.31%  64767      -1   20        0
0 0 0 0 (Book move)

e2e4

# Key: 823c9b50fd114196
# Random Weight (0-100): 58.08%
# Move  Weight  Score    CP     Depth   Learn
# ----  ------  -----    --     -----   -----
# c7c5  38.85%  44543     -32   20      251
# e7e5  22.38%  44031     -32   20      253
# e7e6  15.20%  49407     -24   20      237
# c7c6   8.63%  43263     -33   20        6
# d7d6   5.71%  41471     -36   20      242
# d7d5   5.46%  34815     -46   20        0
# g8f6   3.77%  31487     -51   20        0
0 0 0 0 (Book move)

[Ferdy]

This?:

double s = entries.score >= 30000 ? (double)entries.score / 65536 - 1.0 : (double)entries.score / 65536;
cp = (int)(s*100);

Code: Select all

# Key: 463b96181691fc9c
# Random Weight (0-100): 39.37%
# Move  Weight  Score    CP     Depth   Learn
# ----  ------  -----    --     -----   -----
# d2d4  51.60%   4096       6   20       14
# e2e4  41.96%  12800      19   20       34
# c2c4   4.13%   3840       5   20       60
# g1f3   2.31%  64767      -1   20        0
0 0 0 0 (Book move)

e2e4

# Key: 823c9b50fd114196
# Random Weight (0-100): 58.08%
# Move  Weight  Score    CP     Depth   Learn
# ----  ------  -----    --     -----   -----
# c7c5  38.85%  44543     -32   20      251
# e7e5  22.38%  44031     -32   20      253
# e7e6  15.20%  49407     -24   20      237
# c7c6   8.63%  43263     -33   20        6
# d7d6   5.71%  41471     -36   20      242
# d7d5   5.46%  34815     -46   20        0
# g8f6   3.77%  31487     -51   20        0
0 0 0 0 (Book move)

If I try your formula, it will not display the correct cp from epd analysis.

Here is the epd with ce. Import it to a book using Import EPD.

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm d2d4; ce 60; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 55; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm g1f3; ce 50; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm c2c4; ce 45; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm b2b3; ce 40; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm g2g3; ce 10; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm f2f4; ce 5; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm b1c3; ce 0; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm b2b4; ce -5; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e3; ce -75; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm d2d3; ce -80; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm a2a3; ce -100; acd 22;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm g2g4; ce -120; acd 22;

Here is the output using your formula.

Code: Select all

Move   Weight(%)  Score     CP  Depth  Learn
d2d4       36.14  15360     23     22      0
e2e4       36.14  14080     21     22      0
g1f3       12.05  12800     19     22      0
c2c4       12.05  11520     17     22      0
b2b3        1.20  10240     15     22      0
g2g3        1.20   2560      3     22      0
f2f4        1.20   1280      1     22      0
b1c3        0.00      0      0     22      0
b2b4        0.00  64511     -1     22      0
e2e3        0.00  46591    -28     22      0
d2d3        0.00  45311    -30     22      0
a2a3        0.00  40191    -38     22      0
g2g4        0.00  35071    -46     22      0

See the d2d4 move, based from epd, it should be 60 cp.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm d2d4; ce 60; acd 22;
But in the table, it is 23.

Code: Select all

Move   Weight(%)  Score     CP  Depth  Learn
d2d4       36.14  15360     23     22      0

With my new formula it will show the same value as in the ce in epd.

The formula that I posted couple of days before as reply to your post was incorrect.