Polyglot extension tool

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Polyglot extension tool

Post by Ferdy »

Rebel wrote: Fri Mar 08, 2019 8:25 pm
Ferdy wrote: Fri Mar 08, 2019 11:00 am
Rebel wrote: Fri Mar 08, 2019 10:17 am A possible explanation can be that you haven't cleared the scores (the "Clear Analysis" function) and the "g2-g4" comes from a previous session, although I must say that a score of 0.31 remains suspect.

Maybe you can run it again with "Clear Analysis"?
Tried but still same result, I sent a PM with d.bin and new.epd download link.
Thanks.

When I load d.bin in ProDeo I get the same list except for g2-g4, not 0.31 (according to your list) but -0.69, see below. 31+69=100, maybe some sort of sign issue? -0.69 looks pretty correct compared to +0.31 for 1.g4

Code: Select all

Opening Book : books\d.bin
Positions    : 381.111

Book Percent Score Depth Learn    Move   Weight(%)  Score  Depth  Learn
d2d4  36.14%     3   20     0     d2d4       36.00   0.03     20      0
e2e4  36.14%     1   20     0     e2e4       36.00   0.02     20      0
g1f3  12.05%     0   20     0     g1f3       12.00  -0.00     20      0
c2c4  12.05%    -9   19     0     c2c4       12.00  -0.09     19      0
b2b3   1.20%    -5   19     0     b2b3        1.00  -0.05     19      0
g2g3   1.20%   -14   21     0     g2g3        1.00  -0.14     21      0
f2f4   1.20%   -21   19     0     f2f4        1.00  -0.21     19      0
b1c3   0.00%   -14   19     0     b1c3        0.00  -0.14     19      0
b2b4   0.00%   -45   19     0     b2b4        0.00  -0.45     19      0
e2e3   0.00%     0   21     0     e2e3        0.00   0.00     21      0
d2d3   0.00%   -21   19     0     d2d3        0.00  -0.21     19      0
a2a3   0.00%   -10   18     0     a2a3        0.00  -0.11     18      0
g2g4   0.00%   -69   18     0     g2g4        0.00   0.31     18      0
BTW I tried to import a single epd to d.bin by Import Analysis.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e2e4; ce 20; acd 22;
But after checking with Poly statistics, Analyzed Positions is 0. So this was not imported. So what is the requirement of the epd format that can be imported?
I will look into that later, have little time at the moment.
Instead of importing via "Import Analysis", I use "Import EPD" to d.bin. The imported file is new.epd (this is the output from poly "Analyze").

Here is the table.

Code: Select all

Move   Weight(%)  Score     CP  Depth  Learn
d2d4       36.14   9984     39     19      0
e2e4       36.14  19968     78     18      0
g1f3       12.05      0      0      0      0
c2c4       12.05      0      0      0      0
b2b3        1.20      0      0      0      0
g2g3        1.20      0      0      0      0
f2f4        1.20      0      0      0      0
b1c3        0.00      0      0      0      0
b2b4        0.00      0      0      0      0
e2e3        0.00      0      0      0      0
d2d3        0.00      0      0      0      0
a2a3        0.00      0      0      0      0
g2g4        0.00      0      0      0      0
This is consistent with the contains of the new.epd file. That new.epd has

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - po 5662400; id sf10x.exe; bm d2d4; ce 39; acd 19; c1 Threads 1, Hash 256, Time 2000ms;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - po 710928; id sf10x.exe; bm e2e4; ce 78; acd 18; c1 Threads 1, Hash 256, Time 2000ms;
Notice the CP in the table and the ce in the epd, they are the same.

I wonder why when using "Import Analysis" importing new.epd to d.bin, some moves have score (even if new.epd don't have it) as in the following table.

Code: Select all

Opening Book : books\d.bin
Positions    : 381.111

Book Percent Score Depth Learn    Move   Weight(%)  Score  Depth  Learn
d2d4  36.14%     3   20     0     d2d4       36.00   0.03     20      0
e2e4  36.14%     1   20     0     e2e4       36.00   0.02     20      0
g1f3  12.05%     0   20     0     g1f3       12.00  -0.00     20      0
c2c4  12.05%    -9   19     0     c2c4       12.00  -0.09     19      0
b2b3   1.20%    -5   19     0     b2b3        1.00  -0.05     19      0
g2g3   1.20%   -14   21     0     g2g3        1.00  -0.14     21      0
f2f4   1.20%   -21   19     0     f2f4        1.00  -0.21     19      0
b1c3   0.00%   -14   19     0     b1c3        0.00  -0.14     19      0
b2b4   0.00%   -45   19     0     b2b4        0.00  -0.45     19      0
e2e3   0.00%     0   21     0     e2e3        0.00   0.00     21      0
d2d3   0.00%   -21   19     0     d2d3        0.00  -0.21     19      0
a2a3   0.00%   -10   18     0     a2a3        0.00  -0.11     18      0
g2g4   0.00%   -69   18     0     g2g4        0.00   0.31     18      0
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Polyglot extension tool

Post by Rebel »

Ferdy wrote: Sun Mar 10, 2019 8:32 am This is consistent with the contains of the new.epd file. That new.epd has

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - po 5662400; id sf10x.exe; bm d2d4; ce 39; acd 19; c1 Threads 1, Hash 256, Time 2000ms;
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - po 710928; id sf10x.exe; bm e2e4; ce 78; acd 18; c1 Threads 1, Hash 256, Time 2000ms;
Notice the CP in the table and the ce in the epd, they are the same.
Finally someone who notices :wink:

As a SF-source-code-layman I assumed the "ce" high scores in the opening were binary and the scores SF10 produced in ChessPartner confirmed my wild guess and so while displaying the score in ProDeo I divide the scores by 2.56. Later I found out this was wrong and "ce" is truly decimal and I decided to leave it that way because the narrower scores fit better when tuning the book with the "Vary" option.
I wonder why when using "Import Analysis" importing new.epd to d.bin, some moves have score (even if new.epd don't have it) as in the following table.
Well, that's because as long as you don't clear the analysis of previous sessions the results stay.
90% of coding is debugging, the other 10% is writing bugs.
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Polyglot extension tool

Post by Ferdy »

Rebel wrote: Sun Mar 10, 2019 9:01 pm
I wonder why when using "Import Analysis" importing new.epd to d.bin, some moves have score (even if new.epd don't have it) as in the following table.
Well, that's because as long as you don't clear the analysis of previous sessions the results stay.
Cleared it but same thing happened.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Polyglot extension tool

Post by Rebel »

Ferdy wrote: Mon Mar 11, 2019 3:12 am
Rebel wrote: Sun Mar 10, 2019 9:01 pm
I wonder why when using "Import Analysis" importing new.epd to d.bin, some moves have score (even if new.epd don't have it) as in the following table.
Well, that's because as long as you don't clear the analysis of previous sessions the results stay.
Cleared it but same thing happened.
Are you sure that you used "Clear Analysis" (option 2) and not "Clear Learning" (option 1) and checked if the counters are all zero with " Poly statistics" (option 3) ?

Else maybe you can mail me an example.

Can't imagine it, but bugs are the inevitable result of programming, it's something you learn by experience, but then it's already too late, you are adicted and can't stop.
90% of coding is debugging, the other 10% is writing bugs.
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Polyglot extension tool

Post by Ferdy »

Rebel wrote: Mon Mar 11, 2019 7:24 am
Ferdy wrote: Mon Mar 11, 2019 3:12 am
Rebel wrote: Sun Mar 10, 2019 9:01 pm
I wonder why when using "Import Analysis" importing new.epd to d.bin, some moves have score (even if new.epd don't have it) as in the following table.
Well, that's because as long as you don't clear the analysis of previous sessions the results stay.
Cleared it but same thing happened.
Are you sure that you used "Clear Analysis" (option 2) and not "Clear Learning" (option 1) and checked if the counters are all zero with " Poly statistics" (option 3) ?
Yes I use Clear Analysis and checked with Poly statistics Analyzed positions are zero.
Else maybe you can mail me an example.

Can't imagine it, but bugs are the inevitable result of programming, it's something you learn by experience, but then it's already too late, you are adicted and can't stop.
The link to the d.bin and new.epd I sent to you before via PM is what I use. You can still download it.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Polyglot extension tool

Post by Rebel »

Ferdy wrote: Sun Mar 10, 2019 8:32 am
I wonder why when using "Import Analysis" importing new.epd to d.bin, some moves have score (even if new.epd don't have it) as in the following table.

Code: Select all

Opening Book : books\d.bin
Positions    : 381.111

Book Percent Score Depth Learn    Move   Weight(%)  Score  Depth  Learn
d2d4  36.14%     3   20     0     d2d4       36.00   0.03     20      0
e2e4  36.14%     1   20     0     e2e4       36.00   0.02     20      0
g1f3  12.05%     0   20     0     g1f3       12.00  -0.00     20      0
c2c4  12.05%    -9   19     0     c2c4       12.00  -0.09     19      0
b2b3   1.20%    -5   19     0     b2b3        1.00  -0.05     19      0
g2g3   1.20%   -14   21     0     g2g3        1.00  -0.14     21      0
f2f4   1.20%   -21   19     0     f2f4        1.00  -0.21     19      0
b1c3   0.00%   -14   19     0     b1c3        0.00  -0.14     19      0
b2b4   0.00%   -45   19     0     b2b4        0.00  -0.45     19      0
e2e3   0.00%     0   21     0     e2e3        0.00   0.00     21      0
d2d3   0.00%   -21   19     0     d2d3        0.00  -0.21     19      0
a2a3   0.00%   -10   18     0     a2a3        0.00  -0.11     18      0
g2g4   0.00%   -69   18     0     g2g4        0.00   0.31     18      0
Ok, back to this, which move according to you has a score not in new.epd?

If you mean "g2g4", it's there, precisely in line 77573 -

rnbqkbnr/pppppppp/8/8/6P1/8/PPPPPP1P/RNBQKBNR b KQkq - po 1674896; id sf10x.exe; bm d7d5; ce 178; acd 18; c1 Threads 1, Hash 256, Time 2000ms;

Position after 1.g4 is analysed for black, score for black 178 and thus for white -178. -178/2.56 = -69, see the ProDeo score. And "po" is the offset in the Polyglot book where -69 and the depth are stored.
90% of coding is debugging, the other 10% is writing bugs.
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Polyglot extension tool

Post by Ferdy »

Rebel wrote: Mon Mar 11, 2019 11:01 am
Ferdy wrote: Sun Mar 10, 2019 8:32 am
I wonder why when using "Import Analysis" importing new.epd to d.bin, some moves have score (even if new.epd don't have it) as in the following table.

Code: Select all

Opening Book : books\d.bin
Positions    : 381.111

Book Percent Score Depth Learn    Move   Weight(%)  Score  Depth  Learn
d2d4  36.14%     3   20     0     d2d4       36.00   0.03     20      0
e2e4  36.14%     1   20     0     e2e4       36.00   0.02     20      0
g1f3  12.05%     0   20     0     g1f3       12.00  -0.00     20      0
c2c4  12.05%    -9   19     0     c2c4       12.00  -0.09     19      0
b2b3   1.20%    -5   19     0     b2b3        1.00  -0.05     19      0
g2g3   1.20%   -14   21     0     g2g3        1.00  -0.14     21      0
f2f4   1.20%   -21   19     0     f2f4        1.00  -0.21     19      0
b1c3   0.00%   -14   19     0     b1c3        0.00  -0.14     19      0
b2b4   0.00%   -45   19     0     b2b4        0.00  -0.45     19      0
e2e3   0.00%     0   21     0     e2e3        0.00   0.00     21      0
d2d3   0.00%   -21   19     0     d2d3        0.00  -0.21     19      0
a2a3   0.00%   -10   18     0     a2a3        0.00  -0.11     18      0
g2g4   0.00%   -69   18     0     g2g4        0.00   0.31     18      0
Ok, back to this, which move according to you has a score not in new.epd?

If you mean "g2g4", it's there, precisely in line 77573 -

rnbqkbnr/pppppppp/8/8/6P1/8/PPPPPP1P/RNBQKBNR b KQkq - po 1674896; id sf10x.exe; bm d7d5; ce 178; acd 18; c1 Threads 1, Hash 256, Time 2000ms;

Position after 1.g4 is analysed for black, score for black 178 and thus for white -178. -178/2.56 = -69, see the ProDeo score. And "po" is the offset in the Polyglot book where -69 and the depth are stored.
All right that explains it. Thanks.