A question regarding Crafty EPD analysis

Marc Lacrosse · Post by **Marc Lacrosse** » Mon Jul 28, 2008 2:59 pm

plattyaj wrote:But that looks like it is exactly correct. Side to move is black. The crafty analysis output shows a small negative (e.g. black winning) score and the epd shows the same score but reversed to be show positive for side on the move and negative against.

Perhaps you should try this position and verify that the epd score is extremely positive:!

6K1/3q4/4k3/8/8/8/8/8 b - - 0 1

Andy.

I cannot understand your comment for two reasons.

1. My example is simply the position with Black to play after white's 1.e4.
No engine on earth evaluates this position as superior for Black. You are wrong when you say : " The crafty analysis output shows a small negative (e.g. black winning) score ". It is exactly the opposite. In its onscreen output crafty follows EPD spec : it gives -0.21 for "black losing" and NOT "black winning" to follow your own terminology.

2. At the opposite the analysed EPD as saved on disk has "ce 21" and not "ce -21". First this is contradictory to the screen output. Secondly this does not conform with EPD spec.

Marc

plattyaj · Post by **plattyaj** » Mon Jul 28, 2008 3:32 pm

Sorry, I missed what the position was and didn't realize Crafty's analysis changed from the standard when in that mode. I was running regular analysis on clearly winning positions and seeing what I expected to see.

Andy.

bob · Post by **bob** » Mon Jul 28, 2008 6:59 pm

Marc Lacrosse wrote:Dear Bob

Sorry for the delay : I had no access to my PC during this week-end.

I confirm that crafty's epd output does not comply with the PGN specification with this precise version on my PC :

Here is my epd file (Test01.epd):

Code: Select all

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq -

Here is my rc file :

Code: Select all

st=30
mt=1
log off
learn off
hash=512M
hashp=64M

Here is crafty output during analysis :

Code: Select all

C:\ch\crafty221_ja>crafty221_ja.exe

Initializing multiple threads.
System is SMP, not NUMA.
EPD Kit revision date: 1996.04.21
unable to open book file [./book.bin].
book is disabled
unable to open book file [./books.bin].
search time set to 30.00.
max threads set to 1
book learning disabled
result learning disabled
hash table memory =  384M bytes.
pawn hash table memory =   40M bytes.

Crafty v22.1 JA (1 cpus)

White(1): epdpfga test01.epd test01-result.epd
PFGA: EPD record: 1
              time surplus   0.00  time limit 30.00 (+0.00) (30.00)
              depth   time  score   variation (1)
               11     0.38  -0.20   1. ... d5 2. Nf3 Nc6 3. d4 Bf5 4. Nh4
                                    Bd7 5. Nc3 Nf6 6. Nf3 Bf5
               11->   0.41  -0.20   1. ... d5 2. Nf3 Nc6 3. d4 Bf5 4. Nh4
                                    Bd7 5. Nc3 Nf6 6. Nf3 Bf5
               12     0.97  -0.22   1. ... d5 2. Nf3 Nc6 3. d4 Qd6 4. Nc3
                                    Nf6 5. Bd2 e5 6. Nb5 Qe7 7. e3 <HT>
               12->   1.20  -0.22   1. ... d5 2. Nf3 Nc6 3. d4 Qd6 4. Nc3
                                    Nf6 5. Bd2 e5 6. Nb5 Qe7 7. e3 <HT>
               13     3.03  -0.11   1. ... d5 2. d4 Nc6 3. Bf4 Nf6 4. e3
                                    Bg4 5. Be2 Bf5 6. Nf3 Nb4 7. Na3 e6
               13     5.22  -0.25   1. ... e5 2. Nf3 e4 3. Nd4 Nc6 4. Nxc6
                                    dxc6 5. Nc3 Nf6 6. d3 Bc5 7. dxe4 Qxd1+
                                    8. Kxd1 Bxf2
               13->   5.28  -0.25   1. ... e5 2. Nf3 e4 3. Nd4 Nc6 4. Nxc6
                                    dxc6 5. Nc3 Nf6 6. d3 Bc5 7. dxe4 Qxd1+
                                    8. Kxd1 Bxf2
               14    11.63  -0.23   1. ... e5 2. Nf3 e4 3. Nd4 c5 4. Nb5
                                    d5 5. c4 d4 6. d3 Nf6 7. dxe4 Nxe4
                                    8. Nd2
               14->  12.38  -0.23   1. ... e5 2. Nf3 e4 3. Nd4 c5 4. Nb5
                                    d5 5. c4 d4 6. d3 Nf6 7. dxe4 Nxe4
                                    8. Nd2
               15    25.35  -0.21   1. ... e5 2. e4 Nc6 3. Nc3 Nf6 4. Nf3
                                    d5 5. exd5 Nxd5 6. Bb5 Qd6 7. Nxd5
                                    Qxd5 8. Bxc6+ Qxc6 9. Nxe5 Qxg2
              time=30.14  mat=0  n=22912571  fh=91%  nps=760K
              ext-> check=786K 1rep=51K mate=533 pp=0 reduce=13.4M/1.8M
              predicted=0  evals=18.0M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/512  elap=30.14
Black(1):

Please note that final evaluation is negative (it is a relative one) : -0.21

And now the resulting epd :

Code: Select all

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - acd 15; acn 22912571; acs 30; ce 21; pv e5 e4 Nc6 Nc3 Nf6 Nf3 d5 exd5 Nxd5 Bb5 Qd6 Nxd5 Qxd5 Bxc6+ Qxc6 Nxe5 Qxg2;

Here the final evaluation is positive : "ce 21".

This is _not_ epd-specification-compliant.

And there is an evident contradiction between crafty output to standard-out during analysis and the recording with opposite sign in the resulting epd file.

Marc

That looks correct to me. In the search, crafty's evaluation is -.21 which is "good for black". In the EPD output, the evaluation is +.21, which says "side on move is better".

How is that wrong?

Here's the important stuff from above:

15 25.35 -0.21 1. ... e5 2. e4 Nc6 3. Nc3 Nf6 4. Nf3
d5 5. exd5 Nxd5 6. Bb5 Qd6 7. Nxd5
Qxd5 8. Bxc6+ Qxc6 9. Nxe5 Qxg2

Which clearly says "black is better" and then:

code]rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - acd 15; acn 22912571; acs 30; ce 21; pv e5 e4 Nc6 Nc3 Nf6 Nf3 d5 exd5 Nxd5 Bb5 Qd6 Nxd5 Qxd5 Bxc6+ Qxc6 Nxe5 Qxg2;
[/code]

which clearly says "side on move (black according to the EPD record) is better (ce +21).

bob · Post by **bob** » Mon Jul 28, 2008 7:00 pm

Marc Lacrosse wrote:Dear Bob

Sorry for the delay : I had no access to my PC during this week-end.

I confirm that crafty's epd output does not comply with the PGN specification with this precise version on my PC :

Here is my epd file (Test01.epd):

Code: Select all

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq -

Here is my rc file :

Code: Select all

st=30
mt=1
log off
learn off
hash=512M
hashp=64M

Here is crafty output during analysis :

Code: Select all

C:\ch\crafty221_ja>crafty221_ja.exe

Initializing multiple threads.
System is SMP, not NUMA.
EPD Kit revision date: 1996.04.21
unable to open book file [./book.bin].
book is disabled
unable to open book file [./books.bin].
search time set to 30.00.
max threads set to 1
book learning disabled
result learning disabled
hash table memory =  384M bytes.
pawn hash table memory =   40M bytes.

Crafty v22.1 JA (1 cpus)

White(1): epdpfga test01.epd test01-result.epd
PFGA: EPD record: 1
              time surplus   0.00  time limit 30.00 (+0.00) (30.00)
              depth   time  score   variation (1)
               11     0.38  -0.20   1. ... d5 2. Nf3 Nc6 3. d4 Bf5 4. Nh4
                                    Bd7 5. Nc3 Nf6 6. Nf3 Bf5
               11->   0.41  -0.20   1. ... d5 2. Nf3 Nc6 3. d4 Bf5 4. Nh4
                                    Bd7 5. Nc3 Nf6 6. Nf3 Bf5
               12     0.97  -0.22   1. ... d5 2. Nf3 Nc6 3. d4 Qd6 4. Nc3
                                    Nf6 5. Bd2 e5 6. Nb5 Qe7 7. e3 <HT>
               12->   1.20  -0.22   1. ... d5 2. Nf3 Nc6 3. d4 Qd6 4. Nc3
                                    Nf6 5. Bd2 e5 6. Nb5 Qe7 7. e3 <HT>
               13     3.03  -0.11   1. ... d5 2. d4 Nc6 3. Bf4 Nf6 4. e3
                                    Bg4 5. Be2 Bf5 6. Nf3 Nb4 7. Na3 e6
               13     5.22  -0.25   1. ... e5 2. Nf3 e4 3. Nd4 Nc6 4. Nxc6
                                    dxc6 5. Nc3 Nf6 6. d3 Bc5 7. dxe4 Qxd1+
                                    8. Kxd1 Bxf2
               13->   5.28  -0.25   1. ... e5 2. Nf3 e4 3. Nd4 Nc6 4. Nxc6
                                    dxc6 5. Nc3 Nf6 6. d3 Bc5 7. dxe4 Qxd1+
                                    8. Kxd1 Bxf2
               14    11.63  -0.23   1. ... e5 2. Nf3 e4 3. Nd4 c5 4. Nb5
                                    d5 5. c4 d4 6. d3 Nf6 7. dxe4 Nxe4
                                    8. Nd2
               14->  12.38  -0.23   1. ... e5 2. Nf3 e4 3. Nd4 c5 4. Nb5
                                    d5 5. c4 d4 6. d3 Nf6 7. dxe4 Nxe4
                                    8. Nd2
               15    25.35  -0.21   1. ... e5 2. e4 Nc6 3. Nc3 Nf6 4. Nf3
                                    d5 5. exd5 Nxd5 6. Bb5 Qd6 7. Nxd5
                                    Qxd5 8. Bxc6+ Qxc6 9. Nxe5 Qxg2
              time=30.14  mat=0  n=22912571  fh=91%  nps=760K
              ext-> check=786K 1rep=51K mate=533 pp=0 reduce=13.4M/1.8M
              predicted=0  evals=18.0M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/512  elap=30.14
Black(1):

Please note that final evaluation is negative (it is a relative one) : -0.21

And now the resulting epd :

Code: Select all

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - acd 15; acn 22912571; acs 30; ce 21; pv e5 e4 Nc6 Nc3 Nf6 Nf3 d5 exd5 Nxd5 Bb5 Qd6 Nxd5 Qxd5 Bxc6+ Qxc6 Nxe5 Qxg2;

Here the final evaluation is positive : "ce 21".

This is _not_ epd-specification-compliant.

And there is an evident contradiction between crafty output to standard-out during analysis and the recording with opposite sign in the resulting epd file.

Marc

That looks correct to me. In the search, crafty's evaluation is -.21 which is "good for black". In the EPD output, the evaluation is +.21, which says "side on move is better".

How is that wrong?

Here's the important stuff from above:

15 25.35 -0.21 1. ... e5 2. e4 Nc6 3. Nc3 Nf6 4. Nf3
d5 5. exd5 Nxd5 6. Bb5 Qd6 7. Nxd5
Qxd5 8. Bxc6+ Qxc6 9. Nxe5 Qxg2

Which clearly says "black is better" and then:

code]rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - acd 15; acn 22912571; acs 30; ce 21; pv e5 e4 Nc6 Nc3 Nf6 Nf3 d5 exd5 Nxd5 Bb5 Qd6 Nxd5 Qxd5 Bxc6+ Qxc6 Nxe5 Qxg2;
[/code]

which clearly says "side on move (black according to the EPD record) is better (ce +21).

BTW I agree that -.21 would be better in the EPD.

Then it would always be "score from white's perspective" but that isn't the standard.

bob · Post by **bob** » Mon Jul 28, 2008 7:02 pm

Marc Lacrosse wrote:
plattyaj wrote:But that looks like it is exactly correct. Side to move is black. The crafty analysis output shows a small negative (e.g. black winning) score and the epd shows the same score but reversed to be show positive for side on the move and negative against.

Perhaps you should try this position and verify that the epd score is extremely positive:!

6K1/3q4/4k3/8/8/8/8/8 b - - 0 1

Andy.
I cannot understand your comment for two reasons.

1. My example is simply the position with Black to play after white's 1.e4.
No engine on earth evaluates this position as superior for Black. You are wrong when you say : " The crafty analysis output shows a small negative (e.g. black winning) score ". It is exactly the opposite. In its onscreen output crafty follows EPD spec : it gives -0.21 for "black losing" and NOT "black winning" to follow your own terminology.

2. At the opposite the analysed EPD as saved on disk has "ce 21" and not "ce -21". First this is contradictory to the screen output. Secondly this does not conform with EPD spec.

Marc

on screen -.21 means "black is better" not "black is worse". Why it would prefer black over white is unknown there, but that is a different issue...

Nick C · Post by **Nick C** » Mon Jul 28, 2008 9:22 pm

After 1.e4 the analyzed EPD string should have a negative value for ce, because it's blacks turn to move. I know it sucks, it means you have to sort the games you want to analyze by "white to move" and "black to move", whereas if the EPD spec was absolute, this step would not be necessary. In any case, if what Marc says is true, and that an EPD analyzed by crafty has a non-negative ce after the first move, then crafty is non-compliant.

bob · Post by **bob** » Mon Jul 28, 2008 11:14 pm

Nick C wrote:After 1.e4 the analyzed EPD string should have a negative value for ce, because it's blacks turn to move. I know it sucks, it means you have to sort the games you want to analyze by "white to move" and "black to move", whereas if the EPD spec was absolute, this step would not be necessary. In any case, if what Marc says is true, and that an EPD analyzed by crafty has a non-negative ce after the first move, then crafty is non-compliant.

Did you look at the output? Crafty thinks black is better. So how can it be non-compliant unless the EPD standard says that after one move by each side, the centipawn evaluation _must_ be good for white or bad for black.

The current version does not think this:

search output first:

Code: Select all

               12->   3.96   0.19   1. ... e5 2. Nc3 Nc6 3. Nf3 Nf6 4.
                                    d4 exd4 5. Nxd4 Bb4 6. Qd3 O-O 7. Bf4
                                    Bxc3+ 8. bxc3
               13     4.93   0.19   1. ... e5 2. Nc3 Nc6 3. Nf3 Nf6 4.
                                    d4 exd4 5. Nxd4 Bb4 6. Qd3 O-O 7. Bf4
                                    Bxc3+ 8. bxc3

So +.19, good for white, bad for black.

epd output produced by "epdpfga xx.epd xx.out":

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - acd 13; acn 7610873; acs 5; ce -19; pv e5 Nc3 Nc6 Nf3 Nf6 d4 exd4 Nxd4 Bb4 Qd3 O-O Bf4 Bxc3+ bxc3;

EPD record says -19, bad for black, which is good for white.

this is also compliant.

Marc Lacrosse · Post by **Marc Lacrosse** » Tue Jul 29, 2008 12:12 am

bob wrote: Did you look at the output? Crafty thinks black is better. So how can it be non-compliant unless the EPD standard says that after one move by each side, the centipawn evaluation _must_ be good for white or bad for black.

The current version does not think this:

search output first:
12-> 3.96 0.19 1. ... e5 2. Nc3 Nc6 3. Nf3 Nf6 4.
d4 exd4 5. Nxd4 Bb4 6. Qd3 O-O 7. Bf4
Bxc3+ 8. bxc3
13 4.93 0.19 1. ... e5 2. Nc3 Nc6 3. Nf3 Nf6 4.
d4 exd4 5. Nxd4 Bb4 6. Qd3 O-O 7. Bf4
Bxc3+ 8. bxc3

So +.19, good for white, bad for black.

This is OK for me although not compliant with epd spec.

bob wrote: epd output produced by "epdpfga xx.epd xx.out":

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - acd 13; acn 7610873; acs 5; ce -19; pv e5 Nc3 Nc6 Nf3 Nf6 d4 exd4 Nxd4 Bb4 Qd3 O-O Bf4 Bxc3+ bxc3;

EPD record says -19, bad for black, which is good for white.

Yes but :

1. This is not what I get (I get +21 : black better and not -19 as you say now)

2. _You_ seemed to agree with opposite numbers in this very same thread when you said in your comment (dated Jul 28 4:59 pm):
"rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - acd 15; acn 22912571; acs 30; ce 21 (...) which clearly says side on move (black according to the EPD record) is better (ce +21)."

So for the same position analysed by crafty and saved to epd you first say :
"ce 21 (...) which clearly says side on move (black according to the EPD record) is better (ce +21)"
And less than five hours later you now say :
"EPD record says -19, bad for black, which is good for white. "

OK Bob let's stop here...

I won't discuss this topic any more.
Either i am crazy or you need some rest.

Marc

Nick C · Post by **Nick C** » Tue Jul 29, 2008 12:25 am

Did you look at the output? Crafty thinks black is better.

I do not understand how any version of crafty could think that black is better after 1.e4.

bob · Post by **bob** » Tue Jul 29, 2008 1:08 am

Somehow we are not communicating. When crafty searches, it produces scores from white's perspective no matter which side it is playing. If it shows -.19, it thinks black is better. But that isn't EPD compliant, I agree. But crafty's normal output has _never_ been epd compliant because there is no EPD specification for program output.

But for the epd record output, crafty's score was +.19, which is from black's perspective.

If you consider that in a normal output, -.19 means "black is better by 19 centipawns, and you look at the epd record output of +19, which says "side on move (which just happens to be black) is better by 19 centipawns, this seems to be perfectly compliant with EPD, since the score _is_ "+ is good for the side on move, - is bad for the side on move.

One says black is better, the other says black is better, so I am not sure why anyone thinks it is wrong???

Now if the search output has said +.19, and the EPD output said +19, then I agree it would be wrong, because +.19 from the search says white is better, and +19 in the EPD record says "side on move (black) is better." But I am not seeing that. Nor the inverse where the search says -.19, and the EPD says -19, which would be reversed as well...

A question regarding Crafty EPD analysis

Re: A question regarding Crafty EPD analysis

Re: A question regarding Crafty EPD analysis

Re: A question regarding Crafty EPD analysis

Re: A question regarding Crafty EPD analysis

Re: A question regarding Crafty EPD analysis

Re: A question regarding Crafty EPD analysis

Re: A question regarding Crafty EPD analysis

Re: A question regarding Crafty EPD analysis

Re: A question regarding Crafty EPD analysis

Re: A question regarding Crafty EPD analysis