A question regarding Crafty EPD analysis

[Marc Lacrosse]

I just had a look at crafty 22.1 (Jim Ablett'compile).
I tried the "epdpfga" function (automated epd analysis).

For this I used a very simple epd file with just one position, the one that occurs after 1.e4 :
rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq -

In the resulting analysed epd file I see "ce 21" which means an evaluation of this position as 21 centipawns (=0.21).

My problem is that I suppose that it should be "-0.21" and not "0.21".

According to EPD specification (I just have 1995.11.26 version here) the ce opcode "takes a single operand, an optionally signed integer that gives an evaluation of the position from the viewpoint of the active player; i.e., the player with the move. Positive values indicate a position favorable to the moving player while negative values indicate a position favorable to the passive player" (chapter 6.7).

So I suppose that after 1.e4 one should read "ce -0.21" and not "ce 21", the position being slightly favorable to the opponent of the player who has to move.

This is annoying because one of the programs into which I import these EPD files needs a pefectly formed epd string and does interpret this epd file at the opposite of what it should be.

So, although general trend is to use "absolute" evaluations (and not anymore relative ones as in the epd spec) i think it would be nice to have a switch in crafty rc file allowing to switch back to relative evaluations when epd analysis is performed.

Your opinion ?

Marc

[bob]

I just had a look at crafty 22.1 (Jim Ablett'compile).
I tried the "epdpfga" function (automated epd analysis).

For this I used a very simple epd file with just one position, the one that occurs after 1.e4 :
rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq -

In the resulting analysed epd file I see "ce 21" which means an evaluation of this position as 21 centipawns (=0.21).

My problem is that I suppose that it should be "-0.21" and not "0.21".

According to EPD specification (I just have 1995.11.26 version here) the ce opcode "takes a single operand, an optionally signed integer that gives an evaluation of the position from the viewpoint of the active player; i.e., the player with the move. Positive values indicate a position favorable to the moving player while negative values indicate a position favorable to the passive player" (chapter 6.7).

So I suppose that after 1.e4 one should read "ce -0.21" and not "ce 21", the position being slightly favorable to the opponent of the player who has to move.

This is annoying because one of the programs into which I import these EPD files needs a pefectly formed epd string and does interpret this epd file at the opposite of what it should be.

So, although general trend is to use "absolute" evaluations (and not anymore relative ones as in the epd spec) i think it would be nice to have a switch in crafty rc file allowing to switch back to relative evaluations when epd analysis is performed.

Your opinion ?

Marc

Interesting issue. However, the idea is somewhat problematic. I personally believe that it is far easier/clearer to give evaluations from the white perspective. In every book I own, +/- means white is winning, not the side on move. The EPD specification unfortunately predates that "revelation"...

I think the right way to deal with this would be to modify the spec. If you see a position where it is black to move, and you see a + evaluation, you have to investigate and ask "OK, is black winning according to that or is white winning? You have to look at the position and see who is on move. Then you have to think "OK, is this evaluation after the move or before the move, etc. In short, I always found it confusing, and that's why I "went against the grain" back in 1995 or so and started the evaluation from white's perspective always approach. It eliminated tons of questions on ICC when I had crafty providing analysis for in-progress GM games, and the feedback was almost universally positive when I made the change...

Wouldn't it be ugly if your bank statement showed some numbers from your perspective, and some from the bank's perspective, so that you never know whether - is good or bad for your financial status? IMHO a standard is always better than ad hoc in such cases...

[Marc Lacrosse]

I completely agree Bob : "absolute" signing of evaluation numbers (positive : white is better; negative : black is better) is much easier to use, more "natural" and this has become the de facto standard for most modern programs.
The problem is that the other way of proceeding is clearly described as being part not only of the epd file specification but also of the PGN spec.
Sure, protocols are there to be improved.
I would surely agree for such a modification.
BUT my problem is : a whole bunch of programs and engines have been built using the old specification. And I still like using some of them (an interesting example is the bookup database which is much more powerful than its poor reputation and poor interface would let guess).
So I just ask you this small addition to your present crafty parameters : please do create an additional parameter for crafty RC files so that when I add "relative position evaluations = true" I will get negative values in a epd analysis when crafty as black thinks he is worse than white.

Marc

[michiguel]

I completely agree Bob : "absolute" signing of evaluation numbers (positive : white is better; negative : black is better) is much easier to use, more "natural" and this has become the de facto standard for most modern programs.
The problem is that the other way of proceeding is clearly described as being part not only of the epd file specification but also of the PGN spec.
Sure, protocols are there to be improved.
I would surely agree for such a modification.
BUT my problem is : a whole bunch of programs and engines have been built using the old specification. And I still like using some of them (an interesting example is the bookup database which is much more powerful than its poor reputation and poor interface would let guess).
So I just ask you this small addition to your present crafty parameters : please do create an additional parameter for crafty RC files so that when I add "relative position evaluations = true" I will get negative values in a epd analysis when crafty as black thinks he is worse than white.

Marc

EPD is not really meant to be read by a human player as a first priority, but its intention is to allow a portable (text) way of communication between chess programs. Bob's philosophy makes sense but I do not see a reason to depart from the EPD standard in the EPD output/input. It won't confuse anybody to have an authentic EPD output (we are not talking about what the user see in the GUI). Otherwise, the interaction between Crafty and other chess tools may be broken.

Miguel

[bob]

I completely agree Bob : "absolute" signing of evaluation numbers (positive : white is better; negative : black is better) is much easier to use, more "natural" and this has become the de facto standard for most modern programs.
The problem is that the other way of proceeding is clearly described as being part not only of the epd file specification but also of the PGN spec.
Sure, protocols are there to be improved.
I would surely agree for such a modification.
BUT my problem is : a whole bunch of programs and engines have been built using the old specification. And I still like using some of them (an interesting example is the bookup database which is much more powerful than its poor reputation and poor interface would let guess).
So I just ask you this small addition to your present crafty parameters : please do create an additional parameter for crafty RC files so that when I add "relative position evaluations = true" I will get negative values in a epd analysis when crafty as black thinks he is worse than white.

Marc

I'll see what I can do... Doesn't sound like it should be too terribly difficult...

[bob]

OK, can you give me a couple of positions (EPD) where you think it is wrong? I just checked and apparently it is already doing the +=good for side to move in the epdpfga output file so far as I can tell...

[Marc Lacrosse]

OK, can you give me a couple of positions (EPD) where you think it is wrong? I just checked and apparently it is already doing the +=good for side to move in the epdpfga output file so far as I can tell...

Just this simple one (black to move after 1.e4): I repeatedly get positive scores. I cannot imagine that black feels better than white here...

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq -

[bob]

I pasted that into a file xx.epd, then used the following command:

epdpfga xx.epd xx.out

and got this in xx.out:

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - acd 12; acn 3176742; acs 3; ce -30; pv d5 exd5 Qxd5 Nc3 Qe6+ Nge2 Nf6 d4 Qb6 Nf4 Nc6 Nb5;

[Marc Lacrosse]

Dear Bob

Sorry for the delay : I had no access to my PC during this week-end.

I confirm that crafty's epd output does not comply with the PGN specification with this precise version on my PC :

Here is my epd file (Test01.epd):

Code: Select all

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - 

Here is my rc file :

Code: Select all

st=30
mt=1
log off
learn off
hash=512M
hashp=64M

Here is crafty output during analysis :

Code: Select all

C:\ch\crafty221_ja>crafty221_ja.exe

Initializing multiple threads.
System is SMP, not NUMA.
EPD Kit revision date: 1996.04.21
unable to open book file [./book.bin].
book is disabled
unable to open book file [./books.bin].
search time set to 30.00.
max threads set to 1
book learning disabled
result learning disabled
hash table memory =  384M bytes.
pawn hash table memory =   40M bytes.

Crafty v22.1 JA (1 cpus)

White(1): epdpfga test01.epd test01-result.epd
PFGA: EPD record: 1
              time surplus   0.00  time limit 30.00 (+0.00) (30.00)
              depth   time  score   variation (1)
               11     0.38  -0.20   1. ... d5 2. Nf3 Nc6 3. d4 Bf5 4. Nh4
                                    Bd7 5. Nc3 Nf6 6. Nf3 Bf5
               11->   0.41  -0.20   1. ... d5 2. Nf3 Nc6 3. d4 Bf5 4. Nh4
                                    Bd7 5. Nc3 Nf6 6. Nf3 Bf5
               12     0.97  -0.22   1. ... d5 2. Nf3 Nc6 3. d4 Qd6 4. Nc3
                                    Nf6 5. Bd2 e5 6. Nb5 Qe7 7. e3 <HT>
               12->   1.20  -0.22   1. ... d5 2. Nf3 Nc6 3. d4 Qd6 4. Nc3
                                    Nf6 5. Bd2 e5 6. Nb5 Qe7 7. e3 <HT>
               13     3.03  -0.11   1. ... d5 2. d4 Nc6 3. Bf4 Nf6 4. e3
                                    Bg4 5. Be2 Bf5 6. Nf3 Nb4 7. Na3 e6
               13     5.22  -0.25   1. ... e5 2. Nf3 e4 3. Nd4 Nc6 4. Nxc6
                                    dxc6 5. Nc3 Nf6 6. d3 Bc5 7. dxe4 Qxd1+
                                    8. Kxd1 Bxf2
               13->   5.28  -0.25   1. ... e5 2. Nf3 e4 3. Nd4 Nc6 4. Nxc6
                                    dxc6 5. Nc3 Nf6 6. d3 Bc5 7. dxe4 Qxd1+
                                    8. Kxd1 Bxf2
               14    11.63  -0.23   1. ... e5 2. Nf3 e4 3. Nd4 c5 4. Nb5
                                    d5 5. c4 d4 6. d3 Nf6 7. dxe4 Nxe4
                                    8. Nd2
               14->  12.38  -0.23   1. ... e5 2. Nf3 e4 3. Nd4 c5 4. Nb5
                                    d5 5. c4 d4 6. d3 Nf6 7. dxe4 Nxe4
                                    8. Nd2
               15    25.35  -0.21   1. ... e5 2. e4 Nc6 3. Nc3 Nf6 4. Nf3
                                    d5 5. exd5 Nxd5 6. Bb5 Qd6 7. Nxd5
                                    Qxd5 8. Bxc6+ Qxc6 9. Nxe5 Qxg2
              time=30.14  mat=0  n=22912571  fh=91%  nps=760K
              ext-> check=786K 1rep=51K mate=533 pp=0 reduce=13.4M/1.8M
              predicted=0  evals=18.0M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/512  elap=30.14
Black(1):

Please note that final evaluation is negative (it is a relative one) : -0.21

And now the resulting epd :

Code: Select all

rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - acd 15; acn 22912571; acs 30; ce 21; pv e5 e4 Nc6 Nc3 Nf6 Nf3 d5 exd5 Nxd5 Bb5 Qd6 Nxd5 Qxd5 Bxc6+ Qxc6 Nxe5 Qxg2;

Here the final evaluation is positive : "ce 21".

This is _not_ epd-specification-compliant.

And there is an evident contradiction between crafty output to standard-out during analysis and the recording with opposite sign in the resulting epd file.

Marc

[plattyaj]

But that looks like it is exactly correct. Side to move is black. The crafty analysis output shows a small negative (e.g. black winning) score and the epd shows the same score but reversed to be show positive for side on the move and negative against.

Perhaps you should try this position and verify that the epd score is extremely positive:!

6K1/3q4/4k3/8/8/8/8/8 b - - 0 1


Andy.