compact score value representation

[elcabesa]

I have seen in stockfish that score value are represented in a compact way, opening value in the high half and ending value in the low half, and addition are done in a single instruction.
In the past I have tried to replicate this behavior in Vajolet but i got huge elo loss.

Today I tried to understand why and I write a program that verify the correctness of adding positive/ negative value using the compact way.
I have seen that the resulting value are wrong in the high half by +-1. Is this known and accepted because the score is then scaled, or am I doing something wrong?

this is my code

Code: Select all

#include <iostream>
#include <stdlib.h>
using namespace std;

union comp{
	short int si[2];
	unsigned int i;
}compostoRes,composto1,composto2;
int main() {
	cout << "start" << endl; // prints

	int maxErr=0;
	short int start=-1,stop=0;
	for(short int a=start;a<stop;a++){
		for(short int b=start;b<stop;b++){
			//std::cout<<a<<":"<<b<<" "<<maxErr<<std::endl;
			for(short int c=start;c<stop;c++){
				for(short int d=start;d<stop;d++){
					int	sum1=a+b;
					int sum2=c+d;
					composto1.si[0]=a;
					composto1.si[1]=c;
					composto2.si[0]=b;
					composto2.si[1]=d;
					compostoRes.i=composto1.i+composto2.i;

					if(abs(compostoRes.si[0]-sum1)>maxErr){
						maxErr=abs(compostoRes.si[0]-sum1);
					}
					if(abs(compostoRes.si[1]-sum2)>maxErr){
						maxErr=abs(compostoRes.si[1]-sum2);
					}
					if(compostoRes.si[0]!=sum1 || compostoRes.si[1]!=sum2){
						std::cout<<"a:"<<a<<",b:"<<b<<",c:"<<c<<",d:"<<d<<std::endl;
						std::cout<<"res:"<<compostoRes.si[0]<<","<<compostoRes.si[1]<<std::endl;
						std::cout<<"sum:"<<sum1<<","<<sum2<<std::endl;
						std::cout<<"composto1:"<<composto1.i<<std::endl;
						std::cout<<"composto2:"<<composto2.i<<std::endl;

					}
					std::cout<<"res:\t"<<compostoRes.i<<std::endl;
					compostoRes.si[0]=sum1;
					compostoRes.si[1]=sum2;
					std::cout<<"wanted:\t"<<compostoRes.i<<std::endl;
					/*else{
						std::cout<<"good"<<std::endl;

					}*/
				}
			}
		}
	}

	std::cout<<"Max err: "<<maxErr<<std::endl;
	cout << "end" << endl; // prints

and this is the result I got:

Code: Select all

a:-1,b:-1,c:-1,d:-1
res:-2,-1
sum:-2,-2
composto1:4294967295
composto2:4294967295
res:	4294967294
wanted:	4294901758
Max err: 1
end

[elcabesa]

I have just checked a little better and i saw that stockfish routine doen't have this problem

Code: Select all

inline int makeScore(int mg, int eg) { return int((mg << 16) + eg); }
inline int mg_value(int s) { return int(((s + 32768) & ~0xffff) / 0x10000); }

inline int eg_value(int s) {
  return int((int)(unsigned(s) & 0x7fffu) - (int)(unsigned(s) & 0x8000u));
}

could someone explain me the math or point me to some link with math explanation??

thank you

[Chan Rasjid]

Hello,
I think you mean this.

Your int PST table stores the opening scores in the high 16 bits and the ending scores in the lower 16 bits. Then you could just update the PST scores in makemove "in parallel".
EDIT : in parallel means you could just assume it is a single PST score value.

You retrieve with :

Code: Select all

int pstOpenEndScore, pst_end, pst_open;

// assumes 16 bits PST will not set the signed bit in the 32-bit integer  
    pst_end = (pstOpenEndScore << 16) >> 16;
    pst_open = (pstOpenEndScore - pst_end) >> 16;

Best Regards,
Rasjid.

[elcabesa]

yes I'm searching something like this

[Gerd Isenberg]

I have just checked a little better and i saw that stockfish routine doen't have this problem

Code: Select all

inline int makeScore(int mg, int eg) { return int((mg << 16) + eg); }
inline int mg_value(int s) { return int(((s + 32768) & ~0xffff) / 0x10000); }

The point is that eg, if negative, borrows from mg. Therefor, to extract mg, adding 0x8000 (16-bit sign bit) compensates that borrow.

Div by 0x10000 is like shift arithmetic right (sar) 16, but adds 0xffff if negative before the shift to compensate the asymmetry that 1 >> 16 is zero but -1 >> 16 is still -1. Since the lower 16-bits are cleared before the div anyway, one can safely replace div 0x10000 by sar 16.

Code: Select all

inline int eg_value(int s) {
  return int((int)(unsigned(s) & 0x7fffu) - (int)(unsigned(s) & 0x8000u));
}

could someone explain me the math or point me to some link with math explanation??

Extracting eg requires a 32-bit sign extension of the stored signed 16-bit integer. Taking the 15 significant bits and subtracting the sign from 2**32 does that cheaply.

[elcabesa]

I tried using an union for this, but it has a problem the lower 16bits will overflow in the high only when the sum of the unsigned value is >= 0x10000.

I think it's not so easy to understand from the result if the lower 16 bits overflowed or not.

so probably it's better to use the stockfish code, it's more complex but of course make_Score , mg_value, eg_value are not used lots of times so the complexity seem to be not a problem.

[hgm]

You are not seriously believing that a difference of 1 score point in the opening evaluation can cause 'huge Elo loss', do you? You must have had some other bug.

As a 16-bit range is rather generous, you could multiply all your scores by 8, so that the difference is only 1/800 of a Pawn in stead of a centi-Pawn. This is much less than the precision loss due to the rounding when averaging end-game and middle-game scores.

If you really worry about this, you could keep the incremental eval in 'excess encoding', i.e. add 0x80008000 so that it always becomes an unsigned number. To 'flip sign' you then use (difEval ^ -1) in stead of -difEval. And after averaging you then subtract the 0x8000 (for wtm) or 0x7FFF (for btm).

[Sven]

I tried using an union for this, but it has a problem the lower 16bits will overflow in the high only when the sum of the unsigned value is >= 0x10000.

I think it's not so easy to understand from the result if the lower 16 bits overflowed or not.

so probably it's better to use the stockfish code, it's more complex but of course make_Score , mg_value, eg_value are not used lots of times so the complexity seem to be not a problem.

Please see also this thread from last year which might have a lot of useful information for you about this topic.

Sven

[sje]

My general take on these things is that tricky object compactication (a neologism) rarely pays off except as extra development time and as bug magnets. Perhaps the only place it's warranted is for structures like a transposition table entry which may have tens of millions of in-memory copies.

The time spent on trickery would almost certainly be better spent on other improvements.

[Daniel Shawul]

Indeed, I don't use such tricks where structs/unions will do. The only place for it in chess engines is for representing a MOVE where bitfields may be slower.

Code: Select all

typedef struct tagHASH {
	HASHKEY hash_key;
	union {
		HASHKEY data_key;
		struct {
			UBMP32  move;
			BMP16   score;
			UBMP8   depth;
			UBMP8   flags;
		};
	};
}HASH,*PHASH;