Why material imbalance tables are needed

[hgm]

I think pawns are extremely important, and as Kaufmann's paper shows, at least the value of knights and rooks vary considerably with the number of pawns.

Yes, but making the value of a Bishop dependent on the number of Pawns is equivalent to making the base value of a Pawn dependent on the number of Bishops.

My future plans for Joker involve to keep track of a number of key features of the Pawn structure, such as number of passers, number of backward Pawns on half-open lines, total number of Pawns, number of a- and h-Pawns. These numbers would be either differentially updated, or go into a Pawn hash table (whichever proves faster). In the evaluation these features would be weighted by factors depending on the (non-pawn) piece makeup. This scheme fully allows dependencies of the Kaufman type, which are simple products of Pawn count and Rook or Bishop count.

[Michael Sherwin]

Hi Michael,

you're on the right way
I'm working on such stuff already and you don't need a hashkey for your table, which saved a lot of space. Just calculate the max possible material if we assume only 2 minor or 2 major can be exist. If we have 3 bishops for example you don't need such table.

Ok, lets talk about some secrets and ideas.
So , here we go:

Code: Select all

typedef struct material_t {
	int light;			//material white
	int dark;			//material black
	int state;			//bit based flagtable
	EVAL_S evalFactors; //stored integer factors for mobility, 
						//pawn damged, king attack and so on..
} MATERIAL_STRUCT;

//2,4 mb total size
MATERIAL_STRUCT MaterialTable[9*3*3*3*2*9*3*3*3*2];

//key build on material:
key =  bq  +  br *2 + bb *2*3 + bn *2*3*3 + bp *2*3*3*3 + wq *2*3*3*3*9 + wr *2*3*3*3*9*2  + wb *2*3*3*3*9*2*3 +  wn *2*3*3*3*9*2*3*3 +  wp *2*3*3*3*9*2*3*3*3;

Best,
Daniel

Hi Daniel,

Thanks! Also very interesting. I take it that bq, br, etc. are the numbers for each type of piece? But, how are you able to store mobility and king safety if the exact pawn structure is not stored?

Best,
Mike

[Aleks Peshkov]

I think the main problem is how to collect material evaluation chess knowledge. And how to make sure that this data is better then random numbers.

The task is similar to Piece-Square-Tables: the function is very easy to evaluate, it is obviously better then nothing, but exact values are mainly a pure common sense guess.

[Tord Romstad]

Most of the time, simplistic ends up being better than complicated. The pawn structure and piece mobility factors may be best handled in the traditional style.

Maybe, but it is not so easy to separate the two. For instance, as pointed out elsewhere in the thread, the main reason why knights tend to be strong when there are many pawns on the board and rooks strong when there are few pawns on the board is mobility. If you already evaluate mobility, your knights already are evaluated as stronger (on average) in positions with many pawns, even before you collect statistics from a database of GM games and use it to score different material configurations.

The material balance tables will just supply a gentle modifier to keep from making mistakes based on static piece values that are 'out of whack' with praxis. The most appealing aspect of what I propose, is that it is within my ability to do it! And it can be done quickly, so it would not cost all that much if it were a failure. And if it does work, then it would provide a 'measureing stick' to judge the possible gain by trying something more complicated.

Yes, it is very easy to try - but I believe that it would just add useless noise to the evaluation function. With a few exceptions, I think the values of the individual pieces depend much less on what other pieces are present than on other aspects of the position.

It's possible that I am wrong, of course - it wouldn't be the first time. I already have most of the code I need to experiment with this, so I may as well try.

I think that hashing in the pawn structure and having specialized piece square tables as well as material composition factors is best for openings and can be stored in seperate files based on eco codes. These files would be tuned to the programs opening book and it would also allow for learning based on an engines results.

If I recall correctly, Jeff Rollason does something like this in Shotest. He calls it a "fuzzy book".

Tord

[Tord Romstad]

Yes, but making the value of a Bishop dependent on the number of Pawns is equivalent to making the base value of a Pawn dependent on the number of Bishops.

Clever! That would probably work. It seems very obvious in hindsight, but I didn't think of it.

Tord

[Michael Sherwin]

I would suppose that using a human game database would be best for playing against humans and using a computer game database would be best for playing against other computers.

[Allard Siemelink]

A problem with all these things is of course how it interacts with the previously defined evaluation terms, especially mobility. For instance, the main reason why rooks are usually stronger with few pawns on the board is that they have higher mobility.
Tord

Well, what I do is calculate scores for the patterns and compare that to the score of the evaluation function. The difference is the number of points to adjust the eval for the pattern.
In fact I do not just calculate scores for material imbalances, but also for most other evaluation terms e.g mobility, passed pawns per rank and so on.

So for the rooks, I would calculate mobility scores for the different numbers of squares that they see depending on the number of own/opponent pawns on the board. Then adjust the eval accordingly.

I am not sure that this approach is making Bright stronger in all cases though. To do it well, probably more correlation patterns are needed.
e.g. a rook's value probably also correlates with number of opponent pawns on open files/ranks and their vulnerability.
Or maybe it will turn out to be yet another dead-end strategy, I am not sure at this point.

[Harald]

This is a reply to ideas in several postings, not only the one above.

Keys made of material or piece signatures are suggested for
evaluation modifiers.

If you want to have a pawn signature in 48 bit you can do this:
For white pawns set a bit (0 to n=23) if a pawn is on the square.

Code: Select all

x x x x x x x x
g h i j k l m n
g h i j k l m n
8 9 a b c d e f
8 9 a b c d e f
0 1 2 3 4 5 6 7
0 1 2 3 4 5 6 7
x x x x x x x x

For black pawns use the bits 24 to 47. This even gives an estimate about
passed pawns or isolated pawns or backward pawns on half open lines and so on.
If the 48 bits are too much and a full table would be too big, then use a
hash value of this and put the signature keys in a list starting at the hash
index. Chose only the most important or common signatures that you
find in a PGN database. You can attach any other evaluation terms and
flags to the signatures.

We could also use king position bits (2*3 bit or 2*4 bit). Do you want to
see the diagrams?

We could use additional bits for the key. Like bishop pair or passed pawn
existence.

If we have a complex evaluation modifier based on PGN databases, there
would certainly not be enough to fit to all positions in the search of our
own engine's game. Even if we probe with symmetry in mind. May be we
just get hints at some nodes in the search tree. Can we use the modifiers
for every deeper node of a matched position? Should the modification fade out?

Tord mentioned a conflict of mobility and pawn number based material
values. Can we just abandon the mobility then? Mobility cost more time.

Harald