Piece/square table challenge

[Bloodbane]

What kind of a typo did you make? If you just changed the value of one square from positive to negative or something like that don't bother - it won't influence the result.

Also looks like I have some PST tuning to do.

[Harald]

Hi

Here is some help for transforming PSTs from code to data.

Code: Select all

// Turn source code layout to board square layout.
// A1=0, H1=7, A8=56, H8=63
#define TURN_TABLE( \
            a8,b8,c8,d8,e8,f8,g8,h8,\
            a7,b7,c7,d7,e7,f7,g7,h7,\
            a6,b6,c6,d6,e6,f6,g6,h6,\
            a5,b5,c5,d5,e5,f5,g5,h5,\
            a4,b4,c4,d4,e4,f4,g4,h4,\
            a3,b3,c3,d3,e3,f3,g3,h3,\
            a2,b2,c2,d2,e2,f2,g2,h2,\
            a1,b1,c1,d1,e1,f1,g1,h1 \
                  ) {               \
            a1,b1,c1,d1,e1,f1,g1,h1,\
            a2,b2,c2,d2,e2,f2,g2,h2,\
            a3,b3,c3,d3,e3,f3,g3,h3,\
            a4,b4,c4,d4,e4,f4,g4,h4,\
            a5,b5,c5,d5,e5,f5,g5,h5,\
            a6,b6,c6,d6,e6,f6,g6,h6,\
            a7,b7,c7,d7,e7,f7,g7,h7,\
            a8,b8,c8,d8,e8,f8,g8,h8 \
                    }

// Turn source code layout to board square layout.
// H1=0, A1=7, H8=56, A8=63
#define TURN_TABLE_H1A8( \
            a8,b8,c8,d8,e8,f8,g8,h8,\
            a7,b7,c7,d7,e7,f7,g7,h7,\
            a6,b6,c6,d6,e6,f6,g6,h6,\
            a5,b5,c5,d5,e5,f5,g5,h5,\
            a4,b4,c4,d4,e4,f4,g4,h4,\
            a3,b3,c3,d3,e3,f3,g3,h3,\
            a2,b2,c2,d2,e2,f2,g2,h2,\
            a1,b1,c1,d1,e1,f1,g1,h1 \
                  ) {               \
            h1,g1,f1,e1,d1,c1,b1,a1,\
            h2,g2,f2,e2,d2,c2,b2,a2,\
            h3,g3,f3,e3,d3,c3,b3,a3,\
            h4,g4,f4,e4,d4,c4,b4,a4,\
            h5,g5,f5,e5,d5,c5,b5,a5,\
            h6,g6,f6,e6,d6,c6,b6,a6,\
            h7,g7,f7,e7,d7,c7,b7,a7,\
            h8,g8,f8,e8,d8,c8,b8,a8 \
                    }

Harald

[Evert]

I think it's important to bear in mind a few points about what piece square tables are useful for, what they should contain and what you would expect to find from an experiment like this.

Don't get me wrong: I think this is a very fun and interesting experiment and I think a lot can be learned from it. That said, one should be careful to not over-interpret the outcome because the piece square tables that will come out best from this experiment will not be the ones that are most generally best.

To me (and this is certainly my personal opinion) piece square tables should be generally valid statements. That is to say, they should be universal. I very much dislike asymmetric piece square tables, because the chess board is symmetric and any asymmetry suggests that the table tries to compensate for some missing evaluation term (pawns are a slightly special case because their movement is asymmetric). This includes things like advancing pieces to the enemy side of the board, pushing advanced pawns, rook on the 7th and even keeping the king out of the centre. The reason I dislike having these things in the piece square tables is that you will unconditionally apply this "knowledge" in positions where it does not apply, which can really hurt.

However, if you have a chess program with a very light evaluation that has very little knowledge, then a piece square table that implicitly contains a proxy for missing knowledge will perform best. As such, I would expect Adam's empirical tables to be very hard to beat.

As for what's wrong with Lucas' tables: I suggest adding an advancement bonus to the pawn end-game tables.

[velmarin]

Asymmetric set of Adam has an advantage when compared with a similar engine with little knowledge.
It gives you an extra aggressive.

I tried it on a similar program against a weak opponent and a suite of openings and was not very good.

The set of Luydmil me was not good, I do not understand the lack of assessment of pawns on seventh, ect.

Tool by Steve Maughan
http://www.chessprogramming.net/chess-e ... re-tables/

Code: Select all

I’ve been thinking about Piece-Square tables.  They’re not the most exciting part of computer chess but they seem quite important (based on Ed Schroder’s experiments), so I thought I’d put some time into thinking about reasonable values.  I’ve created a simple spreadsheet in Excel to help create them.  You can download it here (Piece-Square-Tables.xlsx).  It assumes your board representation starts with A1 = 0 and H8 = 63.

[Adam Hair]

Hi Adam,

Do you have a description of how you derived the tables? I'd love to understand more.

Steve

Here is a corrected version of how I derived the PST values. I did this sometime in 2012 and forgot the exact details:

I was interested in measuring the value of material imbalances from engine matches to see if they were roughly the same as what Larry found when analyzing human matches. So, I collected long time control engine games and filtered them for the higher Elo engines (2700+ CCRL Elo). Then, I followed Larry's process of looking for an imbalance that existed for 6 plies and found the average score of all games that contained that situation. I did this for White and Black and adjusted the scores to remove White advantage. I then converted this to Elo with the logistic model. From this study by Sune Fischer and Pradu Kannan, as well as my own study using these long time control games, it seems that 1 Elo approximately equals 1 centipawns. So, I converted Elo into centipawns.

I extracted data for the typical material imbalances and for pawn imbalances with various major and minor pieces left on the board. I used a branch of Scid called ChessDB to find the average scores for each type of imbalance. Scid has a material filter where you can specify the number and type of pieces and how many plies that particular collection exists, and Scid will find all games in which those conditions existed.

The Scid material filter also allows you to specify that a certain piece is on a particular square. I used this to find the piece/square values. As I did with the material imbalance data, I specified that the filter conditions had to exist for at least 6 plies in a game. For midgame PSTs, I set the filter so that each side had (IIRC):
0 or 1 queen
1 or 2 rooks
2 to 4 minors
5 to 8 pawns
Also, I set the filter to look at moves 13 through 40. Almost all the engine games used books or opening positions that are 12 moves or less, so I mostly avoided gathering book data and got data resulting from engine play.

For endgame PSTs, I set the filter to look after move 40 and so that each side had:
0 or 1 queen
0 or 1 rook
0 to 2 minors
0 to 4 pawns

For the individual piece/square values, in addition to the above filter conditions, I would specify that a particular piece was on a particular square, for White and for the comparable square for Black. So, I would get the average score for when a White pawn was on a3 and for when a Black pawn was on a6. I then combined the scores together in a way to adjust for White advantage:
((White score-Black score)/2)+50%
The formula assumes that the difference from 50% when White score and Black score are averaged together is due to a bias for White (or Black). It seems to be a decent assumption. For the midgame filters, the average of Black and White scores for a particular filter are around 52% to 56%. For the endgame, the average is around 50%, which I interpret to mean that the advantage for White dissipates the longer a game lasts.

I converted the adjusted score for each piece and square into centipawns in the manner described above. If there were not a sufficient amount of games for a particular piece/square for White and Black (IIRC there had to be at least 150 games for each color), I did not record a value. That is why there are 0s in places in my midgame PSTs. Given the lack of continuity of the midgame PSTs (adjacent squares can have very different values), I did not know correct values to put in those squares.

The endgame PST values I found from the data were not as extreme as the midgame values. For example, here is what I found for a knight in the endgame (a1 is on bottom left):

Code: Select all

   0  -5  26   9  12  14   9   0
   8  13   5  31  20  16   9   7
  -1  10  11  22  35  25  18  25
  -3 -10  21  21  31  29  26   3
 -39  -1  10   6  13  16  11   3
 -75 -31 -23 -10  -6 -14 -13 -35
 -98 -61 -47 -42 -37 -32 -44 -78
 -99 -99 -87 -80 -73 -85 -80 -99

So, I decided to smooth out the endgame values (and fill in for missing values) by finding equations that best match those values. I used the square placement for the x and y coordinates and the extracted value as the z coordinate. For example, from the table above one data point is (3,5,21). I then found the 3 dimensional surface that best fit the data, and used that equation to form a new endgame table:

Code: Select all

 -21  -3  10  16  16  10  -3 -21
  -7  12  25  31  31  25  12  -7
  -2  17  30  36  36  30  17  -2
  -7  12  25  31  31  25  12  -7
 -22  -3  10  16  16  10  -3 -22
 -46 -27 -15  -9  -9 -15 -27 -46
 -81 -62 -49 -43 -43 -49 -62 -81
 -99 -99 -94 -88 -88 -94 -99 -99

Smoothing out the endgame PSTs did not cause a large effect when I tested the PSTs with Cupcake (Java engine written by Dan Honeycutt), but it was a possible small improvement. I was not successful in doing the same for the midgame PSTs. They need more work.

[Adam Hair]

I think it's important to bear in mind a few points about what piece square tables are useful for, what they should contain and what you would expect to find from an experiment like this.

Don't get me wrong: I think this is a very fun and interesting experiment and I think a lot can be learned from it. That said, one should be careful to not over-interpret the outcome because the piece square tables that will come out best from this experiment will not be the ones that are most generally best.

To me (and this is certainly my personal opinion) piece square tables should be generally valid statements. That is to say, they should be universal. I very much dislike asymmetric piece square tables, because the chess board is symmetric and any asymmetry suggests that the table tries to compensate for some missing evaluation term (pawns are a slightly special case because their movement is asymmetric). This includes things like advancing pieces to the enemy side of the board, pushing advanced pawns, rook on the 7th and even keeping the king out of the centre. The reason I dislike having these things in the piece square tables is that you will unconditionally apply this "knowledge" in positions where it does not apply, which can really hurt.

However, if you have a chess program with a very light evaluation that has very little knowledge, then a piece square table that implicitly contains a proxy for missing knowledge will perform best. As such, I would expect Adam's empirical tables to be very hard to beat.

As for what's wrong with Lucas' tables: I suggest adding an advancement bonus to the pawn end-game tables.

In the discussions that Miguel Ballicora and I had about the PST data, Miguel had an interesting idea. In general, the empirical values would not work for an engine that has a moderate amount of evaluation terms. But it might be possible to use a little asymmetry in the PSTs, if the asymmetry followed the kings.

[lucasart]

I very much dislike asymmetric piece square tables, because the chess board is symmetric and any asymmetry suggests that the table tries to compensate for some missing evaluation term (pawns are a slightly special case because their movement is asymmetric).

I agree with everything you say, which boils down to respecting the principle of orthogonality when building an eval.

However, regarding symmetry: are you referring to file-wise symmetry ? rank-wise ? or both ?

My take on this is that file-wise symmetry and strict minimality/generality knowledge in PST is an axiom on which I build my eval. Any patch that improves the engine by violating this axiom is heretic by nature, and should raise the question: what is this compensating? is there an eval feature I lack or is badly tuned interacting with this?

But I think that some PST do not need rank-wise symmetry. For example the king opening PST. At some point there needs to be a penalty in your eval for a king being far away from safety (where "safety" is typically the canonical squares B1 and G1 for white). If you put it in your dynamic king safety, there has to be a part that is triggered even in the absence of king attacks. It's just more efficient to put it in the PST.

Now here is an example where I would never break rank-wise symmetry: pawn endgame symmetry. The whole idea of increasing PST with rank is basically to compensate for a deficient passed/candidate/space evaluation. Best to leave a zero PST and re-tune the aforementionned eval features instead.

Of course optimal PST in a pure PST competition like this one, has nothing to do with optimal PST in the presence of any given reasonable eval. I didn't expect DiscoCheck's PST to be best here, but I was just curious how they would fare.

[Lyudmil Tsvetkov]

Asymmetric set of Adam has an advantage when compared with a similar engine with little knowledge.
It gives you an extra aggressive.

I tried it on a similar program against a weak opponent and a suite of openings and was not very good.

The set of Luydmil me was not good, I do not understand the lack of assessment of pawns on seventh, ect.

Tool by Steve Maughan
http://www.chessprogramming.net/chess-e ... re-tables/

Code: Select all

I’ve been thinking about Piece-Square tables.  They’re not the most exciting part of computer chess but they seem quite important (based on Ed Schroder’s experiments), so I thought I’d put some time into thinking about reasonable values.  I’ve created a simple spreadsheet in Excel to help create them.  You can download it here (Piece-Square-Tables.xlsx).  It assumes your board representation starts with A1 = 0 and H8 = 63.

Hi Jose.

Did you try my set?
What values did you use for the king table, as I filled only the 5 other tables? Also, that was supposed to match Stockfish piece values, 2.5 times bigger than normal. I can not imagine it could do bad, as it is closest in philosophy to Adam's set, which did best. On the other hand, sets that had just centralisation, did relatively bad.

Not that I care, but this is spreading rumour and misconceptions.

If psqt should be the basis of your engine, then start with something similar to Adam's or mine.

[Evert]

In the discussions that Miguel Ballicora and I had about the PST data, Miguel had an interesting idea. In general, the empirical values would not work for an engine that has a moderate amount of evaluation terms. But it might be possible to use a little asymmetry in the PSTs, if the asymmetry followed the kings.

Yes, this makes perfect sense to me - in spite of what I said earlier about PSTs being symmetric. The difference is that conceptually I would split this in a symmetric part (the PST) and an asymmetric part, which records how well pieces are placed with respect to specific targets in the current position. A form of king tropism, if you will.

[Evert]

I agree with everything you say, which boils down to respecting the principle of orthogonality when building an eval.

Essentially, yes. With the added statement that a PST is perhaps not the best way to include certain types of knowledge.

However, regarding symmetry: are you referring to file-wise symmetry ? rank-wise ? or both ?

Actually I was thinking of full symmetry: rank, file and reflection along the main diagonal.

Of course, in practice each of these symmetries will be broken to one degree or another, the diagonal one being the most obvious: a bishop on c4 is generally better placed than one on d3 because the bishop on d3 is in the way of its own pieces and does not attack the enemy position as efficiently as one on c4. It's probably entirely safe, at least in the middle game, to build breaking this symmetry in to the piece square tables (which I actually do myself as well).

Breaking the file (king/queen side) symmetry can be done based on the location of the enemy king (or some other type of weakness), but that is probably best handled explicitly.

The rank symmetry is clearly broken by the presence of pawns and unless you're in a very late end-game should be pretty safe to encode within the piece square tables - although I prefer doing this differently myself.

But I think that some PST do not need rank-wise symmetry. For example the king opening PST. At some point there needs to be a penalty in your eval for a king being far away from safety (where "safety" is typically the canonical squares B1 and G1 for white). If you put it in your dynamic king safety, there has to be a part that is triggered even in the absence of king attacks. It's just more efficient to put it in the PST.

Sure.
The evaluation term that gives rise to the asymmetry should be something like "stay behind your own pawns" + "stay somewhere that is relatively hard to attack by the enemy army (ie, the corner)" + "stay away from enemy pieces (reverse tropism; prefer g1 to h2)". It's certainly easier and almost certainly as effective to just tell it to stick to g1. Reasons for not doing it that way are more philosophical than practical.

All of that breaks down a bit if you're trying to handle (arbitrary) chess variants as well as normal chess, of course, which is partly why I try to find more generally valid principles and try to keep my PSTs strictly symmetric (not much a concern for Jazz, which only plays chess, more of a concern for Leonidas and a major concern for Sjaak, where the user can define his own variants).

Now here is an example where I would never break rank-wise symmetry: pawn endgame symmetry. The whole idea of increasing PST with rank is basically to compensate for a deficient passed/candidate/space evaluation. Best to leave a zero PST and re-tune the aforementionned eval features instead.

Agreed 100%. Whether pawns should be advanced or not is not something that you should leave up to the PST to decide (not that I've found a really satisfactory way of doing it otherwise, but...)

Of course optimal PST in a pure PST competition like this one, has nothing to do with optimal PST in the presence of any given reasonable eval. I didn't expect DiscoCheck's PST to be best here, but I was just curious how they would fare.

Well, removing any incentive to advance pawns in the end-game may hurt badly enough that it dominates whatever merit there is in the rest of the tables.

I still have to extract my own tables for this competition (temporarily no permanent internet connection + the need to find a moment to do it makes this a bit of a hassle), but I'll (probably) stick some advancement bonus in there even though that's not normally handled by the PSTs.

EDIT: WTF?! My post just reverted to just the quoted text... should be fixed now.