Testing of Kaufman Material Values

[mjlef]

Has anyone done some meaningful testing of the Larry Kaufman material values? In particular, does decreasing knight values as pawns come off the board or increasing rook values for the same seem to help?

Perhaps Bob has done a nice long, meaningful run on his machines?

Mark

[bob]

Has anyone done some meaningful testing of the Larry Kaufman material values? In particular, does decreasing knight values as pawns come off the board or increasing rook values for the same seem to help?

Perhaps Bob has done a nice long, meaningful run on his machines?

Mark

Tracy wrote some code to do the things Larry suggested. However we found nothing at all that made any difference. The things he suggested with respect to pawns and pieces, redundant pieces, coordinated pieces, etc, all had no significant effect, plus or minus.

May well work for others, but for us it was a non-working idea. Note that many terms are sort of redundant. If you evaluate mobility for a rook, then rook on open file scoring overlaps, since you have more mobility on an open file. And you have more mobility as pawns come off. So the overall effect probably is sensitive to what you are already evaluating...

[Osipov Jury]

From my test: Rybka 3 without material imbalance values - minus 68 points Elo.

[MattieShoes]

"material imbalance values"? By this, do you mean RP vs BN, etc?

[Osipov Jury]

"material imbalance values"? By this, do you mean RP vs BN, etc?

Yes, I mean RP vs BN, PPP vs B or N, and full complex of another combinations. Rybka 3 calcullate this values in start of engine. Function contain approximately 200 lines of code.

[Uri Blass]

From my test: Rybka 3 without material imbalance values - minus 68 points Elo.

I suspect that you can get most of this improvement without material imbalance tables only by changing the value of pieces.

Uri

[bob]

"material imbalance values"? By this, do you mean RP vs BN, etc?

Yes. Ideas like a second knight is not as valuable as the first, due to redundancy, etc. Which is an extension of the usual BN vs RP is bad. But it is not quite as bad if you have a second knight as getting rid of one also removes a little bit of redundancy. Other ideas include "queen is more valuable with more pawns on the board, rooks are more valuable as pawns come off" and such.

I tested the queen idea using a bonus of 0 to 12 for each pawn on the board. This only does anything when you have a Q vs 2R for example. Since there could be 16 pawns, the bonus went up to 12*16. The result was absolutely no Elo change. I played 32,000 games for each of those bonus values, 0, 1, ..., 12. Zero change. All ended within 3 Elo of each other, which is well inside the error bar for 32,000 games.

[bob]

From my test: Rybka 3 without material imbalance values - minus 68 points Elo.

I suspect that you can get most of this improvement without material imbalance tables only by changing the value of pieces.

Uri

I agree. We have done a few million games of testing to choose our current piece values. After that, none of Larry's suggestions have helped at all. Static values are far easier/faster to use than dynamic values. And if the static values are as accurate, there's no reason to not use them.

[mjlef]

Programs with material tables tend to encode lost of knowledge handled by rules in other programs (like a KB vs KP is most like a draw, and no real advantage to the B side). I wonder how much of the rating increase is due to these simple rules, versus the full board material imbalance terms?

[hgm]

My suspicion is that most of the Elo difference is caused by the Bishop pair. I assume that its value is implemented in the material tables; as it could be implemented there by simply initializing the tables differently, it would be silly to implement it by a seperate eval rule, which would take execution time. But of course any self-respecting program that does not use materia tables, still implements the Bishop pair through a dedicated rule.

So it is really a bit of a meaningless comparison. You cannot switch of the subtle effects of the material table without removing very basic knowledge that almost very engine has.