Estimating piece strength

Discussion of chess software programming and technical issues.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
grandmastermac
Posts: 15
Joined: Mon Apr 20, 2020 5:07 am
Full name: Paul Macdonald

Estimating piece strength

Post by grandmastermac » Sun Apr 26, 2020 9:26 am

Hi there, I am developing a chess variant and I was wondering if anybody could shed some light on some tried and proven methods/formula for evaluating/determining relative piece strength? The pieces I am trying to evaluate are the Chancellor, Wildebeest and Arch-bishop in relation to traditional chess pieces Queen, King and Pawn.

Vinvin
Posts: 4766
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Estimating piece strength

Post by Vinvin » Sun Apr 26, 2020 10:55 am

Basically : play a lot of games with a lot of values and hold the version with the best score.
There are methodologies to converge fast on good values.
See the paper here : http://www.ke.tu-darmstadt.de/publicati ... riants.pdf

User avatar
leanchess
Posts: 84
Joined: Sun Dec 08, 2019 7:16 pm
Full name: Dmitry Shechtman
Contact:

Re: Estimating piece strength

Post by leanchess » Sun Apr 26, 2020 11:04 am

Some research on chancellor and archbishop values on a 10x8 board:

https://gothicchess.info/articles_02.shtml

Hope that helps.

User avatar
hgm
Posts: 25062
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Estimating piece strength

Post by hgm » Sun Apr 26, 2020 2:00 pm

No, that will NOT help. The values given at the link are completely wrong. The Archbishop is off by as much as 2 Pawns.

The problem is that this kind of research is what we call 'fact free'. Basically it is a pure guess, made in a round-about way: rather than guessing a value directly, you guess a way in which values could depend on number of moves and type of moves, and then use that guessed way to exactly calculate a value. But the value that comes out will still be as wrong as the guessed theory was. Scientific theories are only any good when they have been tested against ampirical observations. The known values of the orthodox Chess pieces do not offer enough constraints to determine which of the many possible theoretical formulas is the correct one. So never believe values based on a calculation!

So indeed, the only reliable way to get piece values is the empirical one: play games to see how the pieces perform in practice. Many games, in order to get a result that is statistically significant.

The method suggested by the second poster (changing an engine's idea of the values, and determine by playing games what gives the best result) is very inefficient, though. The reason is that a wrong idea of the value only has an effect if it causes you to make a wrong trade. E.g. if you would put a Queen at 800 cP instead of 950, and play against an opponent that values it normally, in most games the Queens would still be traded simply against each other. Because both players will still think that this is an equal trade. If only 10% of the games offer the opportunity to trade a Queen against a combination of other material (which both engines will then happily take because of the Q-value difference), the amount of damage a Queen on average does must be determined purely from that 10% of the games, as in the other games in cancelled out. But the other 90% contributes to the statistical noise. The result on the score will be 10 times smaller than the effect of having a Queen, and to determine it with the same precision will then require 100 times as many games. If you would somehow filter out the games where the Queens were traded against each other, and base your result only on the 10% that is left, you would still need to play 10 times as many games as you will eventually use.

This can all be avoided by making sure all games involve a Queen against some combination of other material, by already putting such an imbalance in the initial position from which you start the games. E.g. from the FIDE setup delete a black Queen, and 2 white Knights and a white Bishop. The games you then play will tell you if the (opening) value of a Queen is larger than that of 3 minors, or smaller. Be sure to start the games from a quiet position, though. Otherwise the material balance will immediately change in all of the games, and you will not be measuring what you think you are measuring.
Get rid of the shit: vote for SHID!

User avatar
Evert
Posts: 2929
Joined: Fri Jan 21, 2011 11:42 pm
Location: NL
Contact:

Re: Estimating piece strength

Post by Evert » Sun Apr 26, 2020 2:02 pm

This has been discussed a couple of times here. Look up posts by HGM on the topic (use the forum search).
The short form is this: for sliders it is hard to derive an empirical expression based on the move set because they can be blocked and so have highly variable mobility. Part of their value comes from their increased strength as the board empties up.

Having said that, you could do worse than picking the values defined by FairyMax or SjaakII (or Nebiyu, presumably, although I don’t know if it comes with presets for all of these).

grandmastermac
Posts: 15
Joined: Mon Apr 20, 2020 5:07 am
Full name: Paul Macdonald

Re: Estimating piece strength

Post by grandmastermac » Mon Apr 27, 2020 7:44 am

Thanks to all for your efforts and insights. Very helpful. Thank you!! :D

Post Reply