measuring complexity of chess positions

[Uri Blass]

Here is my idea of measuring complexity of a chess position.

take a set of 56 deterministic chess engines(I already did it with 56 deterministic chess engines when all use single core and fixed number of nodes).
practically my list of engines have 8 engines but I tested them at 7 different fixed nodes per move.

2)Do engine-engine tournament when everybody play everybody with white and black(I got 110 games for every engine)

3)Have a rating list for the engines based on the games(banksia gui calculates the rating wrong because engines with the same number of points can get a different rating)

Look at the rating gap between place 14 and place 42 in the list
If the gap is bigger than it means the position is more complex.

Note that I choose place 14 and 42 and not first place and last place because I am afraid last place may have a bug in some specific position that may cause the rating gap to be bigger so I prefer not to be influenced by last place to define complexity of the position.

[towforce]

Until we have a God algorithm for chess, complexity is very important. Good chess programs should aim for complexity as a way to win games for two reasons:

1. Complex positions throttle programs' ability to search: search depth is reduced dramatically

2. NNs (and human brains) are less able to understand complex positions

For these two reasons, it would be a good idea for engines to measure the complexity of positions, and, other things being approximately equal, to prefer complexity: blinding the opponent is a good way to increase the chances of your opponent making a mistake, and hence your chances of winning.

[Uri Blass]

Until we have a God algorithm for chess, complexity is very important. Good chess programs should aim for complexity as a way to win games for two reasons:

1. Complex positions throttle programs' ability to search: search depth is reduced dramatically

2. NNs (and human brains) are less able to understand complex positions

For these two reasons, it would be a good idea for engines to measure the complexity of positions, and, other things being approximately equal, to prefer complexity: blinding the opponent is a good way to increase the chances of your opponent making a mistake, and hence your chances of winning.

In order to help engine to measure complexity we should have some epd file with compexity score for chess positions when the complexity score is calculated based on the gap in playing strength based on tournament with a lot of participants.

I think the tournament should include some leagues at different levels because having too many cases when you can be sure about 100% results even in the opening position is a waste of testing time and we should have some rating gap based on the results that measure complexity.

Note that the idea that I practically would like the top engines in every league to get 100% against the bottom engines but top against bottom is not going to be majority of the games in the league.

[jefk]

some years ago, besides the score indicator there was a 'sharpness' indicator (like some kind
of thermometer indicator) in some older (but not too old) Fritz versions/GUI's.
For example positions from a Sicilian Najdorf poisoned pawn variation got
-understandably- higher sharpness indications (which were measured in real time
by the engine, besides the value). Dunno why they abandoned it, it then
already looked like a good idea to me.
Anyway such a 'sharpness' seems a bit similar to a complexity indication which you
suggest. Nowadays the Lc0 engine can give w/l/d scores, and lower draw scores
should give higher 'sharpness'. It would be nice to get such scores (back) for other
engines (instead of the meaningless 0.0 SF scores).
When watching online GM games, instead of the b/w vertical bar, often halfway
in the middle (until a mistake is made), it would be much more interesting for spectators
(also in sharp endgames) to see a colored bar with w/d/l scores (e.g. red/white/green).

Some years ago, regarding the topic of choosing opening variations, I suggested
that an ideal opening book should not only be minimaxed on the basis of score, but
also -optionally- based on sharpness; thus you could get two books, one with
sharp openings, another (if you choose low 'sharpness) with positional openings,
thus against lower rated players you -or the engine- can choose the 'sharp' (or unbalanced,
or complex) opening lines and against the higher rated players (if you aim for a draw),
'positional' openings. Not surprisingly, I would expect that a lot of -'solid'- gambits
would occur in the 'sharper' books,

[chesskobra]

I think the following parameters are relevant to measuring complexity.

1. Simple to quantify: How many options are available that do not deviate much from the best engine evaluation? This may be estimated as the size of the subtree from the given position on which the eval remains within a small interval of the minmax eval, regardless of what variation in the subtree the players follow. If this is large, then there is more calculation work to do in the position. If there is a forced line and almost every other line is bad then the position is easier to play in general, especially for a machine. But not necessarily for a human player, who may miss the only good move.

2. Somewhat simple to quantify:Too many permutations in which captures can be made or threats can be created or defended against. Coach Andreas on YouTube says if there are multiple captures possible in a position, you must calculate all possible permutations of captures. Such positions are clearly not simple.

3. Difficult to define, somewhat subjective: How likely is it that a human player will miss a good move.

[towforce]

I think the following parameters are relevant to measuring complexity.

1. Simple to quantify: How many options are available that do not deviate much from the best engine evaluation? This may be estimated as the size of the subtree from the given position on which the eval remains within a small interval of the minmax eval, regardless of what variation in the subtree the players follow. If this is large, then there is more calculation work to do in the position. If there is a forced line and almost every other line is bad then the position is easier to play in general, especially for a machine. But not necessarily for a human player, who may miss the only good move.

2. Somewhat simple to quantify:Too many permutations in which captures can be made or threats can be created or defended against. Coach Andreas on YouTube says if there are multiple captures possible in a position, you must calculate all possible permutations of captures. Such positions are clearly not simple.

3. Difficult to define, somewhat subjective: How likely is it that a human player will miss a good move.

1. Simple to quantify: How many options are available that do not deviate much from the best engine evaluation? This may be estimated as the size of the subtree from the given position on which the eval remains within a small interval of the minmax eval, regardless of what variation in the subtree the players follow. If this is large, then there is more calculation work to do in the position. If there is a forced line and almost every other line is bad then the position is easier to play in general, especially for a machine. But not necessarily for a human player, who may miss the only good move.

2. Somewhat simple to quantify:Too many permutations in which captures can be made or threats can be created or defended against. Coach Andreas on YouTube says if there are multiple captures possible in a position, you must calculate all possible permutations of captures. Such positions are clearly not simple.

3. Difficult to define, somewhat subjective: How likely is it that a human player will miss a good move.

4. Easy to quantify: lots of pawns and pieces can get to lots of different locations over the next few moves.

5. Tricky to quantify: lots of tactical features likes traps, pins and threats in the position.

[towforce]

1. Simple to quantify: How many options are available that do not deviate much from the best engine evaluation? This may be estimated as the size of the subtree from the given position on which the eval remains within a small interval of the minmax eval, regardless of what variation in the subtree the players follow. If this is large, then there is more calculation work to do in the position. If there is a forced line and almost every other line is bad then the position is easier to play in general, especially for a machine. But not necessarily for a human player, who may miss the only good move.

2. Somewhat simple to quantify:Too many permutations in which captures can be made or threats can be created or defended against. Coach Andreas on YouTube says if there are multiple captures possible in a position, you must calculate all possible permutations of captures. Such positions are clearly not simple.

3. Difficult to define, somewhat subjective: How likely is it that a human player will miss a good move.

4. Easy to quantify: lots of pawns and pieces can get to lots of different locations over the next few moves.

5. Tricky to quantify: lots of tactical features likes traps, pins and threats in the position.

6. Generate a suite of positions together with complexity rating, and use this to train an NN to recognise complexity.