Algorithm to determine the difficulty of a chess problem

[Peperoni]

Hello,

I am trying to implement a simple algorithm to generate chess problems.
It's pretty simple, I analyze lichess game with an engine and when there is a dramatic change in the evaluation, it means something is happening.
However, I can't figure out a reliable way to determine the difficulty of the problem. I tried using the evaluation of an engine at lower depth, in the hope of using the depth the engine finally sees the dramatic change in evaluation, but that doesn't seem reliable.

Do you guys have any idea?

[towforce]

I think your plan is good!

A good puzzle will have:

1. Sudden change of evaluation ( = something in the position that's difficult to spot)

2. Short distance from evaluation change to absolute clarity that one side has won ( = good puzzle material)

Another approach might be to analyse a database of positions, and look for big differences between evaluations at different depths using the same engine: if a 4 ply search has white winning or equal, but a 6 ply search has white lost, and a 10 ply search has white completely lost, that could be a candidate puzzle position.

[abgursu]

Analyzing time can be a determining factor here. I would analyze positions with two or more engines. Longer the finding time, better the question.

[towforce]

I think there exist databases of games which already contain evaluations of each position (apologies if I am wrong about this). If so, I would definitely use those. The challenge of finding puzzle candidates would then become one of querying a database rather than an elaborate set up of having engines analyse positions in a database.

A possible drawback, though: these positions might be analysed too well, making it a lot more difficult in reality to find puzzle material. Per my previous post, you probably want depths 3, 5 and 8, or something like that.

Also, just guessing, but human games might be more fertile grounds than machine games: computers are probably too good to allow good human puzzle positions to arise (I am guessing you're wanting puzzles for humans).

[Ferdy]

Hello,

I am trying to implement a simple algorithm to generate chess problems.
It's pretty simple, I analyze lichess game with an engine and when there is a dramatic change in the evaluation, it means something is happening.
However, I can't figure out a reliable way to determine the difficulty of the problem. I tried using the evaluation of an engine at lower depth, in the hope of using the depth the engine finally sees the dramatic change in evaluation, but that doesn't seem reliable.

Do you guys have any idea?

Difficulty is relative a problem for a player with an Elo of 1500 may not be problem for a player with an Elo of 2000. So in the first place you have to define your Limit.

Is this intended for human or for computer? For computer select some engines with different ratings and test them. The puzzle can then be classified by the strength of engines. For human it is the same thing, just record the puzzles that were solved and not solved by specific group of players.

Just in case you have not known, I have some algo on generating puzzle in chess-chiller.

[kinderchocolate]

There is no reliable way to work out the difficulty of a problem, unfortunately. That involves human playing levels as part of the equations.

[Chessqueen]

Hello,

I am trying to implement a simple algorithm to generate chess problems.
It's pretty simple, I analyze lichess game with an engine and when there is a dramatic change in the evaluation, it means something is happening.
However, I can't figure out a reliable way to determine the difficulty of the problem. I tried using the evaluation of an engine at lower depth, in the hope of using the depth the engine finally sees the dramatic change in evaluation, but that doesn't seem reliable.

Do you guys have any idea?

The triangulation method could be implemented to top Engines which is used by GMs, but at the moment no programmer has implemented it. The triangulation method is very easy, it is used as a sequence of waiting moves to force the opponent to move pieces and eventually capture those pieces like this.