Blunder option

[Ferdy]

This one by Carlos is true, but a challenge to implement.

[Ferdy]

While this seems to work fine, I was wondering if there is a better/standard way to force blunders?

I am currently working by multipv. Make your engine as strong as possible so that its multipv is more reliable. There is no need to make the engine weaker by weakening its eval, search etc.

By multipv generate all possible moves with score and pv or perhaps less depending on the target level. Once the moves are ordered by score, it is just a matter of selecting a move from the best ordered moves. If level 1 is the weakest, pick a move that are lower in the multipv list. Stronger levels can have some of the top moves in multipv list. Play some games and observe. Easy moves like recapture just use the top move. A move that destroys its own castling right can be assigned at perhaps level 1 to 4, probably assign a target elo for your level. For example use the Lichess rating list, level 1 can be 800 Lichess rating and below and so on. The advantage of mapping your level to Lichess rating is that you can replay some games in Lichess for those specific players. You can even let your engine play there.

Mine your evaluation, get the king safety, mobility, passed pawn, threats, material etc. Factor this out to generate blunder, mistake and dubious moves. If you king safety is bad increase the frequency of blunder or mistake or dubious moves, same with mobility, etc.

Classify moves, a move that gives away material for free can be in level 1 to 3 etc. a rook that captures a pawn and the rook is seen in the pv (remember we save the [move, score, pv] in the multipv list) and is captured in the next ply can be in level 1 to 4 etc. If that rook is captured later in the pv, perhaps this move can be assigned in higher levels of say 5 to 8. A lot can be learned by examining these pv lines.

So after generating all moves by multipv the next task is classify the moves to get candidate move appropriate at the level the engine is supposed to play.

[op12no2]

So after generating all moves by multipv the next task is classify the moves to get candidate move appropriate at the level the engine is supposed to play.

Thanks Ferdy and for the link. Thanks also Sesse for the link and feedback. Interesting stuff and far more involved that I thought it would be

[Sesse]

I am currently working by multipv. Make your engine as strong as possible so that its multipv is more reliable. There is no need to make the engine weaker by weakening its eval, search etc.

This is the most common way (e.g. Stockfish does it, and also the Play Magnus app), and it generates pretty useless results. Your engine will frequently make “blunders” that are only blunders due to some insane tactical shot.

[Ajedrecista]

Hello Colin:

Some people are writing about Multi-PV and I think it is worth a try. I proposed a method back in 2016 which is based in Multi-PV move-picking involving PRNG. I am not aware that anyone has implemented it so far:

Re: Skill levels.

Re: Strategies for weaker play levels.

The second post is a reminder of the first one. The constant K can be multiplicative instead of dividing of my original post, just swapping the roles of K = 0 and K --> +infinity. Good luck with your implemetation!

Regards from Spain.

Ajedrecista.

[Ferdy]

Hello Colin:

Some people are writing about Multi-PV and I think it is worth a try. I proposed a method back in 2016 which is based in Multi-PV move-picking involving PRNG. I am not aware that anyone has implemented it so far:

Re: Skill levels.

I am looking at this:

Code: Select all

Imagine a position where the evals of all the possible moves from the side to move are:

(In pawns): d(i) = eval(i) - eval(best move) = {0, -0.1, -0.2, -0.2, -0.6, -2.9, -4, -4, -4, -9}

Prob(i) = 10^[d(i)/K]/SUM{10^[d(i)/K]}, with K > 0

// i = 1, 2, ..., moves - 1, moves; with moves = perft(1) of the position

F(0) = 0
F(i) = Prob(i) + SUM[Prob(j)], with j < i
// Of course: F(moves) = 1

Get r = a random number from a PRNG.

if [F(i-1) < r =< F(i)] then
  pick move i
end if

/*
Choose wisely. If I am right:
0 < r =< 1 ==> condition: F(i-1) < r =< F(i)
0 =< r < 1 ==> condition: F(i-1) =< r < F(i)
*/

and this.

Code: Select all

Rounding up to 1e-4:

-----------------------------

K = 1, basis 10:

Move   d(i)    P(i)     F(i)
  1     0.0   0.3276   0.3276
  2    -0.1   0.2457   0.5732
  3    -0.2   0.1842   0.7574
  4    -0.2   0.1842   0.9417
  5    -0.6   0.0583   0.9999
  6    -2.9   0.0001   1.0000
  7    -4.0   0.0000   1.0000
  8    -4.0   0.0000   1.0000
  9    -4.0   0.0000   1.0000
 10    -9.0   0.0000   1.0000

-----------------------------

K = 4, basis 10:

Move   d(i)    P(i)     F(i)
  1     0.0   0.2029   0.2029
  2    -0.1   0.1916   0.3945
  3    -0.2   0.1808   0.5753
  4    -0.2   0.1808   0.7561
  5    -0.6   0.1436   0.8998
  6    -2.9   0.0382   0.9380
  7    -4.0   0.0203   0.9583
  8    -4.0   0.0203   0.9786
  9    -4.0   0.0203   0.9989
 10    -9.0   0.0011   1.0000

-----------------------------

K = 0.5, basis 10:

Move   d(i)    P(i)     F(i)
  1     0.0   0.4016   0.4016
  2    -0.1   0.2534   0.6549
  3    -0.2   0.1599   0.8148
  4    -0.2   0.1599   0.9747
  5    -0.6   0.0253   1.0000
  6    -2.9   0.0000   1.0000
  7    -4.0   0.0000   1.0000
  8    -4.0   0.0000   1.0000
  9    -4.0   0.0000   1.0000
 10    -9.0   0.0000   1.0000

And this:

Code: Select all

Get r = a random number from a PRNG.

if [F(i-1) < r =< F(i)] then
  pick move i
end if

So ultimately the move is picked based on random number? Could you explain more about this method?

[Ferdy]

I am currently working by multipv. Make your engine as strong as possible so that its multipv is more reliable. There is no need to make the engine weaker by weakening its eval, search etc.

This is the most common way (e.g. Stockfish does it, and also the Play Magnus app), and it generates pretty useless results. Your engine will frequently make “blunders” that are only blunders due to some insane tactical shot.

This one is cool.

[op12no2]

Good luck with your implemetation!

Regards from Spain.

Ajedrecista.

Thanks Ajedrecista. I have not read any of these threads in full yet but they all look very interesting.

[Ajedrecista]

Hello Ferdinand:

I am looking at this:

Code: Select all

Imagine a position where the evals of all the possible moves from the side to move are:

(In pawns): d(i) = eval(i) - eval(best move) = {0, -0.1, -0.2, -0.2, -0.6, -2.9, -4, -4, -4, -9}

Prob(i) = 10^[d(i)/K]/SUM{10^[d(i)/K]}, with K > 0

// i = 1, 2, ..., moves - 1, moves; with moves = perft(1) of the position

F(0) = 0
F(i) = Prob(i) + SUM[Prob(j)], with j < i
// Of course: F(moves) = 1

Get r = a random number from a PRNG.

if [F(i-1) < r =< F(i)] then
  pick move i
end if

/*
Choose wisely. If I am right:
0 < r =< 1 ==> condition: F(i-1) < r =< F(i)
0 =< r < 1 ==> condition: F(i-1) =< r < F(i)
*/

and this.

Code: Select all

Rounding up to 1e-4:

-----------------------------

K = 1, basis 10:

Move   d(i)    P(i)     F(i)
  1     0.0   0.3276   0.3276
  2    -0.1   0.2457   0.5732
  3    -0.2   0.1842   0.7574
  4    -0.2   0.1842   0.9417
  5    -0.6   0.0583   0.9999
  6    -2.9   0.0001   1.0000
  7    -4.0   0.0000   1.0000
  8    -4.0   0.0000   1.0000
  9    -4.0   0.0000   1.0000
 10    -9.0   0.0000   1.0000

-----------------------------

K = 4, basis 10:

Move   d(i)    P(i)     F(i)
  1     0.0   0.2029   0.2029
  2    -0.1   0.1916   0.3945
  3    -0.2   0.1808   0.5753
  4    -0.2   0.1808   0.7561
  5    -0.6   0.1436   0.8998
  6    -2.9   0.0382   0.9380
  7    -4.0   0.0203   0.9583
  8    -4.0   0.0203   0.9786
  9    -4.0   0.0203   0.9989
 10    -9.0   0.0011   1.0000

-----------------------------

K = 0.5, basis 10:

Move   d(i)    P(i)     F(i)
  1     0.0   0.4016   0.4016
  2    -0.1   0.2534   0.6549
  3    -0.2   0.1599   0.8148
  4    -0.2   0.1599   0.9747
  5    -0.6   0.0253   1.0000
  6    -2.9   0.0000   1.0000
  7    -4.0   0.0000   1.0000
  8    -4.0   0.0000   1.0000
  9    -4.0   0.0000   1.0000
 10    -9.0   0.0000   1.0000

And this:

Code: Select all

Get r = a random number from a PRNG.

if [F(i-1) < r =< F(i)] then
  pick move i
end if

So ultimately the move is picked based on random number? Could you explain more about this method?

Thanks for your interest. My idea is doing a Multi-PV of all the possible moves in a given position and assign a probability to each one according to its score. The last step is generate a pseudorandom number to decide which move will be played, like spinning a casino roulette but with inequal probabilities (favouring the best moves with the help of the constant but without discarding the rest of moves) unless the constant of my method is chosen in order to provide a random mover engine (i.e. equal chances for each move).

I have some doubts about how it works with for example a win-or-draw position where the Multi-PV finds a mate in 30 (let's say eval > 327) and the rest of moves with 0.00 evals. When there are evals ranging some pawns (10 for example), the choices can be realistic with a given constant, but with such extreme cases I think that a tweak on the constant is needed (for example an if clause to change the constant if eval differences are so high); because otherwise it will play such long mates almost for sure unless the random mover constant or a near one is the chosen one (somewhat related to Steinar's mention of insane tactical shots).

OTOH, my method should be valid regardless of the position, I mean, if the engine has a winning position (moves with evals +6, +5.5, +3, ...) or a losing position (-7, -7, -8, -9, ...). This is why I compute relative evals with respect of the best move.

The probability of each move is assigned in a logistic-like curve although other curves might be tried. The constant K favours best moves or not depending on its value, hence relying on a typical Single-PV search if K aims for not randomness (100% of play the best move; if there are N moves with the same exact eval, then 100%/N for each of these moves), full randomness (1/M probability for each move 1, ..., M) or something in between, which must be tuned if there is an interest in knowing Elo differences between the Single-PV search and an arbitrary value of K. Please note that doing Multi-PV and then choose the best move is slower than a Single-PV search, so this possibility is only theoretical but not advisable. If you want to pick the best move, do not waste time with Multi-PV.

Answering your question: yes, the last step is use a pseudorandom number to pick the move. Once you have built the casino roulette with probabilities for each move, it does the task of spin the roulette and fall in a region of the roulette, which is linked to a move.

Please imagine the position after 1.- e4, e5; 2.- Nf3, d6; 3.- d4, f6; 4.- dxe5, dxe5; 5.- Qxd8+. Black has only two moves now: 5.- ..., Kxd8 and 5.- ..., Kf7?? SF 12 at depth 21 returns -0.20 and -23.05 for the side to move. The obvious move is KxQ but toying with the constant K you can modify the probabilities. With the formulas of my original post, K = 4 makes a 99.999806% of KxQ being selected; raising to K = 100, this probability is ca. 62.86% only. Although such big K is near random mover level in usual positions. So there should not be any problems with immediate queen recaptures.

In the Elephant Trap: 1.- d4, d5; 2.- c4, e6; 3.- Nc3, Nf6; 4.- Bg5, Nbd7; 5.- cxd5, exd5; 6.- Nxd5?? Depth 18 of SF 12:

Code: Select all

Move    Eval from the side to move (pawns)
------------------------------------------
Nxd5    +4.12
  c6    -1.25
  h6    -1.34
  a6    -1.39
  c5    -1.40
 Nb6    -1.58
 Nc5    -1.68
 Bd6    -1.70
  a5    -1.79
 Be7    -2.10
  b6    -2.11
  b5    -2.28
 Nb8    -2.30
 Rb8    -2.35
 Ne5    -2.46
 Rg8    -2.50
  g6    -2.56
  h5    -2.99
 Ba3    -5.11
 Bb4+   -5.55
 Bc5    -6.28
 Qe7    -9.17
 Ne4    -9.21
 Ng4    -9.32
 Ng8    -9.37
 Nh5    -9.40

6.- ..., Nxd5 has a probability of being played of ca. 99.81% with K = 1.5, ca. 98.27% with K = 2, ca. 85.47% with K = 3 and ca. 64.95% with K = 4 (other thing is if the pseudorandom number finally picks NxN). Regarding the nine moves with evals > -2: ca. 99.96% with K = 1.5, ca. 99.49% with K = 2, ca. 94.60% with K = 3 and ca. 85.32% with K = 4. In general, it would play suboptimal moves or even the best move but normally not heavy, heavy blunders like 6.- ..., Qe7??

In a quiet position without tactical shots, such as 1.- e4, c5; Nf3, d6; 3.- d4, cxd4; 4.- Nxd4, Nf6; 5.- Nc3, a6; 6.- f4. Depth 17 of SF 12:

Code: Select all

Move    Eval from the side to move (pawns)
------------------------------------------
  e5    -0.14
  g6    -0.25
Nbd7    -0.43
 Qc7    -0.45
 Nc6    -0.50
  e6    -0.52
 Qa5    -0.55
 Qb6    -0.58
  h5    -0.81
  h6    -0.84
 Bg4    -0.95
 Qd7    -0.97
 Bd7    -1.07
  b5    -1.15
  d5    -1.28
Nfd7    -1.29
 Rg8    -1.48
 Ra7    -1.48
 Ng4    -1.71
 Ng8    -1.74
  b6    -2.25
  g5    -3.13
 Be6    -3.19
Nxe4    -3.41
 Kd7    -4.35
 Bh3    -5.01
  a5    -5.13
 Nd5    -5.48
 Nh5    -5.51
 Bf5    -5.78

The twelve moves with evals > -1 have probabilities of being played of ca. 83.47% with K = 1.5, ca. 77.89% with K = 2, ca. 70.20% with K = 3 and ca. 65.08% with K = 4. I think these probabilities can be reasonable within club play level. However, the top move has a probability of being played of only ca. 12.75% with K = 1.5, ca. 10.36% with K = 2, ca. 8.06% with K = 3 and ca. 6.92% with K = 4. This is because of a bunch of 'similar' moves regarding eval (similar in the eyes of a club player, not SF). I think that OTB games at slow TC and without time trouble with both players rated under 2000 Elo feature many moves with 0.6, 1, 1.3, ... pawn loss in SF eval when compared with the top move, but much less times with heavy blunders of -5, -9 and so on.

The whole point of my method is making the top move still playable, not only playing the second or third move, because there can be positions like the first one that is a queen recapture or a queen loss as the only legal moves.

Regards from Spain.

Ajedrecista.

[Ferdy]

Hello Ferdinand:

I am looking at this:

Code: Select all

Imagine a position where the evals of all the possible moves from the side to move are:

(In pawns): d(i) = eval(i) - eval(best move) = {0, -0.1, -0.2, -0.2, -0.6, -2.9, -4, -4, -4, -9}

Prob(i) = 10^[d(i)/K]/SUM{10^[d(i)/K]}, with K > 0

// i = 1, 2, ..., moves - 1, moves; with moves = perft(1) of the position

F(0) = 0
F(i) = Prob(i) + SUM[Prob(j)], with j < i
// Of course: F(moves) = 1

Get r = a random number from a PRNG.

if [F(i-1) < r =< F(i)] then
  pick move i
end if

/*
Choose wisely. If I am right:
0 < r =< 1 ==> condition: F(i-1) < r =< F(i)
0 =< r < 1 ==> condition: F(i-1) =< r < F(i)
*/

and this.

Code: Select all

Rounding up to 1e-4:

-----------------------------

K = 1, basis 10:

Move   d(i)    P(i)     F(i)
  1     0.0   0.3276   0.3276
  2    -0.1   0.2457   0.5732
  3    -0.2   0.1842   0.7574
  4    -0.2   0.1842   0.9417
  5    -0.6   0.0583   0.9999
  6    -2.9   0.0001   1.0000
  7    -4.0   0.0000   1.0000
  8    -4.0   0.0000   1.0000
  9    -4.0   0.0000   1.0000
 10    -9.0   0.0000   1.0000

-----------------------------

K = 4, basis 10:

Move   d(i)    P(i)     F(i)
  1     0.0   0.2029   0.2029
  2    -0.1   0.1916   0.3945
  3    -0.2   0.1808   0.5753
  4    -0.2   0.1808   0.7561
  5    -0.6   0.1436   0.8998
  6    -2.9   0.0382   0.9380
  7    -4.0   0.0203   0.9583
  8    -4.0   0.0203   0.9786
  9    -4.0   0.0203   0.9989
 10    -9.0   0.0011   1.0000

-----------------------------

K = 0.5, basis 10:

Move   d(i)    P(i)     F(i)
  1     0.0   0.4016   0.4016
  2    -0.1   0.2534   0.6549
  3    -0.2   0.1599   0.8148
  4    -0.2   0.1599   0.9747
  5    -0.6   0.0253   1.0000
  6    -2.9   0.0000   1.0000
  7    -4.0   0.0000   1.0000
  8    -4.0   0.0000   1.0000
  9    -4.0   0.0000   1.0000
 10    -9.0   0.0000   1.0000

And this:

Code: Select all

Get r = a random number from a PRNG.

if [F(i-1) < r =< F(i)] then
  pick move i
end if

So ultimately the move is picked based on random number? Could you explain more about this method?

Thanks for your interest. My idea is doing a Multi-PV of all the possible moves in a given position and assign a probability to each one according to its score. The last step is generate a pseudorandom number to decide which move will be played, like spinning a casino roulette but with inequal probabilities (favouring the best moves with the help of the constant but without discarding the rest of moves) unless the constant of my method is chosen in order to provide a random mover engine (i.e. equal chances for each move).

I have some doubts about how it works with for example a win-or-draw position where the Multi-PV finds a mate in 30 (let's say eval > 327) and the rest of moves with 0.00 evals. When there are evals ranging some pawns (10 for example), the choices can be realistic with a given constant, but with such extreme cases I think that a tweak on the constant is needed (for example an if clause to change the constant if eval differences are so high); because otherwise it will play such long mates almost for sure unless the random mover constant or a near one is the chosen one (somewhat related to Steinar's mention of insane tactical shots).

OTOH, my method should be valid regardless of the position, I mean, if the engine has a winning position (moves with evals +6, +5.5, +3, ...) or a losing position (-7, -7, -8, -9, ...). This is why I compute relative evals with respect of the best move.

The probability of each move is assigned in a logistic-like curve although other curves might be tried. The constant K favours best moves or not depending on its value, hence relying on a typical Single-PV search if K aims for not randomness (100% of play the best move; if there are N moves with the same exact eval, then 100%/N for each of these moves), full randomness (1/M probability for each move 1, ..., M) or something in between, which must be tuned if there is an interest in knowing Elo differences between the Single-PV search and an arbitrary value of K. Please note that doing Multi-PV and then choose the best move is slower than a Single-PV search, so this possibility is only theoretical but not advisable. If you want to pick the best move, do not waste time with Multi-PV.

Answering your question: yes, the last step is use a pseudorandom number to pick the move. Once you have built the casino roulette with probabilities for each move, it does the task of spin the roulette and fall in a region of the roulette, which is linked to a move.

Please imagine the position after 1.- e4, e5; 2.- Nf3, d6; 3.- d4, f6; 4.- dxe5, dxe5; 5.- Qxd8+. Black has only two moves now: 5.- ..., Kxd8 and 5.- ..., Kf7?? SF 12 at depth 21 returns -0.20 and -23.05 for the side to move. The obvious move is KxQ but toying with the constant K you can modify the probabilities. With the formulas of my original post, K = 4 makes a 99.999806% of KxQ being selected; raising to K = 100, this probability is ca. 62.86% only. Although such big K is near random mover level in usual positions. So there should not be any problems with immediate queen recaptures.

In the Elephant Trap: 1.- d4, d5; 2.- c4, e6; 3.- Nc3, Nf6; 4.- Bg5, Nbd7; 5.- cxd5, exd5; 6.- Nxd5?? Depth 18 of SF 12:

Code: Select all

Move    Eval from the side to move (pawns)
------------------------------------------
Nxd5    +4.12
  c6    -1.25
  h6    -1.34
  a6    -1.39
  c5    -1.40
 Nb6    -1.58
 Nc5    -1.68
 Bd6    -1.70
  a5    -1.79
 Be7    -2.10
  b6    -2.11
  b5    -2.28
 Nb8    -2.30
 Rb8    -2.35
 Ne5    -2.46
 Rg8    -2.50
  g6    -2.56
  h5    -2.99
 Ba3    -5.11
 Bb4+   -5.55
 Bc5    -6.28
 Qe7    -9.17
 Ne4    -9.21
 Ng4    -9.32
 Ng8    -9.37
 Nh5    -9.40

6.- ..., Nxd5 has a probability of being played of ca. 99.81% with K = 1.5, ca. 98.27% with K = 2, ca. 85.47% with K = 3 and ca. 64.95% with K = 4 (other thing is if the pseudorandom number finally picks NxN). Regarding the nine moves with evals > -2: ca. 99.96% with K = 1.5, ca. 99.49% with K = 2, ca. 94.60% with K = 3 and ca. 85.32% with K = 4. In general, it would play suboptimal moves or even the best move but normally not heavy, heavy blunders like 6.- ..., Qe7??

In a quiet position without tactical shots, such as 1.- e4, c5; Nf3, d6; 3.- d4, cxd4; 4.- Nxd4, Nf6; 5.- Nc3, a6; 6.- f4. Depth 17 of SF 12:

Code: Select all

Move    Eval from the side to move (pawns)
------------------------------------------
  e5    -0.14
  g6    -0.25
Nbd7    -0.43
 Qc7    -0.45
 Nc6    -0.50
  e6    -0.52
 Qa5    -0.55
 Qb6    -0.58
  h5    -0.81
  h6    -0.84
 Bg4    -0.95
 Qd7    -0.97
 Bd7    -1.07
  b5    -1.15
  d5    -1.28
Nfd7    -1.29
 Rg8    -1.48
 Ra7    -1.48
 Ng4    -1.71
 Ng8    -1.74
  b6    -2.25
  g5    -3.13
 Be6    -3.19
Nxe4    -3.41
 Kd7    -4.35
 Bh3    -5.01
  a5    -5.13
 Nd5    -5.48
 Nh5    -5.51
 Bf5    -5.78

The twelve moves with evals > -1 have probabilities of being played of ca. 83.47% with K = 1.5, ca. 77.89% with K = 2, ca. 70.20% with K = 3 and ca. 65.08% with K = 4. I think these probabilities can be reasonable within club play level. However, the top move has a probability of being played of only ca. 12.75% with K = 1.5, ca. 10.36% with K = 2, ca. 8.06% with K = 3 and ca. 6.92% with K = 4. This is because of a bunch of 'similar' moves regarding eval (similar in the eyes of a club player, not SF). I think that OTB games at slow TC and without time trouble with both players rated under 2000 Elo feature many moves with 0.6, 1, 1.3, ... pawn loss in SF eval when compared with the top move, but much less times with heavy blunders of -5, -9 and so on.

The whole point of my method is making the top move still playable, not only playing the second or third move, because there can be positions like the first one that is a queen recapture or a queen loss as the only legal moves.

Regards from Spain.

Ajedrecista.

Thanks for some explanations I will try to experiment with it.