Further weaknesses

[Lyudmil Tsvetkov]

A bit more SF weak spots.

[Lyudmil Tsvetkov]

From the many games I looked at recently, a considerable percentage contained situations where SF underestimates the burden doubled isolated pawns constitute.

Belo one such game:

[pgn][MLNrOfMoves "70"]
[MLFlags "010100"]
[Event "4 Minutes/Game"]
[Site "2 SF 4 min gauntlet, HP-PC"]
[Date "2014.04.08"]
[Round "9.2"]
[White "Stockfish1404061153x64"]
[Black "Houdini4ProC0x64"]
[Result "0-1"]

1. c4 e5 2. Nc3 Nf6 3. Nf3 Nc6 4. e3 Bb4 5. Qc2 d6 6. d3 O-O 7. Bd2 Re8 8. Be2
a5 9. O-O {0.00/22 6s} 9... Bf5 {+0.05/19 4s} 10. Nh4 {0.00/23 2s} 10... Bc8
{0.00/21 10s (Bg4)} 11. a3 {+0.13/21 6s (Nf3)} 11... Bxc3 {+0.08/19 4s} 12. Bxc3
{0.00/23 12s} 12... e4 {+0.01/19 2s (a4)} 13. g3 {+0.08/20 3s} 13... exd3
{+0.04/18 4s} 14. Bxd3 {+0.08/21 2s} 14... Ne5 {+0.05/19 8s} 15. Be2
{+0.22/21 26s} 15... b6 {+0.06/18 6s (Bd7)} 16. e4 {+0.27/21 3s (Rad1)} 16...
Qd7 {+0.08/17 4s} 17. f3 {+0.27/21 6s (Nf3)} 17... Qc6 {+0.06/18 8s} 18. Ng2
{+0.22/22 2s (Bd4)} 18... Bh3 {+0.07/18 8s} 19. Bd4 {+0.22/23 2s (Rfd1)} 19...
Bxg2 {+0.16/18 6s (a4)} 20. Kxg2 {+0.22/24 2s} 20... a4 {+0.09/18 5s} 21. Rad1
{+0.22/24 2s (Rfe1)} 21... Ned7 {+0.11/18 6s} 22. Rfe1 {+0.22/25 2s (Rc1)}
22... Re6 {+0.15/19 2s (h6)} 23. Bf2 {+0.22/22 2s (Qc3)} 23... Rae8
{+0.05/18 4s (h6)} 24. Qc3 {+0.12/22 2s} 24... Ne5 {+0.09/18 5s (Nc5)} 25. Rc1
{+0.22/22 2s (Rd5)} 25... h5 {0.00/17 3s (h6)} 26. Qd2 {+0.30/20 5s} 26... Qa8
{+0.01/18 11s (Nfd7)} 27. Rcd1 {+0.30/20 10s (h3)} 27... h4 {0.00/17 3s (Qc6)}
28. Qe3 {+0.28/20 7s (Qf4)} 28... Qc6 {+0.01/17 8s (Ng6)} 29. Rc1
{+0.31/19 3s (Qf4)} 29... Qa8 {0.00/17 1s (Ng6)} 30. gxh4 {+0.53/19 4s (Rcd1)}
30... Ng6 {-0.04/17 5s (Nh5)} 31. Qc3 {+0.31/21 8s (Kg1)} 31... Nh5
{-0.16/17 2s} 32. Kh1 {+0.28/22 2s} 32... Ngf4 {-0.20/18 2s (Nhf4)} 33. Bf1
{+0.46/18 2s} 33... Rg6 {-0.16/18 1s} 34. Qc2 {+0.43/22 4s} 34... Re5
{-0.17/19 1s (Kf8)} 35. Re3 {+0.46/21 3s (Bd4)} 35... Qc8 {-0.15/18 6s (Ra5)}
36. b4 {+0.44/23 14s} 36... Nh3 {-0.22/18 4s (axb3)} 37. c5 {+0.25/21 2s} 37...
bxc5 {-0.29/18 1s} 38. Rc3 {0.00/22 5s} 38... Nxf2+ {-0.22/18 2s} 39. Qxf2
{0.00/1 0s} 39... Nf4 {-0.24/19 1s} 40. bxc5 {0.00/21 2s} 40... d5 {-0.28/19 1s}
41. Qd2 {-0.05/21 2s} 41... Rf6 {-0.28/17 0s} 42. exd5 {0.00/22 4s (Re3)} 42...
Nxd5 {-0.34/16 1s} 43. Rd3 {0.00/22 3s} 43... c6 {-0.37/18 2s} 44. Qf2
{-0.31/21 5s (Rc4)} 44... Qe8 {-0.48/17 5s} 45. Rdd1 {-0.36/21 4s (Qd2)} 45...
Ne3 {-0.52/17 1s} 46. Rd2 {-0.44/22 4s (Rd4)} 46... Nxf1 {-0.56/18 1s} 47. Rxf1
{-0.44/23 0s} 47... Re3 {-0.61/19 2s (Qe7)} 48. Rd6 {-0.48/19 1s (Rd8)} 48...
Rfxf3 {-1.04/16 1s (Rf5)} 49. Qxf3 {-0.34/20 2s} 49... Rxf3 {-0.99/15 0s} 50.
Rxf3 {-0.41/22 1s} 50... Qe4 {-0.86/18 1s (Qe2)} 51. Kg2 {-0.50/19 0s (Rd8+)}
51... Qg4+ {-1.13/18 1s (Qc2+)} 52. Rg3 {-0.75/20 1s} 52... Qxh4
{-1.13/16 0s (Qe2+)} 53. Rc3 {-0.83/19 1s} 53... Qe4+ {-1.32/17 0s} 54. Kf2
{-1.13/20 2s} 54... g5 {-1.59/17 1s} 55. Rdd3 {-1.36/20 3s (h3)} 55... Kg7
{-1.82/18 1s (f6)} 56. h3 {-1.43/20 1s} 56... Kg6 {-1.82/16 0s} 57. Rf3
{-1.49/21 1s (Re3)} 57... Qb1 {-1.91/19 1s (f5)} 58. Ke2 {-1.49/21 1s (Rfd3)}
58... f5 {-1.80/21 1s (Qb2+)} 59. Kd2 {-1.49/23 0s (Rfd3)} 59... Qb2+
{-1.94/21 1s (f4)} 60. Rc2 {-1.49/24 0s} 60... Qd4+ {-1.86/22 0s} 61. Rd3
{-1.49/27 0s} 61... Qf2+ {-1.94/22 0s (Qf4+)} 62. Kc1 {-1.49/24 0s (Kd1)} 62...
Qe1+ {-1.93/17 2s (Qf1+)} 63. Kb2 {-1.49/24 0s} 63... f4 {-2.47/18 1s (Qf1)} 64.
Rcc3 {-1.49/23 0s} 64... Kf5 {-2.93/18 1s (Qe5)} 65. Kc2 {-2.37/23 4s} 65...
Qe2+ {-3.26/17 0s (Ke5)} 66. Kc1 {-2.51/17 0s (Kb1)} 66... Ke4 {-4.29/18 0s} 67.
Rd8 {-3.06/18 3s (Rd7)} 67... f3 {-4.90/17 0s (Qf2)} 68. Re8+ {-3.60/19 0s}
68... Kd4 {-4.58/16 0s} 69. Rxe2 {-6.67/25 0s} 69... fxe2 {-4.21/15 0s} 70. Kd2
{-7.38/25 0s} 70... e1Q+ {-4.02/14 0s (e1B+)} 0-1
[/pgn]

[d]q3r1k1/2p2pp1/1p1prn2/4n3/p1P1P2p/P3QPP1/1P2BBKP/2R1R3 w - - 0 30
SF now captures 30. gh4.

[d]q3r1k1/2p2pp1/1p1pr3/7n/p1P1Pn1P/P1Q2P2/1P3B1P/2R1RB1K b - - 0 33
SF sees half a pawn white advantage, Houdini thinks black is a bit better. I think Houdini gets it right in this situation. SF underestimates the penalty for pawns that are doubled and isolated at the same time.

[d]4q1k1/5pp1/2p2r2/2Pnr3/p6P/P2R1P2/5Q1P/2R2B1K w - - 0 45
SF starts seeing it is worse, but now it is already too late. Look at the h2 and h4 pawns: thay are less than a pawn, as different squares close to the king are weakened. Do you remember also SF playing 48.gh4 in the recent lost TCEC game against Houdini? There the position was already bad, but still gh4 was another weakening.

In light of the above, my suggestion would be the following: increase slightly penalties for doubled isolated pawns. Please note that you can not try increasing the penalties for all doubled and isolated pawns, as this is not going to work, but only for the doubled pawns that are isolated at the same time (or maybe vice versa, only for the isolated pawns that are doubled at the same time), like h2 and h4.

Comments, output appreciated.

[Lyudmil Tsvetkov]

Below an interesting game where SF allows the opponent to restrict its own king mobility in the endgame to absolute zero for quite some time.

[pgn][MLNrOfMoves "65"]
[MLFlags "000100"]
[Event "4 Minutes/Game"]
[Site "2 SF 4 min gauntlet, HP-PC"]
[Date "2014.04.08"]
[Round "18.3"]
[White "Stockfish140407IPxx64"]
[Black "KomodoTCECrx64"]
[Result "0-1"]

1. e4 c5 2. Nf3 d6 3. Bb5+ Nd7 4. O-O a6 5. Bxd7+ Bxd7 6. Nc3 h6 7. b3 e5 8. a4
Nf6 9. a5 {+0.08/25 5s} 9... Bc6 {+0.13/21 7s (Be7)} 10. d3 {+0.06/26 6s} 10...
Be7 {+0.16/21 4s} 11. Bb2 {+0.06/25 2s} 11... O-O {+0.13/21 6s} 12. Re1
{+0.04/24 3s} 12... Re8 {+0.16/20 9s (Ne8)} 13. Nd2 {+0.18/21 4s} 13... d5
{+0.17/20 4s (Rc8)} 14. exd5 {+0.15/23 3s} 14... Nxd5 {+0.16/18 1s} 15. Rxe5
{+0.25/26 4s (Nxd5)} 15... Bf6 {+0.13/21 7s} 16. Rxe8+ {+0.22/27 10s} 16... Qxe8
{+0.26/20 5s} 17. Nde4 {+0.22/28 4s} 17... Bd4 {+0.19/20 4s (Be5)} 18. Nxd5
{+0.25/26 3s} 18... Bxb2 {+0.11/20 2s} 19. Nc7 {+0.20/27 4s} 19... Qe5
{+0.18/20 3s} 20. Nxa8 {+0.20/28 2s} 20... Bxa1 {+0.17/21 5s} 21. Nb6
{+0.20/28 2s} 21... Bd4 {+0.13/22 7s} 22. Nc4 {+0.23/26 2s} 22... Qd5
{+0.14/22 10s} 23. Qf1 {+0.22/27 2s} 23... g6 {+0.16/20 5s} 24. Ng3
{+0.22/25 2s (h3)} 24... Qe6 {+0.14/20 8s (Kg7)} 25. h3 {+0.22/24 7s (Ne2)}
25... Kg7 {+0.16/21 4s (h5)} 26. Ne2 {+0.20/22 3s} 26... Bf6 {+0.19/19 6s} 27.
Qe1 {+0.18/23 3s (Qd1)} 27... h5 {+0.18/18 2s (Bb5)} 28. Qd2 {+0.25/24 4s (f3)}
28... Qd5 {+0.37/19 5s} 29. f3 {+0.28/24 4s} 29... Bg5 {+0.37/20 5s (Bb5)} 30.
Qe1 {+0.32/24 6s} 30... Bb5 {+0.29/19 5s (Bd8)} 31. Qg3 {+0.26/23 4s (Nb6)}
31... h4 {+0.12/17 1s (Bd8)} 32. Nb6 {+0.32/24 3s (Qb8)} 32... Be3+
{+0.16/20 2s (Qf5)} 33. Kh1 {+0.30/26 4s (Kf1)} 33... hxg3 {+0.07/22 4s (Qg5)}
34. Nxd5 {+0.21/25 4s} 34... Bf2 {-0.14/22 5s} 35. Ndc3 {+0.17/26 3s} 35... Bc6
{-0.10/21 5s} 36. Ne4 {+0.23/26 2s} 36... Bxe4 {-0.33/20 1s (f5)} 37. dxe4
{+0.08/28 3s} 37... Kf6 {-0.42/21 1s} 38. c3 {+0.14/29 2s (Nc1)} 38... Ke5
{-0.70/18 0s} 39. Nc1 {0.00/31 6s} 39... Kd6 {-0.70/18 0s} 40. Nd3 {0.00/32 1s}
40... c4 {-0.66/19 1s} 41. bxc4 {-0.17/32 11s} 41... b6 {-0.70/19 0s (Be3)} 42.
axb6 {-0.22/26 2s} 42... Bxb6 {-0.56/19 0s} 43. Nb2 {-0.36/30 2s} 43... a5
{-1.10/22 4s} 44. f4 {-0.54/32 9s (Na4)} 44... Be3 {-1.10/21 1s} 45. f5
{-0.60/35 4s} 45... g5 {-1.14/22 2s (gxf5)} 46. c5+ {-0.68/28 3s} 46... Kxc5
{-1.28/20 0s} 47. e5 {-0.94/31 6s} 47... Kd5 {-1.44/20 1s} 48. e6
{-0.96/32 1s (Nd3)} 48... fxe6 {-1.44/21 0s} 49. f6 {-1.03/31 1s} 49... e5
{-1.44/24 2s} 50. f7 {-1.06/32 1s} 50... Bc5 {-1.44/24 0s} 51. Na4 {-1.12/32 2s}
51... Bf8 {-2.81/24 11s (Bd6)} 52. Nb2 {-1.38/26 1s} 52... Ke6
{-2.51/21 0s (e4)} 53. Kg1 {-1.54/29 1s} 53... Kxf7 {-2.97/21 1s (e4)} 54. Kf1
{-1.69/29 1s} 54... Ke6 {-2.97/20 0s (e4)} 55. Ke2 {-1.82/29 1s} 55... Kd5
{-3.11/19 1s (e4)} 56. Kd3 {-1.96/29 2s} 56... e4+ {-3.03/20 0s} 57. Ke3
{-2.41/31 4s} 57... Bc5+ {-3.74/19 0s} 58. Kd2 {-2.62/31 1s} 58... Bg1
{-3.90/21 1s (Bd6)} 59. Kc1 {-2.68/33 1s} 59... Be3+ {-4.20/20 0s (Bb6)} 60.
Kd1 {-3.49/27 1s (Kc2)} 60... Ba7 {-4.17/20 0s} 61. Kd2 {-4.28/30 3s} 61... Bc5
{-4.60/21 0s} 62. Kd1 {-4.40/30 1s} 62... Be3 {-5.03/23 3s} 63. Kc2
{-4.64/30 1s (Na4)} 63... Bf2 {-5.03/22 0s} 64. c4+ {-5.35/30 3s (Kd1)} 64...
Kd4 {-5.35/22 0s} 65. c5 {-7.69/31 5s (Na4)} 65... Kxc5 {-5.97/19 0s} 0-1
[/pgn]

[d]8/1p3pk1/p5p1/P1p5/4P3/1P3PpP/2P1NbP1/7K b - - 0 37
For a long time now SF thinks white is a bit better, or later that the position is even, but Komodo rightly is convinced black is more than half a pawn better. Look at the h1 white king: it looks comical, does not have a single available safe mobility square. You can not play without a king in the endgame.

[d]8/5p2/3k2p1/p7/2P1PP2/2P1b1pP/1N4P1/7K w - - 0 45
Only now SF sees it is lost. Black plays with king and bishop vs the lone white knight.

In light of the above, my suggestion would be the following: give a penalty, maybe some 20cps, for a zero safe mobility king in the endgame. This could solve cases like this one.

I also observed in quite many games that SF sometimes allows the enemy king to penetrate in its territory much too easy, so that I suppose psqt values for the king on the 5th and 6th ranks in the endgame could be increased somewhat.

Comments, output appreciated.
I think this is an interesting game to comment on.

[Ferdy]

[d]8/1p3pk1/p5p1/P1p5/4P3/1P3PpP/2P1NbP1/7K b - - 0 37
Check the king pst at this corner, this can be tuned. But even then king dynamic play has to be prioritized.

[lucasart]

In light of the above, my suggestion would be the following: give a penalty, maybe some 20cps, for a zero safe mobility king in the endgame. This could solve cases like this one.

I already tried king safe mobility in the endgame. It failed:
http://tests.stockfishchess.org/tests/v ... 4b84c1a754

[Lyudmil Tsvetkov]

[d]8/1p3pk1/p5p1/P1p5/4P3/1P3PpP/2P1NbP1/7K b - - 0 37
Check the king pst at this corner, this can be tuned. But even then king dynamic play has to be prioritized.

Hi Ferdinand.

I think king pst for h1 is OK, but there are really grave king mobility problems.

What I have observed in a multitude of SF blitz games is however that the engine much too often underestimates enemy king penetration. I think it is absolutely necessary for a good king play in the endgame to give bigger bonus to 5th and 6th ranks, at least central ones. In the 100 games I have looked at recently, there were at least 5 instances of such behaviour. This is too much, bearing in mind that only a third of the games featured endgames.

Sorry guys, if I am bolding some word sequences, it is only to highlight the importance of some logical factor I have observed on the basis of hundreds of browsed games. I think repeated mistakes point to an existing problem. When there is only one game with such a mistake, this might be a fluke, but when you see it repeatedly, this is something serious. I think that browsing through games is a good way of finding weaknesses.

[Lyudmil Tsvetkov]

In light of the above, my suggestion would be the following: give a penalty, maybe some 20cps, for a zero safe mobility king in the endgame. This could solve cases like this one.

I already tried king safe mobility in the endgame. It failed:
http://tests.stockfishchess.org/tests/v ... 4b84c1a754

This was the Senpai patch.

My suggestion is for a penalty for zero safe mobility for the king in the endgame, which is different. So far all low mobility patches tested in STF scored positively, I am certain the king case will be no different.

Sorry Lucas, all I want is to help, I am browsing even through SF code, which is very difficult for me, if I had been able to commit pacthes myself, I would have done so. Unfortunately, all I can do at the moment is suggest some rough ideas.
I know this might be a bit frustrating to the people such ideas are suggested, very possibly so, but, on the other hand, those ideas are based on observations from hundreds of SF games. I almost never post an idea, if I have observed a particular SF failure in just a single game, only when a repeated behaviour is spotted.

I feel that I kind of possibly start annoying people, but, at the same time, I regard SF as a team scientific effort, with people contributing in different ways. Well, I can not help with programming, but probably a bit as an chess/eval enthusiast. I think a good way of eliminating weaknesses is fixing repeated failures in games.

At the end of the day, what counts is the end users that are happy with continuous SF improvement. I think one can surrender a bit more conservative approach, if this will lead to improvement in strength. 20/50/100 main SF developers and their personal concerns should be secondary to the satisfaction of 100 000 users. At least that is my understanding.

[Ferdy]

In light of the above, my suggestion would be the following: give a penalty, maybe some 20cps, for a zero safe mobility king in the endgame. This could solve cases like this one.

I already tried king safe mobility in the endgame. It failed:
http://tests.stockfishchess.org/tests/v ... 4b84c1a754

Code: Select all

Bitboard mobility = ei.attackedBy[Us][KING] & ~pos.pieces(Us, PAWN) & ~ei.attackedBy[Them][PAWN];

That might not be enough.
What about

Code: Select all

ei.attackedBy[Us][KING] & empty_sq & ~ei.attackedBy[Them][all_pieces_including_king];

If this is 0 give penalty, but don't give bonus if this is not 0.

[Henk]

In light of the above, my suggestion would be the following: give a penalty, maybe some 20cps, for a zero safe mobility king in the endgame. This could solve cases like this one.

I already tried king safe mobility in the endgame. It failed:
http://tests.stockfishchess.org/tests/v ... 4b84c1a754

If Lyudmil Tsvetkov is getting short of good ideas he may ask Lucas what he implemented that did not work. So Lyudmil can write a nice post about it why this idea should work. Illustrating it with examples as usual. So we can learn from "What would Lyudmil Tsvetkov do ?" which means "Don't try this at home".

[lucasart]

In light of the above, my suggestion would be the following: give a penalty, maybe some 20cps, for a zero safe mobility king in the endgame. This could solve cases like this one.

I already tried king safe mobility in the endgame. It failed:
http://tests.stockfishchess.org/tests/v ... 4b84c1a754

Code: Select all

Bitboard mobility = ei.attackedBy[Us][KING] & ~pos.pieces(Us, PAWN) & ~ei.attackedBy[Them][PAWN];

That might not be enough.
What about

Code: Select all

ei.attackedBy[Us][KING] & empty_sq & ~ei.attackedBy[Them][all_pieces_including_king];

If this is 0 give penalty, but don't give bonus if this is not 0.

The endgame component of the eval applies in the endgame. When there are only kings and pawns left. With linear interpolation obviously. So it makes no sense to include attacks of other pieces. It's against the principle of eval orthogonality. These attacks are accounted for in king safety, mobility etc. But have nothing to do with king endgame mobility.