Oddity around depths 7-8 with Stockfish 6 & 7

[lucasart]

But that is only a cosmetic fix. The reason why you don't want this behavior is that it corrupts your search in the billions of d<=8 nodes that you visit during a d=30 search.

Yes. And even the correct "fix", which is to disable futility pruning at PV nodes (step 7 and step 13 in search()), is only cosmetic. Sure, you don't do pruning at PV nodes, but the PV comes from what was searched at non PV nodes, where SF does so much "almost random" pruning...

For me this is not a problem, so there's no point in trying to solve it. Pruning is quasi random, and makes lots of mistakes. Like it or not, but that's what makes SF (and modern engines generally) so strong.

Now if people prefer to hide this fact under the carpet, by preventing the user from seeing the horrible blunders that SF can play at shallow non-PV nodes...

[Joerg Oster]

But that is only a cosmetic fix. The reason why you don't want this behavior is that it corrupts your search in the billions of d<=8 nodes that you visit during a d=30 search.

Yes, it is.
And considering how often this topic has shown up in the past, I just wanted to give an easy "solution".

[hgm]

Yes. And even the correct "fix", which is to disable futility pruning at PV nodes (step 7 and step 13 in search()), is only cosmetic. Sure, you don't do pruning at PV nodes, but the PV comes from what was searched at non PV nodes, where SF does so much "almost random" pruning...

Note that I did not propose anywhere to treat PV nodes different than non-PV nodes. If you want to solve this it should of course be solved in every node. I am pretty sure a significant amount of strength would be gained if you would stongly reduce the amount of garbage scores returned by d=8 nodes, at no significant cost.

[gladius]

Yes. And even the correct "fix", which is to disable futility pruning at PV nodes (step 7 and step 13 in search()), is only cosmetic. Sure, you don't do pruning at PV nodes, but the PV comes from what was searched at non PV nodes, where SF does so much "almost random" pruning...

Note that I did not propose anywhere to treat PV nodes different than non-PV nodes. If you want to solve this it should of course be solved in every node. I am pretty sure a significant amount of strength would be gained if you would stongly reduce the amount of garbage scores returned by d=8 nodes, at no significant cost.

I've given this another shot with the cheapest thing I can think of: https://github.com/glinscott/Stockfish/ ... ...906af77

Lets see how it does! http://tests.stockfishchess.org/tests/v ... 0247cdf7ca

My previous experience here is that any changes like this have failed incredibly badly. But who knows, maybe this is the one .

Edit: the change above fixes the problem, new output below:
position fen r1q1nr1k/pp1b2b1/n2p2pp/2pP1p2/2B4B/3Q1N1P/PPP1NPP1/1R3RK1 b - - 0 12
go depth 6
info depth 1 seldepth 1 multipv 1 score cp -34 nodes 32 nps 16000 tbhits 0 time 2 pv g6g5
info depth 2 seldepth 2 multipv 1 score cp 20 nodes 107 nps 53500 tbhits 0 time 2 pv g6g5 h4g5 h6g5
info depth 3 seldepth 3 multipv 1 score cp 20 nodes 153 nps 51000 tbhits 0 time 3 pv g6g5 h4g5 h6g5
info depth 4 seldepth 4 multipv 1 score cp 3 nodes 438 nps 109500 tbhits 0 time 4 pv a6b4 d3d2 b7b5 a2a3
info depth 5 seldepth 5 multipv 1 score cp 18 nodes 585 nps 146250 tbhits 0 time 4 pv a6b4 d3d2 b7b5 a2a3 b4c2 d2c2
info depth 6 seldepth 6 multipv 1 score cp 51 nodes 845 nps 169000 tbhits 0 time 5 pv a6b4 d3d2 b7b5 a2a3 g6g5 a3b4 b5c4
bestmove a6b4 ponder d3d2

[Joerg Oster]

Fingers crossed!

[syzygy]

Perhaps Gary can answer your question.

Here we have a simple bug.
It makes no sense to do move count pruning at depth 1.

Why not?
What makes the root node different from any other PV node in this respect?

At depth 1 in iteration 8 you're not at the root node anymore.

[hgm]

My previous experience here is that any changes like this have failed incredibly badly. But who knows, maybe this is the one .

Note that you could make it even more selective by requiring not just that the last move attacked the from-square, but that it attacked it with a lower-valued piece. Equal or higher attackers are not that dangerous, as the piece could already be sufficiently protected.

[gladius]

My previous experience here is that any changes like this have failed incredibly badly. But who knows, maybe this is the one .

Note that you could make it even more selective by requiring not just that the last move attacked the from-square, but that it attacked it with a lower-valued piece. Equal or higher attackers are not that dangerous, as the piece could already be sufficiently protected.

Yes, that's a good idea.

I just discovered a fun problem with this approach. Since it only cares about the previous move, it pushes problems with hanging pieces over the horizon, and then drops them! Same example position, except now the bishop is threatened by a pawn push, then we make a pawn push to threaten a knight in return, they retreat the knight, and we make a nonsense move and hang the bishop!

So really, we need to do general threat detection for this to work well, which is just far too expensive to do at every node. SF even does it in the eval, however, we cache evals very aggressively, so we'd have to cache the threat results as well. Painful stuff.

position fen r1q1nr1k/pp1b2b1/n2p2pp/2pP1p2/2B4B/3Q1N1P/PPP1NPP1/1R3RK1 b - - 0 12
go depth 8
info depth 1 seldepth 1 multipv 1 score cp -34 nodes 32 nps 32000 tbhits 0 time 1 pv g6g5
info depth 2 seldepth 2 multipv 1 score cp -30 nodes 168 nps 84000 tbhits 0 time 2 pv a6b4 d3d2
info depth 3 seldepth 3 multipv 1 score cp 20 nodes 220 nps 55000 tbhits 0 time 4 pv a6b4 d3d2 g6g5
info depth 4 seldepth 4 multipv 1 score cp 11 nodes 320 nps 40000 tbhits 0 time 8 pv a6b4 d3d2 b7b5 c4b3
info depth 5 seldepth 5 multipv 1 score cp 22 nodes 436 nps 43600 tbhits 0 time 10 pv a6b4 d3d2 b7b5 c2c3 b5c4
info depth 6 seldepth 6 multipv 1 score cp 105 nodes 938 nps 93800 tbhits 0 time 10 pv a6b4 d3e3 b7b5 c2c3 b5c4 c3b4 c5b4
info depth 7 seldepth 7 multipv 1 score cp 197 nodes 2109 nps 191727 tbhits 0 time 11 pv a6b4 d3d2 b7b5 c2c3 b4a6 b2b3 b5c4 b3c4
info depth 8 seldepth 8 multipv 1 score cp 50 nodes 5796 nps 362250 tbhits 0 time 16 pv g6g5 h4g3 f5f4 g3h2 a6b4 d3g6 e8f6 e2f4
bestmove g6g5 ponder h4g3

[Uri Blass]

But that is only a cosmetic fix. The reason why you don't want this behavior is that it corrupts your search in the billions of d<=8 nodes that you visit during a d=30 search.

Yes. And even the correct "fix", which is to disable futility pruning at PV nodes (step 7 and step 13 in search()), is only cosmetic. Sure, you don't do pruning at PV nodes, but the PV comes from what was searched at non PV nodes, where SF does so much "almost random" pruning...

For me this is not a problem, so there's no point in trying to solve it. Pruning is quasi random, and makes lots of mistakes. Like it or not, but that's what makes SF (and modern engines generally) so strong.

Now if people prefer to hide this fact under the carpet, by preventing the user from seeing the horrible blunders that SF can play at shallow non-PV nodes...

I consider it as a problem because it means that stockfish can show decisive advantage when the position is practically equal because of this pruning.

For me it is worse than seeing a draw when there is practically a win that you miss by pruning because it is clear to me that a program cannot see everything but if it see a decisive advantage for one side I want it to check that it has no mistakes by not doing pruning of moves of the side that it believes it is losing and I prefer +6 that is safe at depth 20 and not +6 that may be practically a draw at depth 22.

We talk here about depths 7-8 when stockfish can see correctly at higher depths but I am afraid that the same problem can also happen at depth 19-20 when stockfish can see correctly at higher depths.

[gladius]

I've given this another shot with the cheapest thing I can think of: https://github.com/glinscott/Stockfish/ ... ...906af77

Well, tried a few different approaches, and it appears they don't lose too much ELO, but certainly not a gain.

I measured how often that the different rules kicked in, it was only on 4% of moves pruned. So, movecount pruning is pretty accurate already (at least in terms of dropping pieces .