strange analysis by stockfish1.5

[Tord Romstad]

Code: Select all

22     +1.82   00:33   46492k Bxd5 Kf6 Bxc4

OK, now I've looked at this, and it doesn't seem like a bug. This is what happens:

Because the move Bxd5 fails high, all moves at the child node fail low, and there is no best move there. The move Kf6 is therefore taken from the hash table. It's a move which failed high in the same position soem time earlier in the search. How could the nonsense move Kf6 ever fail high in the position after Bxd5, you ask? By a repetition draw! The move comes from the following variation (where "null" is a null move):

Bxd5 Kf6 null Ke7 null Kf6 draw

I am fairly sure this is also what happened in the case of the equally strange Kd8 move observed by Uri.

What this really shows is that storing repetition draws in the transposition table is somewhat dubious (as has been observed by many people numerous times in the past). Avoiding this, however, is a little messy. Perhaps it's a good idea not to evaluate a position as a repetition draw when there's a null move somewhere along the line between the current position and the previous time it occured?

[Vinvin]

Perhaps it's a good idea not to evaluate a position as a repetition draw when there's a null move somewhere along the line between the current position and the previous time it occured?

It's clearly a good idea !

[Uri Blass]

Code: Select all

22     +1.82   00:33   46492k Bxd5 Kf6 Bxc4

OK, now I've looked at this, and it doesn't seem like a bug. This is what happens:

Because the move Bxd5 fails high, all moves at the child node fail low, and there is no best move there. The move Kf6 is therefore taken from the hash table. It's a move which failed high in the same position soem time earlier in the search. How could the nonsense move Kf6 ever fail high in the position after Bxd5, you ask? By a repetition draw! The move comes from the following variation (where "null" is a null move):

Bxd5 Kf6 null Ke7 null Kf6 draw

I am fairly sure this is also what happened in the case of the equally strange Kd8 move observed by Uri.

What this really shows is that storing repetition draws in the transposition table is somewhat dubious (as has been observed by many people numerous times in the past). Avoiding this, however, is a little messy. Perhaps it's a good idea not to evaluate a position as a repetition draw when there's a null move somewhere along the line between the current position and the previous time it occured?

I suspect that searching null after Kf6 is a bad idea because Kf6 can be refuted faster by non null move.

I wonder if it is not better not to use null move pruning when the evaluation is very bad and instead of it reduce the depth.

You can still continue to use null move pruning if the evaluation is not very bad because in this case reducing the depth seems too risky.

Uri

[mcostalba]

I wonder if it is not better not to use null move pruning when the evaluation is very bad and instead of it reduce the depth.

You can still continue to use null move pruning if the evaluation is not very bad because in this case reducing the depth seems too risky.

Actually we already do that:

Code: Select all

    // Null move search
    if (    allowNullmove
        &&  depth > OnePly
        && !isCheck
        && !value_is_mate(beta)
        &&  ok_to_do_nullmove(pos)
        &&  approximateEval >= beta - NullMoveMargin)
    {

Where

Code: Select all

  // Null move margin. A null move search will not be done if the approximate
  // evaluation of the position is more than NullMoveMargin below beta.
  const Value NullMoveMargin = Value(0x300);

[Zach Wegner]

Perhaps it's a good idea not to evaluate a position as a repetition draw when there's a null move somewhere along the line between the current position and the previous time it occured?

Definitely. The easy way to handle this is to reset the fifty move counter when null moving. This came up in a thread some time ago, it was causing some weird problems for ZCT...

You really should do this if you're using double null move, since otherwise null move won't work when beta > 0 (after null null it's a repetition and the score is zero, so the second null move always fails high). Of course, skipping the ply-2 position, or requiring three repetitions, in your repetition detection avoids this directly, but it should still happen. Using infinite null move (which you use IIRC) only makes it worse.

[bob]

Explanation is pretty obvious here:

* New thing added to Stockfish 1.5 is that it _extends_ PV from transposition table. Usually it works, but sometimes it gives bizarre lines like this. Without extension it would only show "PV = 1. Bb7 x d5".

I still do not understand.

How did the tranposition table get Ke8 after Bxd5

I agree: It does look a little suspicious in this case. I'll have a look at it and try to find out what really happens, if I'm able to reproduce your analysis (should be easy if you used a single search thread, but perhaps very difficult if you used more than one).

You'll always get a good PV in the end, by waiting until the fail high has been resolved. In practice, this can sometimes take a very long time, because Stockfish doesn't try to resolve fail highs at the root immediately, but postpones it to the next iteration instead (most people would probably say that this is absurd and completely crazy and obviously can't work well, and I would be tempted to agree. Fortunately Joona didn't think so, and to my surprise it seems to work extremely well in practice). Sometimes the search will fail high at the next iteration as well, and you may end up waiting several plies for an exact score and a real PV. I wonder if it would be a good idea to disable aspiration windows in analysis mode, to make sure the user always get exact scores and high-quality PVs.

Here is how we originally implemented the PVS stuff in 1978. Ken did this in Belle in 1980, I did it in 1978 while talking to Murray Campbell at the Washington ACM computer chess tournament.

If you get a fail high at the root, make this the best move but do not re-search. If you get a second fail high at the root, now you go back and re-search the first fail-high move to get a score, then do a normal null-window on the second fail-high move. If it fails high here, you know it is better and you did less work than if you also searched it completely to resolve the score.

The only thing I didn't like about this was (a) you can run out of time after only a single move has failed high, and you have no PV, which means no move to ponder. I solved this in Crafty with the "puzzling search" which is used to find a move to ponder if there is none in the PV nor the hash table. (b) I don't know how bad or good the score is, so I can't tell whether to use extra time or not. I might have had a very bad score, followed by a fail high on a different move, and now the question is, does this move bring the score back up to an acceptable level or is it still very bad? I eventually quit using this approach for that very reason. Overall it seemed to make no real difference. One way is faster and helps in some positions, the other way is more accurate and helps in other positions...

[bob]

Perhaps it's a good idea not to evaluate a position as a repetition draw when there's a null move somewhere along the line between the current position and the previous time it occured?

It's clearly a good idea !

Not so "clear". That makes the null-move searches take significantly longer, which will slow things down everywhere. If you get the PV from the hash table, it is simply going to be bogus from time to time. So long as the best move is good, things are ok. Even though you might end up pondering a stupid move for 3 minutes...

[bob]

Perhaps it's a good idea not to evaluate a position as a repetition draw when there's a null move somewhere along the line between the current position and the previous time it occured?

It's clearly a good idea !

Not so "clear". That makes the null-move searches take significantly longer, which will slow things down everywhere. If you get the PV from the hash table, it is simply going to be bogus from time to time. So long as the best move is good, things are ok. Even though you might end up pondering a stupid move for 3 minutes...

I just looked at Crafty. Interestingly, it sets the 50-move counter to zero when making a null-move. I'm going to play around with this to see if it makes sense, since this does disable repetition detection for positions that occur before/after a null-move.

[George Tsavdaris]

So I think that forcing people to use/not to use aspiration window in analysis mode is a bad thing. Should we make this ucioption?

That would be excellent!