Null Move Pruning gives worse results

[Ras]

I'm surprised by this since null move normally gives 50 to 100 ELO gain for most engines. Can someone with a mature engine validate these results?

Tested at 10s/game with no increment, I get +54 Elo over 6k games, comparing NM+LMR+LMP vs. LMR+LMP.

[ciorap0]

Tested at 10s/game with no increment, I get +54 Elo over 6k games, comparing NM+LMR+LMP vs. LMR+LMP.

What evaluation function and move ordering does your engine have? Maybe this could make a big difference.

[Ras]

What evaluation function and move ordering does your engine have? Maybe this could make a big difference.

The eval is material, PST, pawn structure, basic king safety, MG/EG aware - so, pretty complex. Move ordering is hash/PV, MVV/LVA, killers, history. With sufficient depth, also IID. Right before NM, there's also static (no search) reverse futility pruning against beta + margin at low depth.

[ciorap0]

The eval is material, PST, pawn structure, basic king safety, MG/EG aware - so, pretty complex. Move ordering is hash/PV, MVV/LVA, killers, history. With sufficient depth, also IID. Right before NM, there's also static (no search) reverse futility pruning against beta + margin at low depth.

It is similar to what my engine has.

But my engine also shows an Elo difference of ~50 with 10s time control. Probably even faster matches are needed for NMP to have less effect.

[Ras]

But my engine also shows an Elo difference of ~50 with 10s time control. Probably even faster matches are needed for NMP to have less effect.

The point of pruning in general is to reduce the EBF (effective branching factor). The time t needed for a given depth d is t = (EBF)^d. Now, with short thinking time, you don't reach much depth anyway, so the time gain isn't as large. I tested 6k games at 1s/game with no increment, about the fastest my engine is designed for, and now NMP only gains 19 Elo instead of 50.

[jdart]

I'm surprised by this since null move normally gives 50 to 100 ELO gain for most engines. Can someone with a mature engine validate these results?

Correct, it is a very powerful technique and the performance gain should be significant. I think something in the implementation is buggy if it is not working: could be in the actual null move code or elsewhere. I have quite a number of conditions on NMP:

Code: Select all

 // Try to get a fast cutoff using a "null move".  Skip if the side
    // to move is in check or if material balance is low enough that
    // zugzwang is a possibility. Do not do null move if this is an
    // IID search, because it will only help us get a cutoff, not a move.
    // Also avoid null move near the 50-move draw limit.
    if (pruneOk &&
        (depth >= 2*DEPTH_INCREMENT) &&
        !IsNull((node-1)->last_move) &&
        (depth >= 4*DEPTH_INCREMENT || !CaptureOrPromotion((node-1)->last_move)) &&
        board.getMaterial(board.sideToMove()).hasPieces() &&
        IsNull(node->excluded) &&
        node->eval >= node->beta &&
        node->eval >= node->staticEval &&
        ((node->staticEval >= node->beta - int(0.25*Params::PAWN_VALUE) * (depth / DEPTH_INCREMENT - 6)) || (depth >= 12*DEPTH_INCREMENT)) &&
        !Scoring::mateScore(node->alpha) &&
        board.state.moveCount <= 98) {
 

Where "pruneOk" is set as follows:

Code: Select all

const bool pruneOk = !in_check &&
        !node->PV() &&
        !(node->flags & (VERIFY|IID));