Razoring

[bob]

Wrong way of looking at it.

Please try to focus on the fact that we are talking of pruning at node level, not pruning of the single move.....and no, it is not the same thing.

I thought we were talking about "razoring". Which is the process of taking a move directly to q-search rather than pruning it and throwing it away without searching at all.

That IS the subject of the thread?

[Michel]

I thought we were talking about "razoring". Which is the process of taking a move directly to q-search rather than pruning it and throwing it away without searching at all.

Razoring is taking a _position_ directly to quiescence search, not a move.

The idea is that the position is so bad that you would like to discard it.
But as Marco said, the opponent might have left a queen hanging.
So to make sure that does not happen you perform a qsearch.

[Sven]

I thought we were talking about "razoring". Which is the process of taking a move directly to q-search rather than pruning it and throwing it away without searching at all.

Razoring is taking a _position_ directly to quiescence search, not a move.

The idea is that the position is so bad that you would like to discard it.
But as Marco said, the opponent might have left a queen hanging.
So to make sure that does not happen you perform a qsearch.

It seems that the only thing that is agreed between different experts here is that razoring can be applied either as a depth reduction or as directly dropping into q-search (which is a special kind of depth reduction, of course).

Disagreement is there about the question where exactly in the search razoring shall be applied. In contrast to Marco's remark above "it is not the same thing" I have the impression that both ways are possible in general, albeit under different conditions and viewpoints:

a) applying razoring at "node level", i.e. in the top part of a search function before starting the move loop, and

b) applying razoring to one particular move within the move loop.

The paper of Heinz describes his "limited razoring" in the sense of variant a). Bob, however, sees razoring in the sense of b), which I believe is possible, too.

Sven

[mcostalba]

For the record, this is a dangerous way to test an idea. Real games matter, positions don't.

In this case I have to second Bob. And also I add tht I am not comfortable in adding changes to the search that do not show a real gain. This is because we always have an error margin when testing and if the change shows a result within the error bar you can never be sure that it doesn't add a regression. If you are lucky it is an improvment, but if you are unlucky it is a regression. I have learned the hard way that is always better do not leave a known path for an unknown one also if it "seems" a logical change: you risk to find your engine weaker even after many weeks and it becomes extremely difficult to track down the incorrect change.

[mcostalba]

In contrast to Marco's remark above "it is not the same thing" I have the impression that both ways are possible in general, albeit under different conditions and viewpoints:

I don't think so, when you prune a move you always have an upper bound of what you can expect from that move, for instance if it is a capture you always knwow in advance (with somewhat good approsimation) what is the maximum gain of the move that you can expect, even without trying it, hence the possibility to prune without a qsearch() verification if the possible max gain is anyhow not enough to reach beta. In case of razoring a node the qsearch() is instead mandatory to avoid terrible mistakes due to possible opponent's hanging pieces.

[Sven]

In contrast to Marco's remark above "it is not the same thing" I have the impression that both ways are possible in general, albeit under different conditions and viewpoints:

I don't think so, when you prune a move you always have an upper bound of what you can expect from that move, for instance if it is a capture you always knwow in advance (with somewhat good approsimation) what is the maximum gain of the move that you can expect, even without trying it, hence the possibility to prune without a qsearch() verification if the possible max gain is anyhow not enough to reach beta. In case of razoring a node the qsearch() is instead mandatory to avoid terrible mistakes due to possible opponent's hanging pieces.

Let me first react directly (1) on your statement above, before digging deeper into the topic further down (2).

1) I support what you say, but is that an argument for razoring not being applicable in both ways? You just say that in one case you need to do it differently than in the other, not that you can't do it at all.

Furthermore, the same information, like possible maximum gain, could be made available also at the node level since you are dealing just with the previous move of the opponent that had been made before entering the node.


2) After thinking and reading more about razoring I can now also imagine that Bob and you are actually talking about two really different ways of razoring. Both of you may of course correct me if I am wrong. Here is what I understood:

In your version, also implemented in Stockfish, razoring means to skip a node that seems to be bad enough to likely score well below alpha, i.e. razoring leads to failing LOW.

In the version Bob is talking about, and had also implemented in older Crafty versions, razoring means to reduce (or, as tried experimentally, possibly even skip) a move that seems to be bad enough to likely score well below alpha, which in turn means that, when making the recursive search call after that move (i.e. not skipping it), the opponent is left in a pretty good position that lets him fail HIGH.

For instance, in search.c of old Crafty 19.17, lines 456-458 (and essentially in subsequent versions up to 22.1) we find:

Code: Select all

else if &#40;depth >= 3 * INCPLY && depth < 5 * INCPLY &&
    &#40;Material + RAZOR_MARGIN&#41; <= alpha&#41;
  extended -= 60;

within the move loop of Search(), in the non-PV part after MakeMove() has been performed, meaning roughly that a likely weak move is reduced.

Both versions have in common that they are not applied in PV nodes, but both are in fact looking quite different, and yet both were called "razoring" for some reason.

@Marco + Bob: Is my view correct?

Sven

[lucasart]

If Marco wants to try reducing instead of pruning maybe for Stockfish too the following code is not so much worse than the default;

Hi Eelco,

actually I have tested this idea (that was suggested to me by Lucas Braesch) just few days ago.

I tested first the original idea of Lucas:

Code: Select all

// Step 6. Razoring (is omitted in PV nodes)
    if (   !PvNode
        &&  depth < RazorDepth
        && !inCheck
        &&  refinedValue + razor_margin&#40;depth&#41; < beta
        &&  ttMove == MOVE_NONE
        &&  abs&#40;beta&#41; < VALUE_MATE_IN_PLY_MAX
        && !pos.has_pawn_on_7th&#40;pos.side_to_move&#40;)))
    &#123;
        Value rbeta = beta - razor_margin&#40;depth&#41;;
        Value v = qsearch<NonPV>&#40;pos, ss, rbeta-1, rbeta, DEPTH_ZERO&#41;;

        if &#40;v < rbeta && &#40;depth -= ONE_PLY&#41; < 2 * ONE_PLY&#41;
            // Logically we should return &#40;v + razor_margin&#40;depth&#41;), but
            // surprisingly this did slightly weaker in tests.
            return v;
    &#125;

then an extension I figured out, namely keep razoring in qsearch at low depths but reduce of one ply if razor test pass at high depths (note that there is no more the condition on RazorDepth):

Code: Select all

    // Step 6. Razoring (is omitted in PV nodes)
    if (   !PvNode
        && !inCheck
        &&  refinedValue + razor_margin&#40;depth&#41; < beta
        &&  ttMove == MOVE_NONE
        &&  abs&#40;beta&#41; < VALUE_MATE_IN_PLY_MAX
        && !pos.has_pawn_on_7th&#40;pos.side_to_move&#40;)))
    &#123;
        Value rbeta = beta - razor_margin&#40;depth&#41;;
        Value v = qsearch<NonPV>&#40;pos, ss, rbeta-1, rbeta, DEPTH_ZERO&#41;;

        if &#40;v < rbeta && &#40;depth -= ONE_PLY&#41; < RazorDepth&#41;
            // Logically we should return &#40;v + razor_margin&#40;depth&#41;), but
            // surprisingly this did slightly weaker in tests.
            return v;
    &#125;

Both ideas failed to give a measurable gain, although there was no regression actually, so maybe there is something good there, but my test didn't show they were clearly better.


Regarding your idea to use condition

Code: Select all

!(tte && tte->type() == VALUE_TYPE_LOWER)

instead of testing for a ttMove, well I am not very convinced because if there is a ttMove it means that this node _was_ a cut-off node somewhere in the past, so perhaps there is still some danger in the position and the condition on ttMove seems so a bit more safer.

Finally a stylistic note on

Code: Select all

        if (tte && can_return_tt(tte, rDepth, rbeta, ss->ply))
        {
            TT.refresh(tte);
            ss->bestMove = ttMove; // Can be MOVE_NONE
            value = value_from_tt(tte->value(), ss->ply);
        }
        else value = rDepth < ONE_PLY ? qsearch<NonPV>&#40;pos, ss, rbeta-1, rbeta, DEPTH_ZERO&#41; &#58; search<NonPV>&#40;pos, ss, rbeta-1, rbeta, rDepth&#41;; 

This is redundant code IMHO because the same check on TT is done as first thing in the qsearch() call, so you just duplicate that part of code for no real gain.

I'm just not convinced with the

Code: Select all

ttMove == MOVE_NONE

when the node is explored first, it's not in the TT so your search behaves the normal way and stores the result in the TT. when you visit the node again... you probe the TT and prune if depth is sufficient, otherwise you search normally (with the TT move as a "likely to be best" move), but the problem is that suddenly your search behaves differently. this introduces some search inconsistency, and i dont really see what it could possibly improve to compensate it. Is it the result of some testing, or just an intuive idea ?
I'm pretty sure you can remove all the

Code: Select all

ttMove == MOVE_NONE

in the code of StockFish and will see no elo differnce. On the contrary you should reduce the risk of search inconsistencies

[Rebel]

Wrong way of looking at it.

Please try to focus on the fact that we are talking of pruning at node level, not pruning of the single move.....and no, it is not the same thing.

Absolutely and that's were (I think) the confusion comes from.

There are 2 ways of futility pruning.

1. Classic following Heinz model, move pruning, no need to involve QS.

2. Node pruning, try futility before move generation, QS is a must.

[Sven]

Wrong way of looking at it.

Please try to focus on the fact that we are talking of pruning at node level, not pruning of the single move.....and no, it is not the same thing.

Absolutely and that's were (I think) the confusion comes from.

There are 2 ways of futility pruning.

1. Classic following Heinz model, move pruning, no need to involve QS.

2. Node pruning, try futility before move generation, QS is a must.

But our topic is razoring, not futility pruning. Heinz did apply razoring at the node level, look here again. The razoring condition triggers a depth reduction from pre-pre-frontier to pre-frontier. You are right about futility pruning + extended futility pruning but to me it seems Heinz did not razor moves.

Sven

[Rebel]

Wrong way of looking at it.

Please try to focus on the fact that we are talking of pruning at node level, not pruning of the single move.....and no, it is not the same thing.

Absolutely and that's were (I think) the confusion comes from.

There are 2 ways of futility pruning.

1. Classic following Heinz model, move pruning, no need to involve QS.

2. Node pruning, try futility before move generation, QS is a must.

But our topic is razoring, not futility pruning. Heinz did apply razoring at the node level, look here again. The razoring condition triggers a depth reduction from pre-pre-frontier to pre-frontier. You are right about futility pruning + extended futility pruning but to me it seems Heinz did not razor moves.

Sven

As I always understood Heinz means move_futlity_pruning (not node) and razoring means a reduction. I could be wrong, the text does not deserve the beauty prize for clarity, the idea itself does