Mate scores when razoring are wrong

[jordanbray]

I've been looking into a very strange bug with my mate scores. I'm assuming my eval is wrong as well, but mate scores are easier to prove.

This occurs whenever I implement a change to my search which causes any instability. Currently, I'm testing with razoring only (no reductions, no null move pruning, nothing else fancy like that). This does include transposition tables, but that (plus the search) should handle mate scores correctly.

Whenever I turn on razoring, my mate scores become completely insane. From a game against Phalanx, my eval goes 65. -M98/1, 66. -M52/10, 67. -M60/1, 68. -M62/1 69. -M2/3 (no move 70, due to me being checkmated).

Has anyone experienced anything similar to this, or is it just an old-fashioned bug sosmewhere?

Thanks.

[Bloodbane]

Could you post your razoring code? Without it it is pretty hard to say anything, and it's a small piece anyway.

[jordanbray]

Here is the code I use for razoring.

Code: Select all

static inline bool prune_razor(int alpha, int beta, int depth, search_t *search, int *score, move_info_t *mi) {
    int eval = search->static_eval[search->depth_index];
    int margin = 512+32*depth;
    if (!search->board->pin_info.checked && depth <= 3 && depth > 0 && !search->in_pv && eval + margin <= alpha) {
        int ralpha = alpha - margin;
        int val = qsearch(ralpha, ralpha+1, 0, 0, search);
        if (val <= ralpha) {
            *score = val;
            return true;
        }
    }
    return false;
}

I also ensure that checkmate scores are not allowed from the prune_* family of functions (currently, only using this one).

[JVMerlino]

Here is the code I use for razoring.

Code: Select all

static inline bool prune_razor(int alpha, int beta, int depth, search_t *search, int *score, move_info_t *mi) {
    int eval = search->static_eval[search->depth_index];
    int margin = 512+32*depth;
    if (!search->board->pin_info.checked && depth <= 3 && depth > 0 && !search->in_pv && eval + margin <= alpha) {
        int ralpha = alpha - margin;
        int val = qsearch(ralpha, ralpha+1, 0, 0, search);
        if (val <= ralpha) {
            *score = val;
            return true;
        }
    }
    return false;
}

I also ensure that checkmate scores are not allowed from the prune_* family of functions (currently, only using this one).

This is a bit of a shot in the dark, but it looks very likely that your new ralpha value (alpha - margin) could be well below your constant value of -CHECKMATE when alpha is near mate. So you would be returning a score worse than being checkmated, which could cause the kinds of problems you are seeing.

jm

[jordanbray]

This is a bit of a shot in the dark, but it looks very likely that your new ralpha value (alpha - margin) could be well below your constant value of -CHECKMATE when alpha is near mate. So you would be returning a score worse than being checkmated, which could cause the kinds of problems you are seeing.

jm

I'm certain that's not the case, because if it returned something in the checkmate range (for either side), it would still perform a full search. Additionally, removing the pruning code causes it to behave correctly.

The reason I posted this bug here is because this occurs no matter what pruning method I use, including late move reductions. Therefore I was wondering if I was missing some key component.

[JVMerlino]

Ah, ok. Fair enough. I thought it was just the razoring code causing the problem and that looked suspicious to me.

Best of luck finding your bug.

[Bloodbane]

The razoring code seems completely correct to me. I'd check the TT code (if there is any) and make sure nothing is wrong there, but that's just a gut feeling. Otherwise I have no idea.

[Sven]

I've been looking into a very strange bug with my mate scores. I'm assuming my eval is wrong as well, but mate scores are easier to prove.

This occurs whenever I implement a change to my search which causes any instability. Currently, I'm testing with razoring only (no reductions, no null move pruning, nothing else fancy like that). This does include transposition tables, but that (plus the search) should handle mate scores correctly.

Whenever I turn on razoring, my mate scores become completely insane. From a game against Phalanx, my eval goes 65. -M98/1, 66. -M52/10, 67. -M60/1, 68. -M62/1 69. -M2/3 (no move 70, due to me being checkmated).

Has anyone experienced anything similar to this, or is it just an old-fashioned bug sosmewhere?

Thanks.

May I assume that you already ensure to transform TT scores to search scores when retrieving from TT, and vice versa when storing to it, in case of mate scores? Something like this (where "height" is the distance to root):

Code: Select all

static inline int scoreFromTT(int ttScore, int height)
{
    if (ttScore > +MAX_EVAL_SCORE) return ttScore - height;
    if (ttScore < -MAX_EVAL_SCORE) return ttScore + height;
    return ttScore;
}

static inline int scoreToTT(int ttScore, uint height)
{
    if (ttScore > +MAX_EVAL_SCORE) return ttScore + height;
    if (ttScore < -MAX_EVAL_SCORE) return ttScore - height;
    return ttScore;
}

[jordanbray]

May I assume that you already ensure to transform TT scores to search scores when retrieving from TT, and vice versa when storing to it, in case of mate scores? Something like this (where "height" is the distance to root):

Code: Select all

static inline int scoreFromTT(int ttScore, int height)
{
    if (ttScore > +MAX_EVAL_SCORE) return ttScore - height;
    if (ttScore < -MAX_EVAL_SCORE) return ttScore + height;
    return ttScore;
}

static inline int scoreToTT(int ttScore, uint height)
{
    if (ttScore > +MAX_EVAL_SCORE) return ttScore + height;
    if (ttScore < -MAX_EVAL_SCORE) return ttScore - height;
    return ttScore;
}

Yea, I do have that. I don't modify the score being added to the TT, because I handle that in the search itself. Every time the lower depth search returns a mate score, I add one depth to that mate score before touching the TT. But otherwise this code works when razoring is off.

I have been doing some debuging today, though, and it does look like there's another bug in the TT that is the cause. I found a position where the TT should return a mate score, and does not. I'm not sure if that's related at all...

[bob]

May I assume that you already ensure to transform TT scores to search scores when retrieving from TT, and vice versa when storing to it, in case of mate scores? Something like this (where "height" is the distance to root):

Code: Select all

static inline int scoreFromTT(int ttScore, int height)
{
    if (ttScore > +MAX_EVAL_SCORE) return ttScore - height;
    if (ttScore < -MAX_EVAL_SCORE) return ttScore + height;
    return ttScore;
}

static inline int scoreToTT(int ttScore, uint height)
{
    if (ttScore > +MAX_EVAL_SCORE) return ttScore + height;
    if (ttScore < -MAX_EVAL_SCORE) return ttScore - height;
    return ttScore;
}

Yea, I do have that. I don't modify the score being added to the TT, because I handle that in the search itself. Every time the lower depth search returns a mate score, I add one depth to that mate score before touching the TT. But otherwise this code works when razoring is off.

I have been doing some debuging today, though, and it does look like there's another bug in the TT that is the cause. I found a position where the TT should return a mate score, and does not. I'm not sure if that's related at all...

That sounds wrong. For this to work you have to do two things correctly

(1) when you store anything close to a mate score, you have to convert it to "mate in N from this ply" rather than using the "mate in M from the root position" which is the normal mate score you would be backing up.

(2) when you get a hash hit on any score close to the mate window, you have to convert that from "mate in N from this ply" to "mate in M from the root" so that alpha/beta will work correctly.

You can't do one and not the other, otherwise you are going to see wildly jumping mate scores.