Hash Collision?

[Sven]

Anyways, I appreciate the effort people make, but this thread has gone in a direction I never really intended. It started off with a question to Bob about what he meant by, and I paraphrase "I can't tell you how many times I'be tried to debug the problem, only to realise that it isn't a problem" I think, but I am not certain that he was referring to grafting, and at that time I was trying to ascertain whether his remark was relevant to the problem I was having, I thought it might be because my remarks clearly showed that while an incorrect score was being reported (a mate in 2 score instead of a mate in three score), the correct move was being played.

Initially you started this thread reporting problems related to hash table usage, where you got the same best move but different evaluations for few test positions after enabling hash table code.

Yesterday you brought up a different problem you have which seems to be clearly related to the special handling of mate scores. At least the problem looks very different from the previous one for me. You just used the same thread for discussing it.

So it surprises me that you say you didn't intend this thread to go into that direction. In fact it is kind of a different thread where people try to explain your wrong mate-in-2 score.

And if it turns out some day that both problems are in fact caused by only one single bug then you might still be ok with having discussed both in the same thread.

Sven

[Fguy64]

Anyways, I appreciate the effort people make, but this thread has gone in a direction I never really intended. It started off with a question to Bob about what he meant by, and I paraphrase "I can't tell you how many times I'be tried to debug the problem, only to realise that it isn't a problem" I think, but I am not certain that he was referring to grafting, and at that time I was trying to ascertain whether his remark was relevant to the problem I was having, I thought it might be because my remarks clearly showed that while an incorrect score was being reported (a mate in 2 score instead of a mate in three score), the correct move was being played.

Initially you started this thread reporting problems related to hash table usage, where you got the same best move but different evaluations for few test positions after enabling hash table code.

Yesterday you brought up a different problem you have which seems to be clearly related to the special handling of mate scores. At least the problem looks very different from the previous one for me. You just used the same thread for discussing it.

So it surprises me that you say you didn't intend this thread to go into that direction. In fact it is kind of a different thread where people try to explain your wrong mate-in-2 score.

And if it turns out some day that both problems are in fact caused by only one single bug then you might still be ok with having discussed both in the same thread.

Sven

If you look again at my question to Bob from yesterday. I was in fact talking about a correct move but an incorrect eval, and thinking about grafting. It just so happened that the example I gave was a mating line, and the incorrect eval was a mate score. But at no time did I suggest or even suspect that the problem was with the handling of mate scores.themselves. That was HG's suggestion, several posts later. So my previous post to you stands.

[Sven]

well, I tried changing the end of my search function from ...

Code: Select all

if( count == 0 ) {

    if( (gs.white && isSquareAttackedW(gs.wkp)) ||
        (!gs.white && isSquareAttackedB(gs.bkp)) ) return -(10000-ply);

		return 0;
}
	
hKey[hI] = gs.zKey; hType[hI] = hashType; hDepth[hI] = (byte) depth; hVal[hI] = (short) alpha;
return alpha;

to ...

Code: Select all

int score;

if( count == 0 ) {

    if( (gs.white && isSquareAttackedW(gs.wkp)) ||
        (!gs.white && isSquareAttackedB(gs.bkp)) ) score = -(10000-ply);

    else score = 0;

    hashType = exact;
}
else score = alpha;
	
hKey[hI] = gs.zKey; hType[hI] = hashType; hDepth[hI] = (byte) depth; hVal[hI] = (short) score;
return score;

but it made no difference.

It made no difference because with this change you did only add code to store information about "mated in 0 plies" and "stalemate" positions in the TT. But you did not address the problem that you store a mate score with type "exact" but incorrect value, which can also happen if alpha (EDIT: as score of the best move having been searched) is a mate score, either "mated in N plies" or "mate in N plies" but not "mated in 0".

See also my long post from today in this thread, to which you already replied. Whatever you choose from the given alternatives, you must change *something* at least because your current code as you have quoted it above is wrong w.r.t. mate scoring and will let your engine play considerably weaker. It may turn out that your engine becomes unable to actually mate the opponent on the board due to this kind of error, when repeatedly considering the wrong moves as "I can mate in two".

Code: Select all

...
else score = alpha;

int ttScore = score;
// adjust mate scores
if (ttScore > 9000) [
    ttScore += ply;
} else
if (ttScore < -9000) {
    ttScore -= ply;
}
hKey[hI] = gs.zKey;
hType[hI] = hashType;
hDepth[hI] = (byte) depth;
hVal[hI] = (short) ttScore;

return score;

Sven

[Sven]

If you look again at my question to Bob from yesterday. I was in fact talking about a correct move but an incorrect eval, and thinking about grafting. It just so happened that the example I gave was a mating line, and the incorrect eval was a mate score. But at no time did I suggest or even suspect that the problem was with the handling of mate scores.themselves. That was HG's suggestion, several posts later. So my previous post to you stands.

Which seems to prove that there are indeed two different problems. One is about mate scores (this one is a real problem in your code, and you have been presented a couple of possible solutions) and the other one is the one that was discussed initially in this thread, where it was not clear whether any problem exists at all. I do not see any connection between both, unless you show that fixing the mate scoring problem also solves the other one (which I doubt).

Sven

[hgm]

never mind, when using depth <= stored depth the above position gets reported as a mate in 2 even when the depth is set to 4 , which is insufficient for a mate in 3.

sigh

The point is that you started out by asking if it was normal if scores change when you do hashing with inexact-depth cutoffs, or if it pointed to an error. The answer was "this is normal, it would only prove there is an error if it would happen with exact-depth cutoffs".

But then you set an entirely different stage by the quoted post. because there you reported not merey a changing score; you reported a _wrong_ score. That the score changes is normal, as long as it changes from a score that was right to another score that was right. The change is caused by grafting, which effectively replaces the score for one depth by a score for another depth, and scores are known to change from one depth to another.

But not mate scores. A mate in three at 6 ply must still be a mate in three when you search 100 ply. it could disappear at 4 py, but it should not turn into a mate in 2, at 4 ply. reporting a mate in 2 when there is none is always an error, no matter at which depth it occurs, or how much grafting you had.

[Fguy64]

...

See also my long post from today in this thread, to which you already replied. Whatever you choose from the given alternatives, you must change *something* at least because your current code as you have quoted it above is wrong w.r.t. mate scoring ...

fair enough, all I am saying is that evidence suggests something other than the actual handling of mate scores as the cause of the problem. Given that I wasn't even storing mate scores, I now think that conclusion makes sense. Do we agree on that? I'm gonna try and find an example of a situation that does not involve a mate score.

I'm gonna review a few things here...

1. For my current tests, I'm not using iterative deepening, nor am I storing hash moves. Nor am I doing qSearch. Strictly vanilla AlphaBeta pruning. with a transposition table.

2. The problem has only been observed with inexact depth matches in the hash probe (depth <= stored depth). Exact matches show no problems whatsoever.

3. My engine has passed all the perft tests and hash key verification tests I have thrown at it.

regards.

[Fguy64]

never mind, when using depth <= stored depth the above position gets reported as a mate in 2 even when the depth is set to 4 , which is insufficient for a mate in 3.

sigh

The point is that you started out by asking if it was normal if scores change when you do hashing with inexact-depth cutoffs, or if it pointed to an error. The answer was "this is normal, it would only prove there is an error if it would happen with exact-depth cutoffs".

But then you set an entirely different stage by the quoted post. because there you reported not merey a changing score; you reported a _wrong_ score. That the score changes is normal, as long as it changes from a score that was right to another score that was right. The change is caused by grafting, which effectively replaces the score for one depth by a score for another depth, and scores are known to change from one depth to another.

But not mate scores. A mate in three at 6 ply must still be a mate in three when you search 100 ply. it could disappear at 4 py, but it should not turn into a mate in 2, at 4 ply. reporting a mate in 2 when there is none is always an error, no matter at which depth it occurs, or how much grafting you had.

Thank you, that is very much on point. I will keep this in mind as work on this off and on throughout the day,

[Fguy64]

...

See also my long post from today in this thread, to which you already replied. Whatever you choose from the given alternatives, you must change *something* at least because your current code as you have quoted it above is wrong w.r.t. mate scoring ...

fair enough, all I am saying is that evidence suggests something other than the actual handling of mate scores as the cause of the problem. Given that I wasn't even storing mate scores, I now think that conclusion makes sense. Do we agree on that? I'm gonna try and find an example of a situation that does not involve a mate score.

I'm gonna review a few things here...

1. For my current tests, I'm not using iterative deepening, nor am I storing hash moves. Nor am I doing qSearch. Strictly vanilla AlphaBeta pruning. with a transposition table.

2. The problem has only been observed with inexact depth matches in the hash probe (depth <= stored depth). Exact matches show no problems whatsoever.

3. My engine has passed all the perft tests and hash key verification tests I have thrown at it.

regards.

OK, mating position at the horizon will be my focal point. It may be one of the issues. There is info on that in this thread, give me some time treview

oh crap, information overload, ignore this horizon remark, I need to take a break.

[Sven]

...

See also my long post from today in this thread, to which you already replied. Whatever you choose from the given alternatives, you must change *something* at least because your current code as you have quoted it above is wrong w.r.t. mate scoring ...

fair enough, all I am saying is that evidence suggests something other than the actual handling of mate scores as the cause of the problem. Given that I wasn't even storing mate scores, I now think that conclusion makes sense. Do we agree on that? I'm gonna try and find an example of a situation that does not involve a mate score.

I'm gonna review a few things here...

1. For my current tests, I'm not using iterative deepening, nor am I storing hash moves. Nor am I doing qSearch. Strictly vanilla AlphaBeta pruning. with a transposition table.

2. The problem has only been observed with inexact depth matches in the hash probe (depth <= stored depth). Exact matches show no problems whatsoever.

3. My engine has passed all the perft tests and hash key verification tests I have thrown at it.

regards.

I think I have understood what you do.

However, I also think that you have not yet understood what HGM and I are saying. You are constructing a connection between your TT mate scoring bug, which really exists, and the "grafting" topic, which is a different issue - although TT-related, too - but does not necessarily involve any bug.

Sven

[Sven]

fair enough, all I am saying is that evidence suggests something other than the actual handling of mate scores as the cause of the problem. Given that I wasn't even storing mate scores, I now think that conclusion makes sense. Do we agree on that?

No. You *were* storing mate scores even prior to your change, just not the "mated in 0" scores. But "I can mate him in 3 plies" is also a mate score!

Sven