bob wrote:jwes wrote:bob wrote:jwes wrote:Fguy64 wrote:hgm wrote:Are you acceptng exact-depth hits only, or also when the stored depth (draft) >= thn the currently requested search depth? In the latter case you can hve score changes due to grafting.
I am counting a hit as stored depth >= current remaining depth. So it sounds like that sort of thing is to be expected?
Yes, particularly in engames where zugzwang is involved. Try changing your code to only cutoff on = depth and see if the anomaly goes away.
Fguy64 wrote:It appears that another aspect of this situation is as follows.
Normally, for a fixed depth search, if you have the engine play both sides one move at a time, reducing the requested search depth by one ply at each half move, and clearing the hash table at each such reduction, then you can step through the original PV move by move, and the eval will be negated each time. However, for the anomalous positions, the PV changes part way through, and the eval changes at the same time. Furthermore, it appears that the differences happen at the beginning of the process, and then disappear as I step through the PV. Am I clear in what I am describing?
This sounds like what would happen if tree grafting were occurring.
This still won't "fix" the problem. It will make it occur less frequently, but hashing can always change the score or PV.
I don't follow. When would searching a position to a given depth return a different result from the last time that position was searched to that depth, given the conditions of the first post?
fixed-depth, no iterative deepening, no special pruning, just alphaBeta
qSearch disabled
material evaluation only
Do you clear hash after _every_ move played? Do you do extensions that are limited based on remaining depth? Pruning decisions based on alpha/beta bounds. There are a ton of things that let the hash graft branches from one part of the tree to another. Even with equal depths you can produce different scores or sub-trees, and then grafting takes those differences and lets them influence other parts of the tree.
I can't count the number of times I have debugged this problem, only to realize that it not a problem at all. I would never do the draft==depth "fix". Makes no sense at all to throw away good information and replace it with a search that is not free...
If you take only the simple search you described above, the problem may well not show up. But that's an unrealistic search. To expect things to remain the same is about as useful as expecting the Easter bunny. Just changing the hash size will quite frequently change the score, the time used, and even the best move for a given depth, because the replacement policy is another influence.
I've been thinking about your remark about it not being a problem. In an attempt to get to the bottom of the situation, I re-assembled my engine to include perft and hash key verification, and tested both till the cows came home. The issues were exactly the same.
ok, in the situation of vanilla abp, no qSearch, no iterative deepening, no hash moves, just transposition matching, and depth <= stored depth for the hash match.
consider the following position. Which is a mate in three.
[d]1B3k2/P2Q4/7p/8/P1N1B1p1/7b/7b/7K w - -
when I set the depth to 6, which is required to find a mate in 3 because the 6th ply is used to determine there are no legal moves, then my engine plays 1.Bd6 and returns a score of 9997, which is the normal score for a mate in two. A garbage PV is also displayed using the triangular approach. But the move leads to a mate in three. Now if I reduce the depth by 1 and have the engine make a black move, it plays 1...Bxd6 and returns a score of -9996, which means we are back into the correct score, and White now has a mate in two.
So the point is that in a normal game, an incorrect score might get returned, but the correct move still gets played. Is this what you mean by not a problem?
It makes me thing that in my batch job, I should measure a success as score >= expected score, not score == expected score.