hgm wrote:...An efficient way to check it would be to calculate a completely independent hash key next to the one you are using now. (Speed is no issue, so calculate it from scratch in every node, ...
Good idea. I do have a button in my GUI to generate a zkey for a given fen string, and another button in my GUI to generate the fen, to compare with the key that has been updated incrementally, but I have yet to automate the process so that I can test it in high volume. I'll make that a priority.
Note to Bob: Thanks for the offer of running the problem positions through Crafty. I'll do a little investigating myself first, and if I can't find a reason, then I'll send you the positions.
Fguy64 wrote:Well, I now have my iterative deepening running, and aside from a little grafting when using "inexact" depth matches, there is really only one weirdity that I can't explain, with the following position
[d]r4rk1/5ppp/p3q1n1/2p2NQ1/4n3/P3P3/1B3PPP/1R3RK1 w - -
now this position is mate in three, touch wood, starting with 1.Qh6
with my hash probes turned off, no problem. When hash probes are enabled, I have to set my fixed depth limit to the level required of a mate in four, then it proceeds to find the mate in three, without the running the remaining iterations. If I set the depth to the level normally required to find a mate in three, it doesn't get found.
All the other forced mate problems are found.
I don't know what development tools you use, but this should be found easily with a debugger. Just step down the mating line and you will find that either your program does not recognize the mate or that it thinks some other move for black postpones the mate.
What search enhancements do you use, if any? Reductions? null-move? Any of those can hide a mate for a couple of iterations. What about extensions?
One thing is for sure, that tree should be small enough that you can print it out to see why the mating sequence is either not being searched, or is not detecting the mate.
no special enhancements at all, just plain alphabeta. Anyways, It's gonna take me some time to write some proper diagnostics to deal with this. So it may be a little while before I have anything more to say on this topic, but I'll keep an eye on the thread.
This should be easy to find with a debugger. Just step down the mating line and either your program does not recognize the mate or it thinks some other move for black postpones the mate.
Fguy64 wrote:Well, I now have my iterative deepening running, and aside from a little grafting when using "inexact" depth matches, there is really only one weirdity that I can't explain, with the following position
[d]r4rk1/5ppp/p3q1n1/2p2NQ1/4n3/P3P3/1B3PPP/1R3RK1 w - -
now this position is mate in three, touch wood, starting with 1.Qh6
with my hash probes turned off, no problem. When hash probes are enabled, I have to set my fixed depth limit to the level required of a mate in four, then it proceeds to find the mate in three, without the running the remaining iterations. If I set the depth to the level normally required to find a mate in three, it doesn't get found.
All the other forced mate problems are found.
I don't know what development tools you use, but this should be found easily with a debugger. Just step down the mating line and you will find that either your program does not recognize the mate or that it thinks some other move for black postpones the mate.
What search enhancements do you use, if any? Reductions? null-move? Any of those can hide a mate for a couple of iterations. What about extensions?
One thing is for sure, that tree should be small enough that you can print it out to see why the mating sequence is either not being searched, or is not detecting the mate.
no special enhancements at all, just plain alphabeta. Anyways, It's gonna take me some time to write some proper diagnostics to deal with this. So it may be a little while before I have anything more to say on this topic, but I'll keep an eye on the thread.
This should be easy to find with a debugger. Just step down the mating line and either your program does not recognize the mate or it thinks some other move for black postpones the mate.
I'm not sure I follow, JW. Correct me if I'm wrong, but a debugger like you describe may help me to know when there is a problem, but finding out why it is happening is probably something else altogether, Make sense?
Fguy64 wrote:Well, I now have my iterative deepening running, and aside from a little grafting when using "inexact" depth matches, there is really only one weirdity that I can't explain, with the following position
[d]r4rk1/5ppp/p3q1n1/2p2NQ1/4n3/P3P3/1B3PPP/1R3RK1 w - -
now this position is mate in three, touch wood, starting with 1.Qh6
with my hash probes turned off, no problem. When hash probes are enabled, I have to set my fixed depth limit to the level required of a mate in four, then it proceeds to find the mate in three, without the running the remaining iterations. If I set the depth to the level normally required to find a mate in three, it doesn't get found.
All the other forced mate problems are found.
I don't know what development tools you use, but this should be found easily with a debugger. Just step down the mating line and you will find that either your program does not recognize the mate or that it thinks some other move for black postpones the mate.
What search enhancements do you use, if any? Reductions? null-move? Any of those can hide a mate for a couple of iterations. What about extensions?
One thing is for sure, that tree should be small enough that you can print it out to see why the mating sequence is either not being searched, or is not detecting the mate.
no special enhancements at all, just plain alphabeta. Anyways, It's gonna take me some time to write some proper diagnostics to deal with this. So it may be a little while before I have anything more to say on this topic, but I'll keep an eye on the thread.
This should be easy to find with a debugger. Just step down the mating line and either your program does not recognize the mate or it thinks some other move for black postpones the mate.
I'm not sure I follow, JW. Correct me if I'm wrong, but a debugger like you describe may help me to know when there is a problem, but finding out why it is happening is probably something else altogether, Make sense?
Used with perseverance and skill, a debugger can help solve almost any reproducible problem. The debugger is your friend. Learn to use it well, and many things will become easier.
anyways, I ran the set of 100,000 matein3 positions four times under different conditions, with the following results...
All tests are vamilla ABP, with Q search turned off. maximum depth set to requirement to solve mate in 3. A failuer is defined as something other than a mate in 3 score.
1. Iterative deepening with Hash Moves, exact matches on depth - 184 failures
2. Iterative deepening turned off, alpha beta and hash code stays the same, just a single call to alphaBeta. - 3 failures
3. ID off, no hash moves, TT only, depth <= stored depth (inexact) - 107 failures
4. ID off, no hash moves, TT only, exact depth matches - 0 failures
whatever that is worth. Anwyays, it'll take a little time to analyze the results. I'm really not sure what to expect. Something wrong with my hash writes maybe?
It sounds like you have a problem with move sorting when there is a hash move (losing some moves). If I understnd your data correctly, case (2) and (4) only differ in the use of the hash move, and as soon as you use it, you get errors.
Different numbers of errors is less significant than no errors vs errors, because the extra errors can all be indirect errors. E.g. Iterative deepening should increase your number of hash hits, and thus can drive up the number of errors due to using hash moves, without there being anything wrong in the IID code itself.
I have never used the debugger on a chess engine. I always debug using print statements in the code. In MakeMove() I do path[ply] = move, and I have a routine PathMatch(char *path, int ply) that compares the path to the current node with a move sequence given as a text string. I the prefix all the print statements with if(PATH), where PATH is #defined as 0 when I am not debugging (so the optimizer will remove them), or as PathMatch(debugPath, ply) when I am debugging. The debugPath comes from a command-line argument (argv[1]), so that I an easily set it by pasting an earlier printed PV onto the command line, or restart after adding or deleting some moves with the arrow keys.
Every ine printed always starts with ply:depth:iidDepth, so I know which node along the path was responsible for the output. Usually I print the move before doing MakeMove() (so if the program hangs, I know in which sub-tree it hangs), and again (but this time together with score and bestScore) after UnMake(). Other useful places to print are on entering Search() (print the window limits), at hash cutoffs (key, entry nr) of detected repetitions. I usually have all the debug print lines in place, but commened out, so that I can activate them by deleting the // when I am interested in that output.
Thanks HG, TT only means that the hash table is used as a transposition table only, the situation where depth > stored depth is ignored altogether.
Your comments brought something to mind. I can't see how the act of sorting for the purpose of miove ordering can play a role, but I am pretty sure I have forgotten to make sure that a hash move doesn't get searched twice, once when it is retrieved from the table, and again when it is shows up in the normal move list from move Gen. I would not have thought that would do anything but slow down the search, but I'll try it anyway.
hgm wrote:What do you mean by "TT only"? Hash cutoffs?
.... If I understnd your data correctly, case (2) and (4) only differ in the use of the hash move, and as soon as you use it, you get errors.
errm, no I don't think that is quite the point, Case (2) differs from case (1) in that iterative deepening is not used in (2), but all of the code for hashing, including hash moves, is active. And you see that (2) has far few errors than (1). How is that explained? But yes, your comparison of (2) & (4) is accurate, but it seems that the type of depth match has a much bigger effect than the use of hash moves. agreed?
The number of errors tells me little. Switching on something that is completely error-free could greatly drive up the number of errors, because it makes the event that can cause an error happen more often.
Inexact-depth probing could alter root scores, because grafting in effect makes you look beyond the horizon, and see higher depth. But it should _never_ be abe to alter mate-scores of mates that are within the horizon.
Because if there is a forced mate within the horizon, what is beyond the horizon is completely irrelevant.
So it s clear you have a bug.
Quickest way to trace it is print a list of the list of moves & scores at every node along the PV of the correct solution. I would start with case (3), if I were you. Just add a conditional print statement after the return of the recursive call to Search, printing the move just searched and the score that search delivered. Let it print only if the moves leading to the current node match the correct PV.
At some point it must (erroneously) see a defense against the mate, i.e. the move that will force the mate will get a non-mate score. Then you can see in the following node what it thought the refutation was.
The only place where an actual move is written to the table is at the end of alphaBeta, this is an exact write with the best move for the node. All the other writes (beta cutoff, depth=0), there is no move, just 0.
If this is correct, then we look at what I am doing with my hash move. Hash move automatically becomes the first best move for that node. Then I do the same as all other moves, which means an alpha update makes the hash type exact, and if the hash move causes a beta cutoff (I'm not even sure it is possible) then the existing table entry is overwritten with a 0 (move) entry and a lower depth, like I do with all the other beta updates. Could the way I am handling the possibility of a hashMove causing a beta cutoff be the cause of the problem?
