Fguy64 wrote:So do we agree that with just straight up fixed depth negamax/alpha beta with no hashing or other bells and whistles and no considerations to move ordering, then does that validate my debugging strategy, from a mathematically correct perspective?
No, we don't, as I already said in the first place. With NegaMax searching a node at level 1 is something _completely_ different from seaching that same position as root for any move but the first. The first move increases alpha, i.e. changes the window.
OK fine I can see that the window changes. Somehow I feel my point is not getting across, but if I'm wrong it wouldn't be the first time. I've done this technique several times, and it has proved an effective, thought maybe not the most efficient, technique for zeroing in on positions that cause runtime errors somewhere up the tree
Anyways, as a point of interest I'm gonna try to disprove my own theory by looking for a counterexample. If I find one I'll post it. I don't have the knowhow to create or understand a strictly mathematical proof of this, so trial and error will have to suffice.
Your point come across very clearly, but it is just not true. Due to alphabeta the tree searched from a level=1 node if (for all moves except the first) much smaller than when you would search that position from the root, with the window fully open.
Because some branches are cut in one and not in the other tree, a certain position that is visited in both will not always have been preceded by the same position at that level. So if, for instnce, you have a bug that forgets to initialize a vaiable, it finds in memory the value that the previous routine left. If that value is unagreeable, your engine could choke on it. But what it is depends on the previous position searched at that level, and if that is another position (because other moves get in between that were not cut off because of the different window setting), the error goes away.
OK thanks, I will take some time to let this sink in.
It has just occurred to me that just because I have used my technique time and again to zero in on bugs and problem positions, it does not mean I have zeroed in on the same node that caused the error the first time. I suppose it is entirely likely that whatever the bug that has caused the runtime error, that bug will manifest itself at variouspoints all through the tree for that position. So as I step through the nodes and reduce the ply, the odds are pretty good that I will find the bug, just at a different node/position. I suppose it depends on the bug. But that seems to at least go part way to reconciling our positions.
Yes, this could be the case. And of course there are also bugs that are insensitive to what happened before. E.g. when you would scan the ray between a checker and the king in the wrong direction for an onbstruction, so that you run indefinitely far off board. On such bugs your method would work. But the really nasty bugs are those that depend on history and occur extremely rarely. There will come a time that those are the only type of bugs you have left, having fixed all others.
Definitely not. If you would be able to catch various bugs not having any debugging tools in your hand then I call you genius. Really, I had so many bugs in my engine which were sometimes so subtle, mysterious and persistant that it often took weeks to figure what's going on there. For me to have all those debugging and diagnostic tools were absolutely necessary condition of improving my engine.
Filip
Definitely not. If you would be able to catch various bugs not having any debugging tools in your hand then I call you genius. Really, I had so many bugs in my engine which were sometimes so subtle, mysterious and persistant that it often took weeks to figure what's going on there. For me to have all those debugging and diagnostic tools were absolutely necessary condition of improving my engine.
Filip
been there. Anyways I don't use an IDE either, so I suppose I am doubly handicapping myself. I'm no genius, nor am I efficient. It's been a long time coming. But I learned long ago to keep changes small. Write a few lines, recompile, review, test. Then repeat. And make lots of backups.
Fguy64 wrote:OK one thing my program does not have yet is proper diagnostics output, PV or perftt or whatever you want to call it. But I do know what level1 node is being searched when something weird happens. So, if my alpha-beta is working properly, the following should work (I think)
As I see it, If I know what level1 move is being searched when the program craps out, then I should be able to make that move manually, reduce the ply by 1, restart the engine, and the engine should crap out in the exact same position.
Agreed?
Maybe or maybe not. What dynamic information do you use at ply=2 that will not be there if you start over? Hash table info? killer move info? History info if you use that???
Fguy64 wrote:OK one thing my program does not have yet is proper diagnostics output, PV or perftt or whatever you want to call it. But I do know what level1 node is being searched when something weird happens. So, if my alpha-beta is working properly, the following should work (I think)
As I see it, If I know what level1 move is being searched when the program craps out, then I should be able to make that move manually, reduce the ply by 1, restart the engine, and the engine should crap out in the exact same position.
Agreed?
Maybe or maybe not. What dynamic information do you use at ply=2 that will not be there if you start over? Hash table info? killer move info? History info if you use that???
Well, there have been a number of posts since that one you have quoted, and I have since backed off from my position, thanks to the convincing of Mr. Muller . Perhaps my last response to Mr Muller, about finding the same bug but in a different node of the same tree, does a better job of explaining that which I have experienced.
Fguy64 wrote:been there. Anyways I don't use an IDE either, so I suppose I am doubly handicapping myself. I'm no genius, nor am I efficient. It's been a long time coming. But I learned long ago to keep changes small. Write a few lines, recompile, review, test. Then repeat. And make lots of backups.
For me putting in print statements in strategic places is still the most powerful way to debug. In a recursive program like the search of a chess engine you must be able to limit the printout to nodes that you are really interested in, though, but with the if(PATH) method that works quite well. It is a bit of a pain when an error occurs 20 ply deep in the tree, to zoom in on it. But especially once you have implemented hashing it is the only reliable way I found to make errors reproducible.
Fguy64 wrote:been there. Anyways I don't use an IDE either, so I suppose I am doubly handicapping myself. I'm no genius, nor am I efficient. It's been a long time coming. But I learned long ago to keep changes small. Write a few lines, recompile, review, test. Then repeat. And make lots of backups.
For me putting in print statements in strategic places is still the most powerful way to debug. In a recursive program like the search of a chess engine you must be able to limit the printout to nodes that you are really interested in, though, but with the if(PATH) method that works quite well. It is a bit of a pain when an error occurs 20 ply deep in the tree, to zoom in on it. But especially once you have implemented hashing it is the only reliable way I found to make errors reproducible.
Cool, well I know there is room for major improvements in my diagnostics, and your idea seems good to me.
Even in my "seroing in" stategy, that often will tell me more or less "where" in the code the problem manifests itself, then very often I will add a print statement that tells me "what" the problem is.
As a hobbyist, I tend not to bother with a lot of tools like sophisticated debugging and performance evaluation programs, or even IDEs, honestly to me they seem more trouble than they're worth, but I suspect that if I were to seriously consider programming as a profession, then I'd probably have to "get with the program".
. Perhaps my last response to Mr Muller, about finding the same bug but in a different node of the same tree, does a better job of explaining that which I have experienced.