My program can't mate KQK or KRK!

[Sven]

I will ponder a little more to see if I miss the point you make.

I think there is no point in refusing a TT cutoff at node P for the only reason that the alleged best move at P (as found in the TT) would lead to a repetition or 50 moves draw. The only interesting thing is whether P itself is such a drawn position (and you simply find that out by checking for draw prior to probing hash). But what does a draw condition, or its absence, for one successor of P tell you about the correctness of a TT cutoff at P? You might state: if the moving side at P has one move that draws then a TT score < 0 at P would be incorrect. But that is not what you do in your mate_search().

Sven

[Chan Rasjid]

Hello Sven,

I will ponder a little more to see if I miss the point you make.

I think there is no point in refusing a TT cutoff at node P for the only reason that the alleged best move at P (as found in the TT) would lead to a repetition or 50 moves draw. The only interesting thing is whether P itself is such a drawn position (and you simply find that out by checking for draw prior to probing hash). But what does a draw condition, or its absence, for one successor of P tell you about the correctness of a TT cutoff at P? You might state: if the moving side at P has one move that draws then a TT score < 0 at P would be incorrect. But that is not what you do in your mate_search().

Sven

I now understand the point you are making. My IQ-cpu is about Pentium 2, albeit -Pro. There was too much data in the pipeline.

We all test the node P itself, the parent of the node of probehash(), for any repetition and would not call search if it is.

About whether it is ok to delay an exact TT hit cutoff for the purpose of testing if the tt-move is a repetiton, I think it is legitimate - if it has zero cost (if my thinking now is sound!), then I think everyone would be doing it.

Many of us just do cutoff on an exact TT hit and this line may end as a new PV with the tt-move as the terminal of the PV; but the tt-move may be a repetition and this is not what we want. I think this thing about repetition nodes that are embedded into the TT is a legitimate problem but which does not have a complete solution. I once implemented some partial solution to ameliorate this problem but has delected it to simplify my codes for debugging (or until Cowrie opens shop next door to Robert Houdart's).

Best Regards,
Rasjid.

[Sven]

We all test the node P itself, the parent of the node of probehash(), for any repetition and would not call search if it is.

I called P the node from which you possibly return with a TT cutoff. In your mate_search() function you call probehash() at P but test for repetition at successors of P. I don't see how that matches your statement. Repetitions at ply=1 would be missed in case of a mate search, unless your root search function tests for repetition before possibly calling mate_search() for the ply=1 nodes.

Sven

[Chan Rasjid]

Hello Sven,

Hello Sven,

I will ponder a little more to see if I miss the point you make.

I think there is no point in refusing a TT cutoff at node P for the only reason that the alleged best move at P (as found in the TT) would lead to a repetition or 50 moves draw. The only interesting thing is whether P itself is such a drawn position (and you simply find that out by checking for draw prior to probing hash). But what does a draw condition, or its absence, for one successor of P tell you about the correctness of a TT cutoff at P? You might state: if the moving side at P has one move that draws then a TT score < 0 at P would be incorrect. But that is not what you do in your mate_search().

Sven

I now understand the point you are making. My IQ-cpu is about Pentium 2, albeit -Pro. There was too much data in the pipeline.

We all test the node P itself, the parent of the node of probehash(), for any repetition and would not call search if it is.

About whether it is ok to delay an exact TT hit cutoff for the purpose of testing if the tt-move is a repetiton, I think it is legitimate - if it has zero cost (if my thinking now is sound!), then I think everyone would be doing it.

Many of us just do cutoff on an exact TT hit and this line may end as a new PV with the tt-move as the terminal of the PV; but the tt-move may be a repetition and this is not what we want. I think this thing about repetition nodes that are embedded into the TT is a legitimate problem but which does not have a complete solution. I once implemented some partial solution to ameliorate this problem but has delected it to simplify my codes for debugging (or until Cowrie opens shop next door to Robert Houdart's).

Best Regards,
Rasjid.

Yes, I was wrong again in this:

We all test the node P itself, the parent of the node of probehash(), for any repetition and would not call search if it is.

It should be:

We all test the node P itself, the node at which we call probehash(), that it is not a repetition. This is done at the node prior to P and search() would not be called if there is repetition.

I think there there is no mistake otherwise in my the post - except the concern about potential repetition moves that may be stored in the TT. It should be a non-issue. My earlier attempt to provide a partial solution to it may be plain silly.

A new PV at root may be found that is terminated by a tt-move. To worry if the tt-move is a repetition that could cause the engine to finally concede a draw from a winning position involves a combination of too many 'ifs' - the probability may be near zero!

Firstly, the tt-move has to actually be a repetition. In the next root iteration, the repetition detection codes would have avoided the line. Even 'if' the line gets played, it may not reached the tt-move. Even 'if' finally the engine comes to the node that has the tt-move, in a winning situation it is more likely there is another move that would still give the engine a win.

No wonder it seems no one cared about it and no wonder...

Instead of trying to trace the bug in my engine, I am now trying to implement Herr Muller's 'two lines of codes that never failed me' - two lines that may just do a flush of all hidden bugs!


Best Regards,
Rasjid.

[Chan Rasjid]

Hello All,

Bad news and good news.

The bad news is - I have yet to identify the bug.

The good news is - I got back one older version of my program and it is confirmed
to be free of the bug of failing to mate KRK!

I threw away all my old CD's but just found that there is still one; it has many of my other older versions.

I tested KRK with TT enabled and with various opponents jazz, fruit, stockfish,etc..and it mates with any number of replays.

I found differences (fairly different) in the mate_search(). I will look at the other codes to see what causes the bug in my new version. I am confident the other aspect of my new version is much better.

I will post when I find the cause of the bug.

BTW, Herr Mueller's two lines of codes is a headace!

Thanks All for the help.
Rasjid.

[hgm]

But they work like a charm!

[Chan Rasjid]

Hello,

I have finally cleared the bug, but not the the one I posted here which still cannot be resolved. It could only be fixed partially; but the fix is not right as it could not explain why the program could normally mate KRK if TT is disabled.

The real bug is a misconception - that there is a need to take action if root finds a winning mate score; to have codes that would take care of the supposedly special situation arising. There is no need to have much additional codes to handle getting the shortest mate - definitely not writing a mate search function to handle it. At most a few simple additional lines of codes at the root search should suffice (as Michael Hoffman mentioned but which he did not emblazon in the firmament).

I literally gave up on the bug. Then got the idea to see how alpha-beta itself could handle things. I set matesearch = 0 within the loops of my root loops and added exit(-1) to my matesearch() to make sure it was not called. Then I played KRK with TT enabled. It just simply finished the game. I confirmed that there was no hashing bugs by repeating the games a number of times and also with different other engines. Nothing need be done! Alpha-beta is a rather smart algorithm! The only little annoyance is that the iterative deepening would reach the max-depth as it does not know when to stop. But this could easily be fixed.

No wonder God sent Jesus Christ down to earth - to dispel the disease called 'chronic ignorance'.

Happy Chess Programming,
Rasjid.

[Chan Rasjid]

Hello,

I forgot to say what the bug was. It was about setting the aspiration bound alpha.

Whenever a new root iteration is to be started and if it is to call mate_search() instead of the normal pvs_search(), the aspiration bound is set thus:
alpha = best = winning-mate-score;
beta = INFI;
and there is a move associated with the mate.

The bound alpha caused problem; it has to be set to -INFI and the program will mate KRK. But it still may occasionally fail to mate unlike my older version that never failed.

The fix is not right as I can't see why it cannot be set to the mate score. Maybe the answer is obvious but I don't know. The setting worked if TT is disabled. So it has to do with the way I store mate scores. But I cannot trace where I went wrong.

Best Regards,
Rasjid.

[Desperado]

Hello,

I forgot to say what the bug was. It was about setting the aspiration bound alpha.

Whenever a new root iteration is to be started and if it is to call mate_search() instead of the normal pvs_search(), the aspiration bound is set thus:
alpha = best = winning-mate-score;
beta = INFI;
and there is a move associated with the mate.

The bound alpha caused problem; it has to be set to -INFI and the program will mate KRK. But it still may occasionally fail to mate unlike my older version that never failed.

The fix is not right as I can't see why it cannot be set to the mate score. Maybe the answer is obvious but I don't know. The setting worked if TT is disabled. So it has to do with the way I store mate scores. But I cannot trace where I went wrong.

Best Regards,
Rasjid.

Hi Chan,

i really do not have much time at the moment, so here i try to give
another little hint.

To open the window and everything seems to work, but the aspiration
window fails indicates an error of the following type:

within the search you have an artificial window [alpha,beta].

If your search now is only updating on conditions like value > alpha,
and this condition is not met at a certain node, the search will
return a random score below alpha if there is no value better than alpha.

This would be a typical bug already seen in this context.

So you might check if there will always be returned a bestvalue if you use
a failsoft framework.

Hope it helps, must work on now ...

Michael

[Chan Rasjid]

Hello Michael,

Hello,

I forgot to say what the bug was. It was about setting the aspiration bound alpha.

Whenever a new root iteration is to be started and if it is to call mate_search() instead of the normal pvs_search(), the aspiration bound is set thus:
alpha = best = winning-mate-score;
beta = INFI;
and there is a move associated with the mate.

The bound alpha caused problem; it has to be set to -INFI and the program will mate KRK. But it still may occasionally fail to mate unlike my older version that never failed.

The fix is not right as I can't see why it cannot be set to the mate score. Maybe the answer is obvious but I don't know. The setting worked if TT is disabled. So it has to do with the way I store mate scores. But I cannot trace where I went wrong.

Best Regards,
Rasjid.

Hi Chan,

i really do not have much time at the moment, so here i try to give
another little hint.

To open the window and everything seems to work, but the aspiration
window fails indicates an error of the following type:

within the search you have an artificial window [alpha,beta].

If your search now is only updating on conditions like value > alpha,
and this condition is not met at a certain node, the search will
return a random score below alpha if there is no value better than alpha.

This would be a typical bug already seen in this context.

So you might check if there will always be returned a bestvalue if you use
a failsoft framework.

Hope it helps, must work on now ...

Michael

Thanks for the hint.

I have now settled the problem of mate with KQK and KRQ. I have thrown away mate_search() and made some changes mainly to my root search to take care of getting the best mate- nothing really much. Also just some minor changes to the full search. Everything's working fine and I can go on to try other improvements.

We all have different understanding about chess programming and the more difficult may be alpha-beta when we want to implement a 'professional' class version. It will usually be with root aspiration window, fail-soft and its interaction with transposition table. It will then be fairly complicated and bug prone.

I cannot really quantify how good my alpha-beta theory is. About 'artificial bounds' [alpha, beta], I do have some understanding. Aspiration windows are artificial - just bounds set and we 'aspire' to find a best-value within the bounds. Traditional alpha-beta starts with [-inf, +inf] and throughout search, alpha is a 'real' bound, the best score so far that the side has found. Beta is also real; -beta is the best score so far found by the other side. But I am not sure that I know where to look for bugs making use of this knowledge.

Fail soft together with aspiration windows have some subtleties, but I can't really pinpoint how; except that when I come to it and need to somehow make a return without searching and outside of window bounds, I will think and pay more attention. Such a situation is a little more tricky; eg if we do futility pruning and need a value to return without searching.

Finally, the only way I know how to debug my program is:
1) many many asserts.
2) play fast games to look for strange behaviour.

Best Regards,
Rasjid.