Aspiration Windows: Rubbish!

[Gian-Carlo Pascutto]

I know this sounds like a search bug

Right.

Worth nomination for post of the year isn't it?

Also for nomination
http://talkchess.com/forum/viewtopic.php?t=44306

The technical forum is the proper place for newbies to request technical help with the chance of being helped (or ignored in the worst case). Ganging up on them is not really the purpose.

My advice was serious. There is no point in discussing or testing aspiration windows if you have clear and obvious bugs in the more basic aspects of your search.

Get your search working first and then we can discuss which enhancements work or not.

If the original poster doesn't want to accept this he's going to have a hard time.

[michiguel]

I know this sounds like a search bug

Right.

Worth nomination for post of the year isn't it?

Also for nomination
http://talkchess.com/forum/viewtopic.php?t=44306

The technical forum is the proper place for newbies to request technical help with the chance of being helped (or ignored in the worst case). Ganging up on them is not really the purpose.

My advice was serious. There is no point in discussing or testing aspiration windows if you have clear and obvious bugs in the more basic aspects of your search.

Get your search working first and then we can discuss which enhancements work or not.

If the original poster doesn't want to accept this he's going to have a hard time.

I was not referring to your message, but the following two, which did not contribute. One thing is to point there is a bug, another is to laugh in group with no useful feedback.

Miguel

[Don]

I know this sounds like a search bug

Right.

Worth nomination for post of the year isn't it?

Also for nomination
http://talkchess.com/forum/viewtopic.php?t=44306

The technical forum is the proper place for newbies to request technical help with the chance of being helped (or ignored in the worst case). Ganging up on them is not really the purpose.

My advice was serious. There is no point in discussing or testing aspiration windows if you have clear and obvious bugs in the more basic aspects of your search.

Get your search working first and then we can discuss which enhancements work or not.

If the original poster doesn't want to accept this he's going to have a hard time.

There was no problem with your post Gina-Carlo.

Don

[petero2]

Hi Matthew,

1 one ply search is easy to debug. Note the position where it happens and then for every move in the list look which one causes the trouble and why it is going so deep.

You can do this step by step, you will probably note something wrong with the code that lets you bang your head on the wall a few times. (Like one move captures the king or so)

This happened to me quite a few times.

Thomas...

In Texel I have written code that dumps the whole search tree to a file. Things like alhpa, beta, depth, ply, score, score type (upper/lower bound or exact), evaluation score and hash key are stored. I can then run the program in a special mode that lets me navigate in the search tree stored in the file. This makes it relatively easy to understand why the program is doing something stupid even in deep searches and without disabling parts of the search algorithms.

The tree logging/analysis code is in the files treeLogger.hpp and treeLogger.cpp in this zip file: http://web.comhem.se/petero2home/texel101.zip

[Don]

Hi Matthew,

1 one ply search is easy to debug. Note the position where it happens and then for every move in the list look which one causes the trouble and why it is going so deep.

You can do this step by step, you will probably note something wrong with the code that lets you bang your head on the wall a few times. (Like one move captures the king or so)

This happened to me quite a few times.

Thomas...

In Texel I have written code that dumps the whole search tree to a file. Things like alhpa, beta, depth, ply, score, score type (upper/lower bound or exact), evaluation score and hash key are stored. I can then run the program in a special mode that lets me navigate in the search tree stored in the file. This makes it relatively easy to understand why the program is doing something stupid even in deep searches and without disabling parts of the search algorithms.

The tree logging/analysis code is in the files treeLogger.hpp and treeLogger.cpp in this zip file: http://web.comhem.se/petero2home/texel101.zip

That is an excellent idea. I have done this myself but only in an ad-hoc manner to solve a specific bug.

[wgarvin]

In Texel I have written code that dumps the whole search tree to a file. Things like alhpa, beta, depth, ply, score, score type (upper/lower bound or exact), evaluation score and hash key are stored. I can then run the program in a special mode that lets me navigate in the search tree stored in the file. This makes it relatively easy to understand why the program is doing something stupid even in deep searches and without disabling parts of the search algorithms.

The tree logging/analysis code is in the files treeLogger.hpp and treeLogger.cpp in this zip file: http://web.comhem.se/petero2home/texel101.zip

For Windows users: This TabView tool might also be useful for viewing dumps of trees. I've never used it myself, but you ought to be able to dump a whole tree into a text file and use TabView to drill down to the part you are interested in.

[diep]

Hi Matthew,

1 one ply search is easy to debug. Note the position where it happens and then for every move in the list look which one causes the trouble and why it is going so deep.

You can do this step by step, you will probably note something wrong with the code that lets you bang your head on the wall a few times. (Like one move captures the king or so)

This happened to me quite a few times.

Thomas...

In Texel I have written code that dumps the whole search tree to a file. Things like alhpa, beta, depth, ply, score, score type (upper/lower bound or exact), evaluation score and hash key are stored. I can then run the program in a special mode that lets me navigate in the search tree stored in the file. This makes it relatively easy to understand why the program is doing something stupid even in deep searches and without disabling parts of the search algorithms.

The tree logging/analysis code is in the files treeLogger.hpp and treeLogger.cpp in this zip file: http://web.comhem.se/petero2home/texel101.zip

That is an excellent idea. I have done this myself but only in an ad-hoc manner to solve a specific bug.

that's what i do to debug parallel search since 1999. dumping to a textfile had however 2 disadvantages:

a) the files would get too big to download back then (i had a 56k6 modem at home and Bob's quad Xeon is obviously in USA).

b) writing to a textfile takes a lot of system time, so the odds a parallel bug occurs dramatically reduces.

So i store it in shared memory (which means you're got a limited time for the error to occur) then dump that to a binary file. That binary file compressed i can download and then with a parser parse it to a textfile with all information.

It's really useful for all sorts of things, as long as you know where to look at inside those huge files. The expanded textfiles usually were gigabytes huge.

Try to find a parallel bug then!

In 2002 i changed the habit for the parallel search - i wrote on a paper the parallel functionality and tried to prove it correct on paper. That removed a LOT of the pain

[Antonio Torrecillas]

Based on an issue I've had some time ago, here my two cents.
On a normal search, alpha beta full window, when we prune it's because somewhere else we find a PV with value equal to the value of beta. So we can safely prune.
Now with aspiration window, this assumption is no longer valid. If we don't fail in the first move we have a PV node/path valid with his score, and the assumption is valid again. We prune value >= beta because somewhere else we have found beta. But we can now fail in the first/PV node so the PV is not a real PV.
To complete the scenario we add null move to the sauce. Suppose that, for some strange situation, you return in the false PV a null move. It's quite easy that the score will be a Mate score.(your phantom).
Now you wide the window and research, you get a true PV just you don't benefit from the quick Nullmove in the PV and you search to the nominal depth which is 3 ply deeper than your previous aspiration search.
I suggest that when a mate score is returned you inspect the pv to see if there is a null move. Or disable null move to see if the phantom disappear.

I don't think your base search is wrong.

Dissident regards, Antonio.

[asanjuan]

I have tested and tested, and I have one conclusion. They simply don't work for me.

I've watched many a test game where it searches normally, and is doing quite well, then suddenly gets a 'persistent phantom mate score' which pushes the window size to the point where it takes 5 seconds to perform a *one ply* search. This costs the program valuable time, and it loses.

I know this sounds like a search bug, but running with [-ValueInf, +ValueInf] works perfectly. It loses a ply of search, but never has a phantom mate score.

Matthew:out

¿Are you using Futility pruning on PV nodes?
With futility, in a 1 ply search, when you have an infinite window in a PV node, futility doesn't affect. But if you use a small window, it can hurt and return invalid mate values, because you can be pruning all the moves.

hope this helps.
Regards.

[hgm]

You can futility-prune in a PV node after alpha is raised, right? The window is only {-INF, INF} on the search of the first move. After that alpha will be raised to some finite value, which very well might be above the current eval. E.g. suppose the first move captured a hanging Queen. After that, all moves that do not capture at least a Hawk are pretty futile.