Pondering? Yes. Ponder move? Maybe not.

[Daniel Shawul]

So you don't have a transposition table "age" field that would cause many of those positions to be overwritten on this new search since the age would have increased by one?

The effect is minimal. If you do the search up to depth 11 you can choose to skip it ( I think you do that) or
search it again which is ultra fast despite the few transposition misses. Most engines do start search from
depth 1 anyway and there is no blow up of search at all. If you prefer to skip that search,
it means you are assuming you have everything in TT, so I don't understand your "age" field argument here.

And what happens when you search two moves that lead to the same score, a transposition? Or the search tree explodes and the second move takes almost as long as the first since few cutoffs happen? In either case, you really expend search effort that is _guaranteed_ to be 1/2 wasted. Where with "ponder the move" if you can beat 50% ponder hits, you are already ahead of the game and the higher the hit rate the more ahead in the timing game you get.

You are making one assumption I do not agree with. I believe it is better to use _all_ time on a single move. If it fails low, then we find a better move to play since on a fail low we are pretty sure we are pondering the right move. But no matter what happens, I want to search as deeply as possible. Otherwise I would be doing an N-best sort of search all the time, which would simply kill my search performance.

Yes, we have a basic disagreement here. My argument is to improve play against a stronger opponent by assessing the risk
of the selected ponder move often may be a better idea.You are only looking at time saving as the ultimate factor.
I agree with you that you can save more time with your method if your prediction rate is above 50% (lets take 60%),
but the risk involved with choosing a ponder move may not be something you
want to take especially against a strong opponent.

I'll ask the same question again, show me any sane reason for pondering the position, given a hit rate of over 50% (mine seems to be over 60% in most games), and I'll give it some thought. But so far, no mathematical explanation, or logical explanation has been given, just "there must be a better way..."

Ok I will try to do that.
Lets consider a case which I think is a must do even for your method.
Let us say you selected a weak ponder move against a strong opponent, and while pondering you realize that
your score goes up. This is solely due to the weakness of the selected ponder move and has got nothing
to do with you winning. In this case, wouldn't you want to search a better alternative move first instead of continuing pondering
with that ponder move? In all prediction failures (that is 40%!!) against a strong opponent, there is a general tendency for your ponder
score to rise. When you sniff trouble in the ponder search, you might as well start working on it immediately there. But in your case
the ponder search completely wastes its time searching something useless while pondering, and then spends time when it detects the trouble
after the opponent made the "unexpected second move" (which could have been detected while pondering).

[Daniel Shawul]

Suppose we are averaging 20 ply searches in a game. It seems that you are saying to switch sides, and after a 19 ply search, "freeze" the best move and ponder only that for the remainder of the time?

How does that make sense. When I finished the last search I _already_ had a 19 ply search completed for my opponent, since the total depth was 20 plies and I played the first move.

I don't see how this makes any sense at all, you just repeat the same search you already have completed, and with luck it will take almost no time due to the hash table, with bad luck it will take a longer time and hurt worse.

I believe it doesn't matter much to do or not do the search again as I mentioned in my other response. Usually you will get good enough
tt entries from the depth preferred table, so all you get ultra fast search
up to N-1 with short PVs. Most engines I know do this anyway, so I believe the issue is practically irrelevant.If you think it matters, then skip step ii of my proposition.

[bob]

So you don't have a transposition table "age" field that would cause many of those positions to be overwritten on this new search since the age would have increased by one?

The effect is minimal. If you do the search up to depth 11 you can choose to skip it ( I think you do that) or
search it again which is ultra fast despite the few transposition misses. Most engines do start search from
depth 1 anyway and there is no blow up of search at all. If you prefer to skip that search,
it means you are assuming you have everything in TT, so I don't understand your "age" field argument here.

And what happens when you search two moves that lead to the same score, a transposition? Or the search tree explodes and the second move takes almost as long as the first since few cutoffs happen? In either case, you really expend search effort that is _guaranteed_ to be 1/2 wasted. Where with "ponder the move" if you can beat 50% ponder hits, you are already ahead of the game and the higher the hit rate the more ahead in the timing game you get.

You are making one assumption I do not agree with. I believe it is better to use _all_ time on a single move. If it fails low, then we find a better move to play since on a fail low we are pretty sure we are pondering the right move. But no matter what happens, I want to search as deeply as possible. Otherwise I would be doing an N-best sort of search all the time, which would simply kill my search performance.

Yes, we have a basic disagreement here. My argument is to improve play against a stronger opponent by assessing the risk
of the selected ponder move often may be a better idea.You are only looking at time saving as the ultimate factor.
I agree with you that you can save more time with your method if your prediction rate is above 50% (lets take 60%),
but the risk involved with choosing a ponder move may not be something you
want to take especially against a strong opponent.

Problem is, you have not clearly given any good reason why "ponder the position" gives you something that is useful. You really believe that you can start a search from the same position as your opponent, and out-search him to learn something you didn't already know? And if you ponder like that, you are now playing _exactly_ into his hand because your search tree is a perfect sub-tree of what your opponent examined, so you don't get to spring any surprises except purely by luck, which I don't depend on.

I'll ask the same question again, show me any sane reason for pondering the position, given a hit rate of over 50% (mine seems to be over 60% in most games), and I'll give it some thought. But so far, no mathematical explanation, or logical explanation has been given, just "there must be a better way..."

Ok I will try to do that.
Lets consider a case which I think is a must do even for your method.
Let us say you selected a weak ponder move against a strong opponent, and while pondering you realize that
your score goes up. This is solely due to the weakness of the selected ponder move and has got nothing
to do with you winning. In this case, wouldn't you want to search a better alternative move first instead of continuing pondering
with that ponder move?

No argument there, although that is very rare in the games I just looked at. But even so, what do you do to find a _better_ move to ponder? If you try "ponder the position" you will need just as much time as your opponent to see what he sees, and no more, so how does that help? You are a ply or two short of what you would normally search since the first move is for your opponent, not for you.

So while I understand the idea that occasionally a ponder move is a mistake, I do not see how this helps to find that and then somehow give us an advantage in our search. If we ponder the wrong move at present, we just lose out on the time savings, nothing else. If we ponder the position, we lose out on _every_ ponder search because of the loss of a ply caused by searching from the opponent's POV.

In all prediction failures (that is 40%!!) against a strong opponent, there is a general tendency for your ponder
score to rise. When you sniff trouble in the ponder search, you might as well start working on it immediately there. But in your case
the ponder search completely wastes its time searching something useless while pondering, and then spends time when it detects the trouble
after the opponent made the "unexpected second move" (which could have been detected while pondering).

I don't disagree with your "wasted time" point. But I do not see how your solution does anything but waste _more_ time on every single move, not just on 40% of the moves, which seems worse to me.

[hgm]

The general idea is that in the correctly predicted 60% of the moves, there are hardly any sudden rises in the ponder score (corresponding to sudden fail lows for the opponent, making him switch move). So you would not switch away from them very often, as you would not even consider to switch away from them very often.

When pondering a move, you have a head-start of 1 ply over your opponent, so you will complete the search of his best move before he does. But after the fail low he will have to do an equally deep search on one or more alternative better moves, which will take him a long time. If he doesn't switch move, he will be done quickly.

So by switching to the next-best move you either gain a lot of extra time if the opponent also switched to that move, and lose only a little if he did not switch. You need much less than 50% correctness to make this pay off.

[Bill Rogers]

Let's us face it gentlemen if you programs eval is good enough then pondering will almost always give you the best results, if not .........

After playing hundreds of games with many strong programs and watching the output and the moves that they predict I can say the stronger the program the more predictions come true and even when the opponent doe not make the expected move, the move that they actually play is generally inferror to the move they should have made according to the stronger program.

Bill

[mclane]

there is a dedicated chess computer with a program by julio kaplan.
this machine calculates 3 best moves in permanent brain instead of 1.

machine is called:sci sys turbo 16 K

[bob]

The general idea is that in the correctly predicted 60% of the moves, there are hardly any sudden rises in the ponder score (corresponding to sudden fail lows for the opponent, making him switch move). So you would not switch away from them very often, as you would not even consider to switch away from them very often.

When pondering a move, you have a head-start of 1 ply over your opponent, so you will complete the search of his best move before he does. But after the fail low he will have to do an equally deep search on one or more alternative better moves, which will take him a long time. If he doesn't switch move, he will be done quickly.

So by switching to the next-best move you either gain a lot of extra time if the opponent also switched to that move, and lose only a little if he did not switch. You need much less than 50% correctness to make this pay off.

The idea sounds good. _UNTIL_. How do I decide on the second-best move to ponder? I have to repeat the same search my opponent is doing, to the same depth, which you just said is going to take extra time.

That is the step I do not follow. I understand the problem with pondering a move that fails high significantly. I do not understand how to find another move to ponder in any reasonable amount of time and then still save any time on the clock. And there must be time savings or else pondering is ineffective.

[bob]

there is a dedicated chess computer with a program by julio kaplan.
this machine calculates 3 best moves in permanent brain instead of 1.

machine is called:sci sys turbo 16 K

That can be done, but at a cost of approximately 1.5 plies. Most of the search effort is spent on finding the best move, the rest go by quickly. Unless you want to find the best 3 moves. Then you triple the effort to find the best move, and the rest still go by quickly. Slowing you down by a factor of 3, which is about 1.5 plies.

[mclane]

maybe these very old dedicated machines made the normal search completely usual. i doubt they searched for 3 moves in the usual search.

but maybe they stored the 3 best moves the search switched during computation and later when in permanent brain they used these 3 moves
to compute about in permanent brain. of course the permanent brain would not be AS DEEP as you could do with only 1 candidate.
but on the other hand having 3 moves as candidates gives you maybe higher amount of HIT-rate for the permanent brain as normal.
and this could bring you into advantage for the next computation time.

http://www.schach-computer.info/wiki/in ... _Turbo_16K

[bob]

maybe these very old dedicated machines made the normal search completely usual. i doubt they searched for 3 moves in the usual search.

but maybe they stored the 3 best moves the search switched during computation and later when in permanent brain they used these 3 moves
to compute about in permanent brain. of course the permanent brain would not be AS DEEP as you could do with only 1 candidate.
but on the other hand having 3 moves as candidates gives you maybe higher amount of HIT-rate for the permanent brain as normal.
and this could bring you into advantage for the next computation time.

http://www.schach-computer.info/wiki/in ... _Turbo_16K

I have seen some that did a horrible kludge, namely each time the PV changes, just keep the most recent 3 different PVs that have been shown. The problem is that two of those might have come from searches several iterations prior to the current one so the scores aren't comparable because the depths are different.