Large search depths

[Henk]

I don't understand how other chess programs can reach search depths > 15 Do they put R = 7 or so when null move pruning, or what kind of reductions do they use to reach these depths.

I use R= 2 or 3. If I use R > 3 my chess programs makes more blunders.

I also use:

If depth > 3
depth = depth - 3
else depth = depth - 2

to prevent null move pruning from going too early into quiescence search.
I still don't know if this is necessary, but it looks safer.

If you do checking moves in the first ply of q-search, you don't have to do that. If you do not, then preventing the NM search from dropping directly into the q-search is important.

Preventing null move search from dropping directly into the q-search is too expensive to implement. It costs me two plies. Instead I use now
depth > 600 ? depth - 400 : depth - 300.

Allowing checks in q-search is very expensive too.

[Don]

I don't understand how other chess programs can reach search depths > 15 Do they put R = 7 or so when null move pruning, or what kind of reductions do they use to reach these depths.

I use R= 2 or 3. If I use R > 3 my chess programs makes more blunders.

I also use:

If depth > 3
depth = depth - 3
else depth = depth - 2

to prevent null move pruning from going too early into quiescence search.
I still don't know if this is necessary, but it looks safer.

If you do checking moves in the first ply of q-search, you don't have to do that. If you do not, then preventing the NM search from dropping directly into the q-search is important.

Preventing null move search from dropping directly into the q-search is too expensive to implement. It costs me two plies. Instead I use now
depth > 600 ? depth - 400 : depth - 300.

Allowing checks in q-search is very expensive too.

The devil is in the details. Checks on the first ply of quies is the gold standard and WILL slow your program down but is well worth the cost in additional ELO.

[Henk]

I don't understand how other chess programs can reach search depths > 15 Do they put R = 7 or so when null move pruning, or what kind of reductions do they use to reach these depths.

I use R= 2 or 3. If I use R > 3 my chess programs makes more blunders.

I also use:

If depth > 3
depth = depth - 3
else depth = depth - 2

to prevent null move pruning from going too early into quiescence search.
I still don't know if this is necessary, but it looks safer.

If you do checking moves in the first ply of q-search, you don't have to do that. If you do not, then preventing the NM search from dropping directly into the q-search is important.

Preventing null move search from dropping directly into the q-search is too expensive to implement. It costs me two plies. Instead I use now
depth > 600 ? depth - 400 : depth - 300.

Allowing checks in q-search is very expensive too.

The devil is in the details. Checks on the first ply of quies is the gold standard and WILL slow your program down but is well worth the cost in additional ELO.

I doubt that. Null move check is only called when not in PV. The chance of any checks on a level is quite low. By the way if I loose two plies that loss is bigger than the gain. Errors on lower levels may not appear by cuts on higher levels or moves found with higher value than the move with the null move error or maybe a less optimal move but a plausible move is found.

Also if I have two extra plies there will be more null moves above the lowest level that do no drop directly into q-search

[Don]

I don't understand how other chess programs can reach search depths > 15 Do they put R = 7 or so when null move pruning, or what kind of reductions do they use to reach these depths.

I use R= 2 or 3. If I use R > 3 my chess programs makes more blunders.

I also use:

If depth > 3
depth = depth - 3
else depth = depth - 2

to prevent null move pruning from going too early into quiescence search.
I still don't know if this is necessary, but it looks safer.

If you do checking moves in the first ply of q-search, you don't have to do that. If you do not, then preventing the NM search from dropping directly into the q-search is important.

Preventing null move search from dropping directly into the q-search is too expensive to implement. It costs me two plies. Instead I use now
depth > 600 ? depth - 400 : depth - 300.

Allowing checks in q-search is very expensive too.

The devil is in the details. Checks on the first ply of quies is the gold standard and WILL slow your program down but is well worth the cost in additional ELO.

I doubt that. Null move check is only called when not in PV. The chance of any checks on a level is quite low. By the way if I loose two plies that loss is bigger than the gain. Errors on lower levels may not appear by cuts on higher levels or moves found with higher value than the move with the null move error or maybe a less optimal move but a plausible move is found.

Also if I have two extra plies there will be more null moves above the lowest level that do no drop directly into q-search

Ok, whatever you say.

[Henk]

I don't understand how other chess programs can reach search depths > 15 Do they put R = 7 or so when null move pruning, or what kind of reductions do they use to reach these depths.

I use R= 2 or 3. If I use R > 3 my chess programs makes more blunders.

I also use:

If depth > 3
depth = depth - 3
else depth = depth - 2

to prevent null move pruning from going too early into quiescence search.
I still don't know if this is necessary, but it looks safer.

If you do checking moves in the first ply of q-search, you don't have to do that. If you do not, then preventing the NM search from dropping directly into the q-search is important.

Preventing null move search from dropping directly into the q-search is too expensive to implement. It costs me two plies. Instead I use now
depth > 600 ? depth - 400 : depth - 300.

Allowing checks in q-search is very expensive too.

The devil is in the details. Checks on the first ply of quies is the gold standard and WILL slow your program down but is well worth the cost in additional ELO.

I doubt that. Null move check is only called when not in PV. The chance of any checks on a level is quite low. By the way if I loose two plies that loss is bigger than the gain. Errors on lower levels may not appear by cuts on higher levels or moves found with higher value than the move with the null move error or maybe a less optimal move but a plausible move is found.

Also if I have two extra plies there will be more null moves above the lowest level that do no drop directly into q-search

Ok, whatever you say.

I must admit I do not understand why I should add king checks in the first ply of q-search. If I add king checks why not add attacking moves to queen, rook, bishop etc.

[The response to a king check will be a pass (or null move) if there is no way to get out the check. This means that the player giving the check might get a higher value for this q-node than without a check which gives a bigger chance of a fail high. So the parent node has a bigger chance of failing low and no research being performed if pvs is applied at lowest levels of normal seach.][don't know if this last paragraph makes sense]

[Don]

I don't understand how other chess programs can reach search depths > 15 Do they put R = 7 or so when null move pruning, or what kind of reductions do they use to reach these depths.

I use R= 2 or 3. If I use R > 3 my chess programs makes more blunders.

I also use:

If depth > 3
depth = depth - 3
else depth = depth - 2

to prevent null move pruning from going too early into quiescence search.
I still don't know if this is necessary, but it looks safer.

If you do checking moves in the first ply of q-search, you don't have to do that. If you do not, then preventing the NM search from dropping directly into the q-search is important.

Preventing null move search from dropping directly into the q-search is too expensive to implement. It costs me two plies. Instead I use now
depth > 600 ? depth - 400 : depth - 300.

Allowing checks in q-search is very expensive too.

The devil is in the details. Checks on the first ply of quies is the gold standard and WILL slow your program down but is well worth the cost in additional ELO.

I doubt that. Null move check is only called when not in PV. The chance of any checks on a level is quite low. By the way if I loose two plies that loss is bigger than the gain. Errors on lower levels may not appear by cuts on higher levels or moves found with higher value than the move with the null move error or maybe a less optimal move but a plausible move is found.

Also if I have two extra plies there will be more null moves above the lowest level that do no drop directly into q-search

Ok, whatever you say.

I must admit I do not understand why I should add king checks in the first ply of q-search. If I add king checks why not add attacking moves to queen, rook, bishop etc.

[The response to a king check will be a pass (or null move) if there is no way to get out the check. This means that the player giving the check might get a higher value for this q-node than without a check which gives a bigger chance of a fail high. So the parent node has a bigger chance of failing low and no research being performed if pvs is applied at lowest levels of normal seach.][don't know if this last paragraph makes sense]

Every single top program has checks in quies including Critter, Komodo, Stockfish and Houdini. If your program doesn't understand checkmate and the program just passes after a check then there is no point of course but you should be doing that. Komodo's quies will try to get out of check if any quies move is a check, so in this situation we only look at out of check moves, whether they are captures or not.

It too expensive to look at them everywhere, so we look at just the first ply of quies and I think we currently look at a second ply under some restricted conditions but that is not important, the majority of the benefit is in the first ply.

Checks in quies works well with null move pruning by the way, it's even critical. You typically try a null move as a way of detecting threats. Your score is already above beta and you would like to take the cutoff but the opponent has a big threat. So you try a null move and then a seriously reduced search and having checks in quies picks up all sorts of critical tactics in this scenario.

I remember years ago Frans Morsch telling me that "with aggressive null move pruning you must have checks in quies" and every test I have run since then shows his words to be true.

It's also critical to futility pruning - the algorithm where you drop into quies a ply early if you score is far below alpha.

[Henk]

I don't understand how other chess programs can reach search depths > 15 Do they put R = 7 or so when null move pruning, or what kind of reductions do they use to reach these depths.

I use R= 2 or 3. If I use R > 3 my chess programs makes more blunders.

I also use:

If depth > 3
depth = depth - 3
else depth = depth - 2

to prevent null move pruning from going too early into quiescence search.
I still don't know if this is necessary, but it looks safer.

If you do checking moves in the first ply of q-search, you don't have to do that. If you do not, then preventing the NM search from dropping directly into the q-search is important.

Preventing null move search from dropping directly into the q-search is too expensive to implement. It costs me two plies. Instead I use now
depth > 600 ? depth - 400 : depth - 300.

Allowing checks in q-search is very expensive too.

The devil is in the details. Checks on the first ply of quies is the gold standard and WILL slow your program down but is well worth the cost in additional ELO.

I doubt that. Null move check is only called when not in PV. The chance of any checks on a level is quite low. By the way if I loose two plies that loss is bigger than the gain. Errors on lower levels may not appear by cuts on higher levels or moves found with higher value than the move with the null move error or maybe a less optimal move but a plausible move is found.

Also if I have two extra plies there will be more null moves above the lowest level that do no drop directly into q-search

Ok, whatever you say.

I must admit I do not understand why I should add king checks in the first ply of q-search. If I add king checks why not add attacking moves to queen, rook, bishop etc.

[The response to a king check will be a pass (or null move) if there is no way to get out the check. This means that the player giving the check might get a higher value for this q-node than without a check which gives a bigger chance of a fail high. So the parent node has a bigger chance of failing low and no research being performed if pvs is applied at lowest levels of normal seach.][don't know if this last paragraph makes sense]

Every single top program has checks in quies including Critter, Komodo, Stockfish and Houdini. If your program doesn't understand checkmate and the program just passes after a check then there is no point of course but you should be doing that. Komodo's quies will try to get out of check if any quies move is a check, so in this situation we only look at out of check moves, whether they are captures or not.

It too expensive to look at them everywhere, so we look at just the first ply of quies and I think we currently look at a second ply under some restricted conditions but that is not important, the majority of the benefit is in the first ply.

Checks in quies works well with null move pruning by the way, it's even critical. You typically try a null move as a way of detecting threats. Your score is already above beta and you would like to take the cutoff but the opponent has a big threat. So you try a null move and then a seriously reduced search and having checks in quies picks up all sorts of critical tactics in this scenario.

I remember years ago Frans Morsch telling me that "with aggressive null move pruning you must have checks in quies" and every test I have run since then shows his words to be true.

It's also critical to futility pruning - the algorithm where you drop into quies a ply early if you score is far below alpha.

Don't know what to think now. Looks like my chess program got one, two extra plies but is polluted by erroneous null move checks. Repairing it is too costly. Looks like this thread is about minimizing search depths.

Futility pruning does not work for me. Comparing my chess programs with top programs makes no sense. If you loose two plies but already got twenty plies or more you can afford it. I may be happy if I can get ten plies.

I think these stronger chess programs have very fast king check detection.
My program doesn't even have a mate check detection. It just captures the king.

Maybe I should verify the null move fail high with a fast normal search check or mate check. But I think it will all be too expensive.


I read about comments that q-search is crap anyhow. [It's just a way of rounding (stopping) evaluation.] So one should reduce work in q-search as much as possible.


[If I want to be safe I should do no null move check at all.]

I am not sure if applying null move only, when not in PV prohibits too many null move reductions applied after one another, for I read it is malicious.

Applying null move only once (and not recursive) will cost me three plies or more.

[Don]

I don't understand how other chess programs can reach search depths > 15 Do they put R = 7 or so when null move pruning, or what kind of reductions do they use to reach these depths.

I use R= 2 or 3. If I use R > 3 my chess programs makes more blunders.

I also use:

If depth > 3
depth = depth - 3
else depth = depth - 2

to prevent null move pruning from going too early into quiescence search.
I still don't know if this is necessary, but it looks safer.

If you do checking moves in the first ply of q-search, you don't have to do that. If you do not, then preventing the NM search from dropping directly into the q-search is important.

Preventing null move search from dropping directly into the q-search is too expensive to implement. It costs me two plies. Instead I use now
depth > 600 ? depth - 400 : depth - 300.

Allowing checks in q-search is very expensive too.

The devil is in the details. Checks on the first ply of quies is the gold standard and WILL slow your program down but is well worth the cost in additional ELO.

I doubt that. Null move check is only called when not in PV. The chance of any checks on a level is quite low. By the way if I loose two plies that loss is bigger than the gain. Errors on lower levels may not appear by cuts on higher levels or moves found with higher value than the move with the null move error or maybe a less optimal move but a plausible move is found.

Also if I have two extra plies there will be more null moves above the lowest level that do no drop directly into q-search

Ok, whatever you say.

I must admit I do not understand why I should add king checks in the first ply of q-search. If I add king checks why not add attacking moves to queen, rook, bishop etc.

[The response to a king check will be a pass (or null move) if there is no way to get out the check. This means that the player giving the check might get a higher value for this q-node than without a check which gives a bigger chance of a fail high. So the parent node has a bigger chance of failing low and no research being performed if pvs is applied at lowest levels of normal seach.][don't know if this last paragraph makes sense]

Every single top program has checks in quies including Critter, Komodo, Stockfish and Houdini. If your program doesn't understand checkmate and the program just passes after a check then there is no point of course but you should be doing that. Komodo's quies will try to get out of check if any quies move is a check, so in this situation we only look at out of check moves, whether they are captures or not.

It too expensive to look at them everywhere, so we look at just the first ply of quies and I think we currently look at a second ply under some restricted conditions but that is not important, the majority of the benefit is in the first ply.

Checks in quies works well with null move pruning by the way, it's even critical. You typically try a null move as a way of detecting threats. Your score is already above beta and you would like to take the cutoff but the opponent has a big threat. So you try a null move and then a seriously reduced search and having checks in quies picks up all sorts of critical tactics in this scenario.

I remember years ago Frans Morsch telling me that "with aggressive null move pruning you must have checks in quies" and every test I have run since then shows his words to be true.

It's also critical to futility pruning - the algorithm where you drop into quies a ply early if you score is far below alpha.

Don't know what to think now. Looks like my chess program got one, two extra plies but is polluted by erroneous null move checks. Repairing it is too costly. Looks like this thread is about minimizing search depths.

First of all, it's about ELO, not depth. Secondly, if you are getting 2 ply for not doing checks on just the first ply of quies you are doing something seriously wrong. Find your bug.

If you are generating every single move just to see which ones are checks, that will be a major slowdown, but it does not explain 1 or 2 ply of slowdown.

Futility pruning does not work for me. Comparing my chess programs with top programs makes no sense. If you loose two plies but already got twenty plies or more you can afford it. I may be happy if I can get ten plies.

Futility pruning should give you a huge speedup - it is a must have. In Komodo we look at the static score and if it's "significantly" below alpha we make the decision to only try captures and checks at this node. Take care that if there are no captures or checks you don't treat this a position with no legal moves and call it a stalemate. Also, never do this when in check.

I think these stronger chess programs have very fast king check detection.
My program doesn't even have a mate check detection. It just captures the king.

This is probably a mistake, but it's not the worst thing you could do as long as you do have proper mate detection in the main search. You can get away with this in the quies for a small ELO loss.

Maybe I should verify the null move fail high with a fast normal search check or mate check. But I think it will all be too expensive.


I read about comments that q-search is crap anyhow. [It's just a way of rounding (stopping) evaluation.] So one should reduce work in q-search as much as possible.


[If I want to be safe I should do no null move check at all.]

What is a null move check? You should never do null move when you are in check, is that what you mean?

I am not sure if applying null move only, when not in PV prohibits too many null move reductions applied after one another, for I read it is malicious.

Applying null move only once (and not recursive) will cost me three plies or more.

There is nothing wrong with recursive null move pruning, we all do this.

[Henk]

Tried futility pruning several times, it never gave a noticeable speed up, maybe it's because null move pruning skips last two or three levels above quiescence. Or maybe because you need another evaluation in internal nodes:

val = eval();
for each move
{
if ( !move is promotion)
{
int margin = capture != null ? margin2 : capture.Points + margin2;
if ( val + margin < alfa))
continue;
}
...
}

[elpapa]

The devil is in the details. Checks on the first ply of quies is the gold standard and WILL slow your program down but is well worth the cost in additional ELO.

I doubt that.

Henk, Don is the author of one of the strongest engines in the world and your engine played this game against itself:

e4, e5, b1-c3, f8-c5, c3-a4, c5-f8, a4-c3 (draw by repetition)

Maybe you should listen to what he has to say.