Crafty egtb check condition

[jwes]

I was looking at the crafty code that decides if it should do an egtb probe. There is a test to see if the last move was a capture. This makes sense at first glance, but if the last move was not a capture, wouldn't the search have done an egtb probe at the ply before and cut off (except at the top of the tree) which would make the test unnecessary?

[bob]

I was looking at the crafty code that decides if it should do an egtb probe. There is a test to see if the last move was a capture. This makes sense at first glance, but if the last move was not a capture, wouldn't the search have done an egtb probe at the ply before and cut off (except at the top of the tree) which would make the test unnecessary?

You only want to probe after a capture. Since we probe at the start of a ply, the last ply must have been a capture or a probe would be pointless. At any time you capture and end up in an N-piece ending for which you have a table, you get a score and the search stops. If the capture didn't take you to an ending you have a table for (too many pieces, etc) then probing after non-captures can't lead you to a table position either...

[jwes]

I was looking at the crafty code that decides if it should do an egtb probe. There is a test to see if the last move was a capture. This makes sense at first glance, but if the last move was not a capture, wouldn't the search have done an egtb probe at the ply before and cut off (except at the top of the tree) which would make the test unnecessary?

You only want to probe after a capture. Since we probe at the start of a ply, the last ply must have been a capture or a probe would be pointless. At any time you capture and end up in an N-piece ending for which you have a table, you get a score and the search stops. If the capture didn't take you to an ending you have a table for (too many pieces, etc) then probing after non-captures can't lead you to a table position either...

Of course you only want to probe after a capture, but you do not need to test for it. If you probe anytime you have few enough pieces, you will probe only after a capture since if the last move was not a capture, you would have done the egtb probe some plies before when there was a capture. Another (extremely rare) problem is that if you lose castling rights when there are few enough pieces for an egtb probe, you will not do the probe without another capture.

[michiguel]

I was looking at the crafty code that decides if it should do an egtb probe. There is a test to see if the last move was a capture. This makes sense at first glance, but if the last move was not a capture, wouldn't the search have done an egtb probe at the ply before and cut off (except at the top of the tree) which would make the test unnecessary?

You only want to probe after a capture. Since we probe at the start of a ply, the last ply must have been a capture or a probe would be pointless. At any time you capture and end up in an N-piece ending for which you have a table, you get a score and the search stops. If the capture didn't take you to an ending you have a table for (too many pieces, etc) then probing after non-captures can't lead you to a table position either...

This is true for Nalimov, but not for Gaviota TBs because you can choose to probe only the cache (probe "soft"), for instance, close to the leaves. When you do that, you are no guaranteed to get an answer. You may try again down the tree and be successful. Whoever uses Gaviota TBs I may recommend not to include this test if the probing soft is used.

Miguel

[wgarvin]

I was looking at the crafty code that decides if it should do an egtb probe. There is a test to see if the last move was a capture. This makes sense at first glance, but if the last move was not a capture, wouldn't the search have done an egtb probe at the ply before and cut off (except at the top of the tree) which would make the test unnecessary?

You only want to probe after a capture. Since we probe at the start of a ply, the last ply must have been a capture or a probe would be pointless. At any time you capture and end up in an N-piece ending for which you have a table, you get a score and the search stops. If the capture didn't take you to an ending you have a table for (too many pieces, etc) then probing after non-captures can't lead you to a table position either...

Of course you only want to probe after a capture, but you do not need to test for it. If you probe anytime you have few enough pieces, you will probe only after a capture since if the last move was not a capture, you would have done the egtb probe some plies before when there was a capture. Another (extremely rare) problem is that if you lose castling rights when there are few enough pieces for an egtb probe, you will not do the probe without another capture.

I vaguely remember this question being asked a few months ago (or years?) I think the conclusion was that if you don't have a full set of TBs, you might not cut off after the capture -- and then it would be pointless to keep querying after every non-converting move. So I guess the check is useful in that particular case.

In theory you could also have some databases which are just a partial subset of a whole ending (i.e. only some of the pawn placements are covered, for example) in which case you would want to check for any converting move (incl. pawn push) rather than just checking for a capture/promotion. But with Nalimov tablebases that will never be an issue, they are all-or-nothing.

There's another edge case that can happen if your set of tablebases is incomplete: you could get into a situation where you have a tablebase position over the board, and you follow the TB and play a converting move into a different TB which you don't have, and have to go back to thinking. I don't know how Crafty handles that case.

[bob]

I was looking at the crafty code that decides if it should do an egtb probe. There is a test to see if the last move was a capture. This makes sense at first glance, but if the last move was not a capture, wouldn't the search have done an egtb probe at the ply before and cut off (except at the top of the tree) which would make the test unnecessary?

You only want to probe after a capture. Since we probe at the start of a ply, the last ply must have been a capture or a probe would be pointless. At any time you capture and end up in an N-piece ending for which you have a table, you get a score and the search stops. If the capture didn't take you to an ending you have a table for (too many pieces, etc) then probing after non-captures can't lead you to a table position either...

Of course you only want to probe after a capture, but you do not need to test for it. If you probe anytime you have few enough pieces, you will probe only after a capture since if the last move was not a capture, you would have done the egtb probe some plies before when there was a capture. Another (extremely rare) problem is that if you lose castling rights when there are few enough pieces for an egtb probe, you will not do the probe without another capture.

You forget how many people don't have a full set of egtbs, particularly when you factor in 6 piece tables. What then? If you probe only after a capture, and you don't have that table, why would you want to keep probing at any node below that point, where you _still_ won't have the table??? This cuts the probes back to only occur where you could actually jump into a table, and ignores any case where you might already "be there" since if you had the table, you could never "be there" since the search would have already been terminated.

This issue came up many years ago and was discussed at length. I don't see another solution since missing tables are a given.

[jwes]

I was looking at the crafty code that decides if it should do an egtb probe. There is a test to see if the last move was a capture. This makes sense at first glance, but if the last move was not a capture, wouldn't the search have done an egtb probe at the ply before and cut off (except at the top of the tree) which would make the test unnecessary?

You only want to probe after a capture. Since we probe at the start of a ply, the last ply must have been a capture or a probe would be pointless. At any time you capture and end up in an N-piece ending for which you have a table, you get a score and the search stops. If the capture didn't take you to an ending you have a table for (too many pieces, etc) then probing after non-captures can't lead you to a table position either...

Of course you only want to probe after a capture, but you do not need to test for it. If you probe anytime you have few enough pieces, you will probe only after a capture since if the last move was not a capture, you would have done the egtb probe some plies before when there was a capture. Another (extremely rare) problem is that if you lose castling rights when there are few enough pieces for an egtb probe, you will not do the probe without another capture.

You forget how many people don't have a full set of egtbs, particularly when you factor in 6 piece tables. What then? If you probe only after a capture, and you don't have that table, why would you want to keep probing at any node below that point, where you _still_ won't have the table??? This cuts the probes back to only occur where you could actually jump into a table, and ignores any case where you might already "be there" since if you had the table, you could never "be there" since the search would have already been terminated.

This issue came up many years ago and was discussed at length. I don't see another solution since missing tables are a given.

That makes sense. You can have a minor performance gain (one poorly predicted jump) at the cost of a significant performance hit if you have an incomplete set of egtbs.

Another question - I see that you do not do egtb probes if the depth is > iteration_depth. This means that lines that are reduced will do egtb probes until they enter qsearch while lines that are extended may stop doing probes several plies before qsearch. This seems backward to me. If it were based on depth remaining, it could also be configurable as someone with a slow laptop might well not want to do egtb probes as deep as someone with a fast ssd and a lot of memory for egtb cache.

[bob]

I was looking at the crafty code that decides if it should do an egtb probe. There is a test to see if the last move was a capture. This makes sense at first glance, but if the last move was not a capture, wouldn't the search have done an egtb probe at the ply before and cut off (except at the top of the tree) which would make the test unnecessary?

You only want to probe after a capture. Since we probe at the start of a ply, the last ply must have been a capture or a probe would be pointless. At any time you capture and end up in an N-piece ending for which you have a table, you get a score and the search stops. If the capture didn't take you to an ending you have a table for (too many pieces, etc) then probing after non-captures can't lead you to a table position either...

Of course you only want to probe after a capture, but you do not need to test for it. If you probe anytime you have few enough pieces, you will probe only after a capture since if the last move was not a capture, you would have done the egtb probe some plies before when there was a capture. Another (extremely rare) problem is that if you lose castling rights when there are few enough pieces for an egtb probe, you will not do the probe without another capture.

You forget how many people don't have a full set of egtbs, particularly when you factor in 6 piece tables. What then? If you probe only after a capture, and you don't have that table, why would you want to keep probing at any node below that point, where you _still_ won't have the table??? This cuts the probes back to only occur where you could actually jump into a table, and ignores any case where you might already "be there" since if you had the table, you could never "be there" since the search would have already been terminated.

This issue came up many years ago and was discussed at length. I don't see another solution since missing tables are a given.

That makes sense. You can have a minor performance gain (one poorly predicted jump) at the cost of a significant performance hit if you have an incomplete set of egtbs.

Another question - I see that you do not do egtb probes if the depth is > iteration_depth. This means that lines that are reduced will do egtb probes until they enter qsearch while lines that are extended may stop doing probes several plies before qsearch. This seems backward to me. If it were based on depth remaining, it could also be configurable as someone with a slow laptop might well not want to do egtb probes as deep as someone with a fast ssd and a lot of memory for egtb cache.

I've tried lots of ways to limit probing. Even with the current approach, overhead can go through the roof in the right positions. Whether there is a better way or not is unknown. I'm personally not a big fan of egtbs anyway. They are large, inefficient, and as the number of pieces goes up, the overhead goes up exponentially...