reducing too much

[AndrewShort]

My qsearch doesn't examine checks at all. I do have check extensions in the main search, though. For null move, my R = 2 (static, everywhere). Given this information, would it be too risky to use R = 3 ? I think so, but I wanted to ask...

Also, I recently added LMR (the whole hardware/software debate got me curious, and Bob mentioned LMR as a huge advancement in software, so I added it - easy to implement, actually) - I am now noticing huge reductions in tree size, and my engine is reaching deeper depths in less time - it's actually pretty incredible - haven't seen this kind of improvement since I put in null move. In my own testing, some of my positions are performing much better, but others are performing worse - requiring more plies to find mates against the opponent or against itself, for example.

Just as it is possible to over use extensions and explode your tree, it is also possible to over reduce and chop off too many branches of the tree - are there some good test positions for this? In other words, some positions where the engine should find the "answer" (mate, capture, whatever) in a certain number of depth/time/nodes, and if not, then it means you have over reduced. That would help me determine if LMR and other reductions have actually hurt me....because I can't play 30,000 games to measure the ELO change...

I just think there have to be some test positions where any engine should be able to find the proper move within a specfied depth/time/nodes, and if not, then it probably means the engine is reducing too much.

[BubbaTough]

My qsearch doesn't examine checks at all. I do have check extensions in the main search, though. For null move, my R = 2 (static, everywhere). Given this information, would it be too risky to use R = 3 ? I think so, but I wanted to ask...

Also, I recently added LMR (the whole hardware/software debate got me curious, and Bob mentioned LMR as a huge advancement in software, so I added it - easy to implement, actually) - I am now noticing huge reductions in tree size, and my engine is reaching deeper depths in less time - it's actually pretty incredible - haven't seen this kind of improvement since I put in null move. In my own testing, some of my positions are performing much better, but others are performing worse - requiring more plies to find mates against the opponent or against itself, for example.

Just as it is possible to over use extensions and explode your tree, it is also possible to over reduce and chop off too many branches of the tree - are there some good test positions for this? In other words, some positions where the engine should find the "answer" (mate, capture, whatever) in a certain number of depth/time/nodes, and if not, then it means you have over reduced. That would help me determine if LMR and other reductions have actually hurt me....because I can't play 30,000 games to measure the ELO change...

I just think there have to be some test positions where any engine should be able to find the proper move within a specfied depth/time/nodes, and if not, then it probably means the engine is reducing too much.

Risk...bah....there is no such thing as risk. Change your R, run some test games, and enjoy the minor improvement in your chess engine strength once you are convinced it works.

-Sam

[bob]

My qsearch doesn't examine checks at all. I do have check extensions in the main search, though. For null move, my R = 2 (static, everywhere). Given this information, would it be too risky to use R = 3 ? I think so, but I wanted to ask...

Hard to answer. And I never ran this test so I can not speak with any supporting data. I would suspect it is more dangerous to use R=3 if all you do in q-search is captures. But it might be an elo or two, or it might be 10 or 20. I am not sure.

Also, I recently added LMR (the whole hardware/software debate got me curious, and Bob mentioned LMR as a huge advancement in software, so I added it - easy to implement, actually) - I am now noticing huge reductions in tree size, and my engine is reaching deeper depths in less time - it's actually pretty incredible - haven't seen this kind of improvement since I put in null move. In my own testing, some of my positions are performing much better, but others are performing worse - requiring more plies to find mates against the opponent or against itself, for example.

Just remember that these "plies" are no longer the same thing as your older "plies". If you don't change anything, it is generally considered that a ply is worth 50-70 elo. But not _these_ plies. You probably got 2-3 more plies with LMR, but you only got about 80 Elo or so according to my recent testing. So these "plies" are somehow inferior because the reductions introduce errors.

Just as it is possible to over use extensions and explode your tree, it is also possible to over reduce and chop off too many

The only warning I would give is to _not_ rely on test positions. I would trust very fast games over test positions any time.

branches of the tree - are there some good test positions for this? In other words, some positions where the engine should find the "answer" (mate, capture, whatever) in a certain number of depth/time/nodes, and if not, then it means you have over reduced. That would help me determine if LMR and other reductions have actually hurt me....because I can't play 30,000 games to measure the ELO change...[

I just think there have to be some test positions where any engine should be able to find the proper move within a specfied depth/time/nodes, and if not, then it probably means the engine is reducing too much.

[hgm]

There are no general rules for this. Joker80's strength dramatically drops (50-100 Elo) when I used R=3 in stead of R=2. So it pretty much all depends on the engine. The only way to know what works for you is to try it out. Just play some 500 games (e.g. a Nunn match of 20 games against each of 25 selected opponents similar in strength to your engine) with R=2, and then repeat that with R=3, and you will know if it makes a big difference. (They can be 40/1 or 1+1 games, so you can do ~20 games an hour on a single CPU.)

One caveat: make sure your all engines in the test (including your own) avoid repetition draws while ahead. It is very difficult to spot improvements if the engines fall into a draw trap.

[Michael Sherwin]

Things to try:

1.) do not use LMR after a null move.

2.) use LMR after a null move, but increase the number of prior moves tried first while in null move.

3.) use R=3 normally except when in an LMR reduced line, then use R=2

4.) do not use null move if a line is 'over reduced' by LMR.

[AndrewShort]

My qsearch doesn't examine checks at all. I do have check extensions in the main search, though. For null move, my R = 2 (static, everywhere). Given this information, would it be too risky to use R = 3 ? I think so, but I wanted to ask...

Also, I recently added LMR (the whole hardware/software debate got me curious, and Bob mentioned LMR as a huge advancement in software, so I added it - easy to implement, actually) - I am now noticing huge reductions in tree size, and my engine is reaching deeper depths in less time - it's actually pretty incredible - haven't seen this kind of improvement since I put in null move. In my own testing, some of my positions are performing much better, but others are performing worse - requiring more plies to find mates against the opponent or against itself, for example.

Just as it is possible to over use extensions and explode your tree, it is also possible to over reduce and chop off too many branches of the tree - are there some good test positions for this? In other words, some positions where the engine should find the "answer" (mate, capture, whatever) in a certain number of depth/time/nodes, and if not, then it means you have over reduced. That would help me determine if LMR and other reductions have actually hurt me....because I can't play 30,000 games to measure the ELO change...

I just think there have to be some test positions where any engine should be able to find the proper move within a specfied depth/time/nodes, and if not, then it probably means the engine is reducing too much.

Reductions such as null move and LMR can mean that the engine moves, thinking it has a forced mate against you, for example, only to find that after your response, it discovers it didn't really have a forced mate, as the refutation to the mate was reduced away to lala land earlier. Correct? My engine now gloriously announces mate in X, but I'm afraid with reductions, it could end up with egg on its face...

[hgm]

No, it can't. It can only think it has a forced mate (or you have a forced mate) if every branch the opponent can select has a mate within the horizon.

The opposite can happen, though: that there is a forced mate, and the engine doesn't spot it. But this can happen in any search that is not infinitely deep, also without reductions. Only with reductions it is possible that there is a mate in 11 ply, the engine reports searching 12 ply, and still doesn't see it. (I have never seen it happen, but it is theoretically possible.) But then again, without the reductions the engine would likely search only 9 ply in the same time, and it would also not have seen the mate.

One way to avoid this problem with mates is to switch off null-move pruning and LMR when the search window is in the mating score range (e.g. alpha > MATE-100 or beta < -MATE+100).

[AndrewShort]

No, it can't. It can only think it has a forced mate (or you have a forced mate) if every branch the opponent can select has a mate within the horizon.

The opposite can happen, though: that there is a forced mate, and the engine doesn't spot it. But this can happen in any search that is not infinitely deep, also without reductions. Only with reductions it is possible that there is a mate in 11 ply, the engine reports searching 12 ply, and still doesn't see it. (I have never seen it happen, but it is theoretically possible.) But then again, without the reductions the engine would likely search only 9 ply in the same time, and it would also not have seen the mate.

One way to avoid this problem with mates is to switch off null-move pruning and LMR when the search window is in the mating score range (e.g. alpha > MATE-100 or beta < -MATE+100).

I have a mating puzzle where if the engine plays the wrong move, it gets mated on ply 6. With null move (R=2), it requires a depth 7 search to play the proper mate evasion move. (it sees the proper move with depth 6 if R=1). Now that I've added LMR, it needs a level 8 search to see the proper mate evasion move (null move R is still 2). Of course, that level 8 search is fast, since so much was reduced.

So couldn't the engine think, find forced mate for/against, but then on the next move, fail the find the mate that it found one move prior? As you say, the mate announcement was correct, but on the next move, there is no mate announcement, because it failed to see the mate again (even though it's there). I guess your idea above would avoid that...

[hgm]

I am not sure I understand the last part. Is it the winning engine now that makes the announcement and plays the correct move to effect the mate, and then, after your reply, on the next search cannot find the mate anymore? What do you mean by "fail find the mate" anyway? If it searches deep enough, it must find it. So how deep were you searching when it saw it, and how deep were you searching one move later?

It is very unusual that an engine fails to find a mate it saw before, as the moves leading to the mate should still be in the hash table from the previous move.

[AndrewShort]

I am not sure I understand the last part. Is it the winning engine now that makes the announcement and plays the correct move to effect the mate, and then, after your reply, on the next search cannot find the mate anymore? What do you mean by "fail find the mate" anyway? If it searches deep enough, it must find it. So how deep were you searching when it saw it, and how deep were you searching one move later?

It is very unusual that an engine fails to find a mate it saw before, as the moves leading to the mate should still be in the hash table from the previous move.

I think it was simply a timing issue. I can't remember the circumstances, but I'm pretty sure I was playing against my own engine, and after 20 seconds or so it announced a deep mate against me (iter deep had timed out on its own because mate was within the horizon), but then after my reply, I let it think for just a couple of seconds, then I did a user timeout to force it to move, and it moved without a mate announcement. I guess it just didn't have enough time. I must have done the user time out too quickly. I'll keep an eye on anything funny...