Natural TB

[jhellis3]

This is the latest version of "Natural TB", with TB probing moved at the end of the search(*)

(*) I know that Ronald, the syzygy author, thinks this is a "trivial idea", but apart from some general arguments about the smartness of people involved in TB before me, I didn't get any specific refutation, so I have tried it...I love trivial ideas Smile

I think Ronald hinted pretty clearly at the performance implications. Certainly you can search first, but there are two aspects to the benefits of TBs: correct eval and search pruning. If you search first, you are missing out on the latter. So while the TBs may work OK, you are almost certainly losing Elo this way. *Note this is most important/valuable for draws.

Using only WDL, you need to preserve the draws (eventually 50 moves draw) detected by the engine and so you need first the engine to search then

50 move draws are almost never seen by search (engine) until it is far too late. The reason for this is that the engine must search every viable path 50 moves deep, and this is just not going to happen. In turn, this is a large part (often overlooked) of what makes TBs so valuable (Elo wise), because they can tell the engine, "no this a draw, don't waste your time or go down this path". If you let the engine search TB draws, you are missing out on a huge chunk of the performance (Elo) gain.

This is why I always save and return on draws in my patch.

Code: Select all

 if (    abs&#40;v&#41; <= drawScore
              || &#40;abs&#40;ev&#41; < VALUE_KNOWN_WIN && (!ttHit || &#40;ttHit && abs&#40;ttValue&#41; < VALUE_KNOWN_WIN&#41;)))

[syzygy]

I can somewhat understand you: TB is your brainchild, it took you a lot of time (since 2000) and efforts to come up with syzygy and I have huge respect for this and for your persistence on this difficult topic.

Please be aware that I have never asked for inclusion of the code in Stockfish. That it was integrated was not my decision and if it is taken out then I am, of course, completely fine with that. And if instead you prefer to cripple it, then also fine with me and I have anyway no say about it.

I'm just not going to spend time explaining why your changes break something that worked completely fine. (I am not saying that improvements are not possible. Certainly some improvements are possible. Searching "through" TB probes to find mates would be an improvement if this can be done right. Also dealing with cursed wins could probably be improved, e.g. by scoring a cursed win as +0.5 or so while continue to score a cursed loss as -0.01. But just random hacking without understanding is not going to do it.)

The only reason I entered this thread is that you were claiming to have implemented this change:

1) At root, moves that do not preserve the win or the draw are skipped, but the DTZ score of the remaining moves is ignored.

As I already explained:
1. SF already filtered out the moves that did not preserve the win or draw; and
2. The way you are doing it now is broken as it does not take into account the 50-move rule.

So you did not implement something new, you only broke something that worked.

If this is too difficult, then well, what I wrote was "intended for a technical audience".

[syzygy]

Even the whole concept of search instability is filled with myths and legends and anecdotes

And I'm afraid this is why SF usually is unable to find a mate score. Have you never witnessed its wildly oscillating evaluations? Why do you think that happens? (Sure, it does not cost Elo.)

(I haven't checked lately, but I'd be surprised if it has changed. At some point Eelco improved it a lot exactly by paying attention to search stability.)

[syzygy]

And you're perfectly right not to accept general arguments about the smartness of people as a proof, because they dont prove anything.

What does prove something is the code that existed.

So, instead of making random changes without understanding and then asking here whether the random change will work or not, one *could* also:
- accept that the existing code was not random gibberish;
- try to understand the code and understand why things are done the way they are being done;
- and only once the understanding is there, see if something could perhaps be improved.

So this is what I said. The first step is the acceptance that what is there already is not stupid. (Instead, I am seeing things on the level of "why do you make those TBs so large, do you not know that you can leave out the FENs for all draws".)

[mcostalba]

- try to understand the code and understand why things are done the way they are being done;

I think I can qualify as someone that is trying very hard to understand your code. And you know it.

(Instead, I am seeing things on the level of "why do you make those TBs so large, do you not know that you can leave out the FENs for all draws".)

I don't understand this part.

[mcostalba]

So while the TBs may work OK, you are almost certainly losing Elo this way. *Note this is most important/valuable for draws.

Yes probing after the search and also everywhere and not only at rule 50 reset seems inefficient. OTH not all the searches reach the end, many are TT or null move pruned earlier. Indeed number of probed nodes is less than 20% on a bench on endgame positions.

As first step I would like to confirm that current implementation does not show the typical artifacts of legacy syzygy, then I will measure how performance is affected. I agree this is an important point.

Regarding the draws, there is also repetition draw, not only 50 rule.

[syzygy]

- try to understand the code and understand why things are done the way they are being done;

I think I can qualify as someone that is trying very hard to understand your code.

Then how can you miss the basic concept of TB probing: once you get in a TB position, probe it immediately so you know the outcome of the position and can prune the whole subtree. Or decide that probing is too much effort compared to the work it saves and not probe in that subtree at all (i.e. do not probe if depth is smaller than whatever the user set probedepth to). Only if you are really looking for a mate can it make sense to do anything else.

Looking for a mate is the RARE exception. Once you're looking for a mate the game is already won or lost (but you can still blunder the win away by incorrectly using the TBs). Most of the time the engine will simply be looking for a good move and the TBs could have helped to cut off branches for which the TBs immediately give the outcome.

If you probe immediately once you get into the TBs, the position can only be a position immediately after a capture (if you don't know why, think harder).

The longer you delay the probe, the more nodes of the subtree you will end up searching and probing. All of that is wasted effort (unless you don't probe in the subtree at all, but for that there is probedepth or there could be other, smarter heuristics). Yes, it is simply plain thought and logic that shows that it is wasted effort. No need to measure or test anything at all. Some things are just elementary truths. I know you don't believe in that, but that does not make it different.

In my view, what I just wrote is something that should not contain anything surprising to someone who has tried to understand this stuff a little bit.

The cursed win/loss information returned by probe_wdl() can only be trusted for positions with the 50-move counter at 0. Surprise? Think.

[syzygy]

So while the TBs may work OK, you are almost certainly losing Elo this way. *Note this is most important/valuable for draws.

Yes probing after the search and also everywhere and not only at rule 50 reset seems inefficient. OTH not all the searches reach the end, many are TT or null move pruned earlier. Indeed number of probed nodes is less than 20% on a bench on endgame positions.

"seems" inefficient.

The choice is between:

(A) probe immediately, prune the whole subtree and return the correct result

and

(B) do not probe immediately but search the subtree, probing at some of the leaves (which costs more than (A) once you probe twice) and pruning other branches of the subtree (inevitably introducing errors) and finally return merely an estimate of the correct value.

Have fun measuring...

[syzygy]

As first step I would like to confirm that current implementation does not show the typical artifacts of legacy syzygy

Do you have any well-established idea of those "typical artifacts"? Could you write down what they are?

[mcostalba]

If you probe immediately once you get into the TBs, the position can only be a position immediately after a capture (if you don't know why, think harder).

As long as you don't have 32-men TB

Indeed there is one case where probing earlier is always correct and does not introduces artifacts(*), that is when WDL returns a draw position. In this case if we return immediately a draw score it can't be wrong.

But is a waste of resources to probe at the beginning just for consuming a draw score. So I have modified the probing at the end of the search to store in TT a special VALUE_TB_DRAW that in no other case can appear as value in a TT (apart form the usual and harmless SMP races).

When probing TT at the beginning of the search we return immediately and unconditionally in case the position is flagged as VALUE_TB_DRAW.

This has many advantages (avoid searching subtree when useless, do not introduce artifacts, always return correct score, cover the most important cases in real games: when game is already win/lost there is very little added value in using TB above usual search) and no disadvantage (at least I am not able to see them).

With this small change I have halved the time requested to bench over 600 endgames positions and greatly reduced time to depth compared to default NO TB case.

(*) regarding artifacts here we go:

- Very difficult to find mates
- Return very high/low unnatural scores
- Sacrifice piece to simplify position in case the resulting position is win in TB: this is NOT what a good human player would do in such cases and so in unnatural for me.