Natural TB

[zullil]

If the aim is to remove "unnatural behaviour", then I guess the underlying aim is to improve SF for end users. What do end users want? Do they want to be informed that the position they are analysing is a TB win? Or do they prefer to be kept in the dark about whether the engine has already solved the position with the help of TBs?

Do users prefer to be kept in the dark? Marco seems to think so.

Worrying about what end-users want has never seemed a priority of the Stockfish team. Their goal has been to develop a world-class chess engine with source code that is open and clean. This seems like a perfectly fine goal, and I'm puzzled why this one behavior appears to be getting special attention.

[syzygy]

If the aim is to remove "unnatural behaviour", then I guess the underlying aim is to improve SF for end users. What do end users want? Do they want to be informed that the position they are analysing is a TB win? Or do they prefer to be kept in the dark about whether the engine has already solved the position with the help of TBs?

Do users prefer to be kept in the dark? Marco seems to think so.

Worrying about what end-users want has never seemed a priority of the Stockfish team.

Yes, that was an inside joke.

If it had been up to Marco, the TB code would have been where the large page code is today. It accidentally was integrated during a short period in which Marco had relinquished control over SF. Since he took back control he has not stopped complaining about the "completely broken design" etc.

(Of course some parts ot the initial patch were hacky, since I simply added TB support with as few changes to SF as possible. The root probing coding admittedly is somewhat "broken" in the sense that it does not play well with multipv and searchmoves. I submitted a patch to clean this up, but there was no willingness to consider it.)

[jdart]

I think the mate problem could be solved by also allowing the use of a DTM tablebase such as Nalimov or Gaviota. If this reports a mate of a distance that is safely within the 50-move counter, then play that mating move.

--Jon

[hgm]

More generally, you could take the move with the smallest DTM that according to the DTZ50 is won.

[mcostalba]

I'm not sure it is helpful to treat TB wins and TB losses differently in the middle of the tree (unless you are talking about win/loss relative to the root position). It should not matter whether a TB position is entered by a capture by the winning side (in which case the TB probe returns a TB loss) or by a (re)capture by the losing side (in which case the TB probe returns a TB win).

I don't think I understand the above, for instance in the following position

[d]5k2/8/1p6/r7/R7/K7/8/8 w - - 0 1

TB report a draw for Rxa5 and in the follow up

[d]5k2/8/1p6/R7/8/K7/8/8 b - - 0 1

It is still a draw for bXa5.

Can you please better explain your point?

[syzygy]

Can you please better explain your point?

I was clearly not talking about TB draws, so I suggest you carefully re-read what I wrote.

[mcostalba]

Can you please better explain your point?

I was clearly not talking about TB draws, so I suggest you carefully re-read what I wrote.

Ok, in this position after the winning side capture Rxa5 black is losing and TB return loss.

[d]4k3/8/8/p7/R7/8/8/4K3 w - - 0 1

Instead in this position

[d] 4k3/8/1p6/R7/R7/8/8/4K3 b - - 0 1

after losing side capture bxa5, TB return a win score for white.

Is this what you mean? Your point is that in one case search returns TB loss, in teh other case it returns a normal score (probably very high) and you think this asymmetry could be an issue? Is it correct?

[syzygy]

I said I see no reason to treat TB wins inside the tree different from TB losses inside the tree.

You treat them differently: TB loss returns immediately, TB win is ignored.

To explain why there is no reason to treat them differently, I point out that if say, white can reach a TB win from some (non-TB) position in the tree, then the point in the subtree at which the winning path enters the TBs could be a TB win (for white) or a TB loss (for black). In fact, some winning paths may enter the TBs in a TB win position and other paths may enter the TBs in a TB loss position. This is sort of random, and it therefore seems very illogical to treat the two cases differently.

Example: a winning KRPPKRP position may have one winning path entering 6-piece TBs in a losing KRPPKR position (white captured into the TBs, so black is to move and loses) and another winning path entering 6-piece TBs in a winning KRPKRP position (black captured into the TBs, so white is to move and wins). You would return the TB loss from the lost KRPPKR position, but you would ignore the TB win in the KRPKRP position. That does not seem to make a lot of sense.

So my proposal:
- a TB loss/win returns immediately with losing/winning TB score if that score can logically be used for an alpha or beta cutoff (so when you don't need to know the exact distance-to-mate to know that you can take the cutoff: it would be silly to search on).
- if alpha, beta are such that you want to know the distance-to-mate (either winning or losing), then continue the search, but make sure that if no mate score is found, then the value returned still reflects that the position is a known (TB) win or loss.

With this approach, SF's TB probing behaviour is unchanged as long as the aspiration window does not approach mate values. So any extra search effort compared to the current implementation is made only when the search has already determined the game to be won or lost, which means it should not cost any Elo. (There is of course no way to gain Elo from resolving TB wins or losses to mates, since DTZ probing at the root is infallible as it is implemented now).

[mcostalba]

But Michel is right. The "clean" solution to the problem has been pointed out many times in this thread. The simple idea of the solution is that you don't take the TB cutoff if the TB win or loss value is inside the (alpha, beta) window.

Yes, to use TB scores in analogy with upper/lower bounds of TT case makes definitely sense, and I have done it:

Code: Select all

    // Step 4a. Tablebase probe
    if (!PvNode && !rootNode && ss->tbCardinality)
    {
        int piecesCount = pos.count<ALL_PIECES>();

        if (    piecesCount <= ss->tbCardinality
            && (piecesCount <  ss->tbCardinality || depth >= TB::ProbeDepth)
            &&  pos.rule50_count() == 0
            && !pos.can_castle(ANY_CASTLING))
        {
            TB::ProbeState err;
            TB::WDLScore wdl = Tablebases::probe_wdl(pos, &err);

            if (err != TB::ProbeState::FAIL)
            {
                thisThread->tbHits.fetch_add(1, std::memory_order_relaxed);

                int drawScore = TB::UseRule50 ? 1 : 0;

                value =  wdl < -drawScore ? -VALUE_MATE + MAX_PLY + ss->ply
                       : wdl >  drawScore ?  VALUE_MATE - MAX_PLY - ss->ply
                                          :  VALUE_DRAW + 2 * wdl * drawScore;

                // Use TB scores win/loss as bounds (like TT upperbound and
                // lowerbound scores), a draw score is always considered.
                if (value >= beta ? wdl >= TB::WDLDraw : wdl <= TB::WDLDraw)
                {
                    // In case of a draw or a loss, save TB score in TT and return
                    // In case of a winning position keep searching the sub-tree
                    // with a much reduced depth and do not probe TB anymore.
                    if (value <= VALUE_DRAW)
                    {
                        tte->save(posKey, value_to_tt(value, ss->ply), BOUND_EXACT,
                                  std::min(DEPTH_MAX - ONE_PLY, depth + 6 * ONE_PLY),
                                  MOVE_NONE, VALUE_NONE, TT.generation());
                        return value;
                    }

                    depth = std::max(depth / 2, ONE_PLY);
                    ss->tbCardinality = 0;
                }
            }
        }
    }

But this alone does not fix the issues I'd like to target with my patch: mates, sacrifices, etc. In particular as long as the search does not find a mate PV, the "naked" TB scores and behavior will be exposed to user.

[mcostalba]

You treat them differently: TB loss returns immediately, TB win is ignored.

<cut>

That does not seem to make a lot of sense.

Yes, I treat them differently, and not only when entering in TB, but always

The reason for this is to avoid odd sacrifices and TB scores to be exposed to user. If I don't overwrite search value in case of a winning position I assume the above two "features" of TB behavior will not be exposed.

On the other hand it is not like I don't' use TB anymore because:

- I still don't search in draw and losing sub-trees so keeping original TB efficiency

- I spend much less time than usual in winning sub-trees (it is a compromise, I don't' return immediately like in pure TB case, so some efficiency is lost, but because search is greatly reduced, I look for this to be harmless).

You may say that in win position, returning a score form a reduced depth search, instead of a TB win could be very bad. I don't' think so. I don't' throw away TB info, but I implicitly use the very critical knowledge that searching those sub-tree will never fail low even in very deep searches. That's why I search swallow: I already know a priori it will not fail low. This is a very important information. Reason why engines search very deep the PV is not to find a better PV, but to prove the current PV does not fail low. We already know it after a TB probing, so we implicitly use this to do a swallow search and save resources.

Code: Select all

So my proposal:
- a TB loss/win returns immediately with losing/winning TB score if that score can logically be used for an alpha or beta cutoff (so when you don't need to know the exact distance-to-mate to know that you can take the cutoff: it would be silly to search on).
- if alpha, beta are such that you want to know the distance-to-mate (either winning or losing), then continue the search, but make sure that if no mate score is found, then the value returned still reflects that the position is a known (TB) win or loss.

Can you please explain how in this approach you avoid TB artifacts: sacrifices, TB scores, etc.?


My aim is to keep the TB ELO advantage but do not show to the user we are using TB. User looking at the engine should not realize if we are using TB or not. I don't claim my aim is what everybody expects / likes, I just say that this patch is wrote for this reason, not for others.