syzygy questions

[bob]

Where I am is here:

I have a position that is a cursed win (KNNKP, nalimov says mate in 97). I probe the position and I get the expected "cursed win" return. But then when I make one of the legal moves and probe, I get "tb win" and when I probe another move, I get not found. Most of the moves seem to produce the correct result.

I am looking at this.

Something is not yet working

Make sure you don't call probe_wdl_table() and probe_dtz_table() directly. The tables store incomplete data which is completed by the probe_wdl() and probe_dtz() functions.

But a question. The bug you fixed... was it in tbcore or elsewhere? IE do I need to throw out the tbprobe.c and start again or is the fix easy to insert? I will go back and look up the stockfish discussion here to see if there was a fix given...

It was in tbprobe.cpp. There were actually two bug fixes (the second "sign" bug was fixed in SF quite a while ago already):
https://github.com/syzygy1/tb/commit/68 ... 62d9762142
Instead of calling probe_ab() or probe_wdl() at linest 488-490 depending on the existence of ep rights, you could also simply call probe_wdl().

There is another small bug that Marco pointed out and was not yet fixed in my repository (just applied it):
https://github.com/syzygy1/tb/commit/c6 ... e948192dc8

List of all commits:
https://github.com/syzygy1/tb/commits/master

"something is not yet working..."

"no shit sherlock".

I decided to directly use probe_dtz() and probe_wdl() (after getting rid of the static identifiers of course). Now I am getting sensible stuff. Probably an error elsewhere and this is a cleaner deal. I really don't need "probe_root()" since I do my own stuff there anyway. So i am now working on swindle mode and have correct dtz/wdl results to use. In the process I learned that apparently a bool in C++ is a single byte, which never occurred to me (but then I avoid C++ in general, anyway).

I'll look at the fixes when I get the rest of this stuff done. Code in search looks to be perfect, but swindle mode is a bit tricky since one has to watch out for probing before and after a move and remember that things "change" IE side to move changes the result where with Nalimov it was always just a mate score or a zero.

But I am getting pretty close.

[Rein Halbersma]

In the process I learned that apparently a bool in C++ is a single byte, which never occurred to me (but then I avoid C++ in general, anyway).

Actually, sizeof(bool) is not guaranteed to be 1 byte in C++. It's implementation defined. See e.g. http://stackoverflow.com/questions/4897 ... ol-defined

[bob]

In the process I learned that apparently a bool in C++ is a single byte, which never occurred to me (but then I avoid C++ in general, anyway).

Actually, sizeof(bool) is not guaranteed to be 1 byte in C++. It's implementation defined. See e.g. http://stackoverflow.com/questions/4897 ... ol-defined

I know. I had just expected a normal "int".

types are a bit hosed in C and C++. I have resorted to using stdint.h to eliminate many issues (except for bool).

[bob]

OK, so far I am now apparently "fully functional" so far as search probing and the new swindle mode code.

I did not yet make the changes mentioned as the code I have does not exactly match the post you linked to (particularly where the call to ab and idl replaced the probe_ab call.

Rather than trying to use that, is there any reason I can not just do this myself? IE when there is an en passant capture at the root and I am trying to choose the best move, I can probe for each and every move to see the outcome, pretending the EP capture is not possible, and then make one final call after making the EP capture directly to see whether it is better to capture or not capture?

So far I am using probe_dtz and probe_wdl directly with good results. But at the root, I am currently using probe_root() which is where I assume this problem is showing up???

To clarify, I use probe_root() once I have eliminated "swindle mode" as a choice and simply want to play a TB move and move along in the game.

[syzygy]

OK, so far I am now apparently "fully functional" so far as search probing and the new swindle mode code.

I did not yet make the changes mentioned as the code I have does not exactly match the post you linked to (particularly where the call to ab and idl replaced the probe_ab call.

Rather than trying to use that, is there any reason I can not just do this myself?

If you're using probe_dtz(), then you're using "that" code. It's in probe_dtz_no_ep() which is called from probe_dtz().

IE when there is an en passant capture at the root and I am trying to choose the best move, I can probe for each and every move to see the outcome, pretending the EP capture is not possible, and then make one final call after making the EP capture directly to see whether it is better to capture or not capture?

Two problems:

1. What you describe fails if the position without ep rights is stalemate and the ep capture is losing. In that case the ep capture looks worse (so you'd take the draw instead) but the ep capture is in fact obligatory. The probing code already takes care of these subtleties.

2. The DTZ tables store random values for positions with a winning capture or pawn move (well-chosen random values that improve compression). The DTZ probing code must therefore first check, also if it can assume EP not to be available, for winning captures or pawn moves (which will get DTZ=1). This is checked using the WDL tables. The bug that Marco found was in the code that checked pawn moves: even if you may assume that EP is not available in the current position, after a double pawn push EP may be available and may make the difference between win or non-win. That is why probe_wdl() (which takes into account EP) should be called to check pawn moves rather than probe_ab() (which assumes no EP but may be a bit more efficient). In short: even more subtleties that the probing code already takes care of.

And then there's the thing that the DTZ tables are one-sided: if you are probing the "wrong" side, then probe_table() will tell you that and you will have to do a 1-ply round of probing the other side, carefully adjusting the results to obtain the correct final DTZ value. This is also already done by the probing code.

Of course you will anyway have to port the probing code from SF's structures to Crafty's structures. You can either try to do this without changing the logic of the code or you can try to reimplement these routines (a bit what Marco did). The second option has the danger of introducing bugs.

So far I am using probe_dtz and probe_wdl directly with good results. But at the root, I am currently using probe_root() which is where I assume this problem is showing up???

The problem was in probe_dtz() and that is used in probe_root(). SF only calls probe_dtz() through probe_root(), so in SF the bug could only result in probe_root() not precisely filtering out the correct moves, which usually will not actually hurt. It could only really go wrong if the 50-move rule plays a role (which probably is quite rare in positions with double pawn pushes available). This is most likely why the bug was never reported before (that I'm aware of).

To clarify, I use probe_root() once I have eliminated "swindle mode" as a choice and simply want to play a TB move and move along in the game.

The main idea of probe_root() is that the engine tosses out the moves that might lose half a point or more (including due to the 50-move rule). After this filtering step each of the remaining moves is guaranteed to preserve the outcome of the game and the engine may try to find the best "swindle" move among them.

So if the root position is lost, all moves will lose and no moves are filtered out. If the root position is drawn, then all losing moves will be filtered out. If the root position is winning, then all drawing and losing moves and all winning moves with too high DTZ are filtered out.

root_probe() does a bit more. If the root position is lost but the game is getting close to the 50-move rule (or it the root position has already become a blessed loss given the value of the 50-move counter), it filters out all moves that do not maximise DTZ.

In short:
- I would suggest to stay close to how probe_wdl() and probe_dtz() are implemented;
- you might want to integrate some of the probe_root() ideas in your swindle mode.