Page 1 of 2

Current State of EGTB support/probing and such

Posted: Fri Sep 13, 2013 5:17 am
by jshriver
In responce to the syzygy thread:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

I suspect many engines will implement this. There was Nalimov code... which is great hell I still dedicated almost 3TB to it (redundancy) plus the Gaviota code which was the first real... 3-4-5 NOT to require some author "requirements for use".

*Insert RANT*

Nalimov is the king of EGTB still.... thanks to Kirill (and a small amount to me, but mostly Kirll) and supporters... we "revived" the lost Nalimov set.
It's still the King mostly because of the vast support of older engines which were given "access" to use the code. The egtb are free.. but access to the code "requires permission from the authors"... which doesn't sound bad, but from what I've heard from YEARS of trolling here.. they are far and few between. So post 2k getting access to use code, even available freely , but to be hearnest and "respect the ownership" was REALLY rare.

I am lucky, because when I first started my own chess engine was mid-90's and both Nalimov and the compression author were easier to reach and get "permission from". So when I do publish it, I do HAVE the emails giving me permission to use the code.

However I am curious how that permutates to my EGTB Checker program.

What is GREAT by the syzygy EGTB is not ONLY the compactness, but the author seems to give a no-holds bar... BSD-like.... just freaking USE it.

I held off for a couple years REALLY wanting Gavioto to hit this mark, but must admit, will be migrating mostly to Syzygy now. Already trying to decipher the probing code.

If I might humbly ask... of Mr. Ronald de Man. Sorry perhaps cultural walls stand, is saying Mr. Man ok? or Mr. de Man? (english)?

Anyway... raw FEN to probing would be AWESOME.

I'd like to think I'm an ok programmer, love C and some C++.... but looking at your interface code... well... it's not "raw interface code"... it's the stockfish "patch" to interfacing... so requires inquiring minds,.... to also look into stockfish constructs... as a 100% programmer, and author.... it adds a but of an issues, since I do not want to require stockfish code.... to access egtb's..... which I suspect aren't "NEEDED"

-Josh

Keep downloading and SEEDING!!!!

Re: Current State of EGTB support/probing and such

Posted: Fri Sep 13, 2013 8:56 am
by velmarin
Do not forget that Chessbase products supported "Nalimov" natively,
so any engine used in the GUI have their support if desired.
On servers that's how important Playchess or Private tournaments are, ect.
Arena also does the same with "Gaviota".
Other GUIs do not remember by heart.

These Syzigia looks like a good project to try to see other engines.

Stockfish has with Comstock "Robbobases" support of a great way.
Also with the "Gaviota" by Jeremy Bernstein, have not tried but it seems it works great.

Apart the f Robbobases,
the most complete, the most compact and more efficient, a waste. :shock:
:?

Re: Current State of EGTB support/probing and such

Posted: Fri Sep 13, 2013 12:25 pm
by Sharaf_DG
jshriver wrote: Already trying to decipher the probing code.
Ronald's Probing Code most certainly needs attention

Re: Current State of EGTB support/probing and such

Posted: Fri Sep 13, 2013 12:28 pm
by Sharaf_DG
jshriver wrote: Already trying to decipher the probing code.
Ronald's Probing Code most certainly needs attention

Re: Current State of EGTB support/probing and such

Posted: Fri Sep 13, 2013 8:52 pm
by syzygy
Sharaf_DG wrote:
jshriver wrote: Already trying to decipher the probing code.
Ronald's Probing Code most certainly needs attention
Maybe you could stop commenting on things you know nothing of...?

Re: Current State of EGTB support/probing and such

Posted: Fri Sep 13, 2013 9:19 pm
by Sharaf_DG
syzygy wrote:
Sharaf_DG wrote:
jshriver wrote: Already trying to decipher the probing code.
Ronald's Probing Code most certainly needs attention
Maybe you could stop commenting on things you know nothing of...?
refer to your post 1060 'I do not exclude that some bugs are left'

Re: Current State of EGTB support/probing and such

Posted: Fri Sep 13, 2013 10:02 pm
by syzygy
:roll: :roll: :roll:

Re: Current State of EGTB support/probing and such

Posted: Sat Sep 14, 2013 5:12 am
by jdart
What is GREAT by the syzygy EGTB is not ONLY the compactness, but the author seems to give a no-holds bar... BSD-like.... just freaking USE it.

I held off for a couple years REALLY wanting Gavioto to hit this mark, but must admit, will be migrating mostly to Syzygy now. Already trying to decipher the probing code.
I understand you might have some technical objections to Gaviota EGTBs but as for the licensing .. it is the MIT License, which is very similar to BSD, and a very permissive license, allowing redistribution either as source or binary. (The Arasan chess engine is now under the same license).

The LZF and zlib compression code used by Gaviota EGTBs (optionally) have some separate but also quite permissive license terms.

--Jon

Re: Current State of EGTB support/probing and such

Posted: Sat Sep 14, 2013 5:38 pm
by Kirill Kryukov
I posted Syzygybases introduction to the EGTB Online site.

(This is the first update in 5 years. If my time permits (and this is a big if) I'll make the site more up-to-date. The site served its purpose back in the time, but now it's so badly outdated that I might as well start from scratch. :oops:)

Honestly I was too slow to realize how restrictive Nalimov's license is. Back in 2005 it did not look like a big problem, but now I see it as absolute deal breaker.

I feel Syzygybases have potential to become the next standard tablebase format (if such a thing can exist today).

Re: Current State of EGTB support/probing and such

Posted: Sat Sep 14, 2013 7:45 pm
by syzygy
If they do get popular, we might need some conventions on how to deal with them using the UCI protocol:
- how to report "distance to tablebase win" scores;
- how to probe them at the root.

My Stockfish adaptation currently reports a tablebase win in N moves as a mate in 50 + N. Ideally, a tablebase win in 12 moves could be reported as "win12" or maybe "tbwin12", but of course this is non-standard and current GUIs would not understand it. I do not know how this is dealt with by other engines that use WDL tables / bitbases.

Regarding the second point, since the DTZ metric is rather different from the DTM metric (on which Nalimov is based), just playing the "best" move at the root (when the root position is in the tablebases) leads to unsatisfactory play. The current Stockfish adaptation therefore does not pick and immediately play any "best" move, but uses the tables at the root only to filter out the "bad" moves that would not preserve a win or draw (while taking into account the 50-move rule). Stockfish then searches the remaining moves. The reported score is the score returned by the search. If the position is a very deep tablebase win, Stockfish will flawlessly win it (unless it is a "cursed" win because of the 50-move rule, in which case Stockfish will use a "swindle" mode and most likely still win it), but the scores reported might be drawish because the search cannot yet see the win.

In such a situation, there should be a way for the engine to inform the GUI that the root position is a tablebase win (or draw or loss). This could be done by returning a special value for the score instead of the score returned by the search, but I would prefer to report both (because the search will usually quickly be able to find the win and report a mate score). I guess as a compromise I could return mate100 or so if the position is a tablebase win but the search does not yet see it, and the real mate score if the search does see it.

There should also be a way for the GUI to directly probe the tables for each root move, so that the GUI can show which moves are winning / drawing / losing. The current implementation does not allow this, because searching a root move while in the tables does not return immediately with the tablebase score, but starts a real search.