Scorpio bitbases question

[beachknight]

You can find a documentation at https://github.com/dshawul/Scorpio.
Note that the documentation describes the 5-men probing method. The next version of Scorpio will support 6 men (48.1gb in size), which does the probing with null terminated arrays instead of plain piece_1, location_1.

Great news Daniel.

Could we able to produce 6-men bitbases on our computers?

I will ask the same question to Miguel regarding Gaviota 6-men.

Best,

Yes but I don't recommend it for 6 men. Joshua will share them via bittorrent once I manage to send it to him.
Here are the new 5-men which you can use for old engines as well, so you can just replace the old ones
http://shawul.olympuschess.com/egbb/egbb5men.7z
And one 6-men KPPkpp for testing
http://shawul.olympuschess.com/egbb/6men/
You would need a new scorpio version to test the 6-men though because currently no other engine supports them. My current scorpio github code can be compiled to work with them.
Daniel

Thanks Daniel.

Here in Antalya we dont have a data center for downloading
some hundreds of gigas. I had two friends of mine with quick
download speed. I have lost both, while I tried to help
Turkey's number 1 corr player who turned out to be an as...le.

I currently dont have a download possibility. Therefore
I prefer producing them on my ancient quad.

Of course I need a new 4cpu compile of scorpio to be able to
test them.

Thanks again.

[Daniel Shawul]

OK, I found it in the file readme.md. Just for confirmation:
*) Square 1 would be B1?

Careful, square 1 could also be A2. From the Readme.md file one can't tell. All it says is "A1 = 0 ... H8 = 63", which is ambiguous.

Ah... I didn't realize that is what was what HG was talking about. Ofcourse what I meant was also B1=1. But that still won't be a full description, who knows what kind of path one may think of that passes through those three squares.

[Daniel Shawul]

The squares are standard 64 squares table where A1=0 and H8=63. If you are planning to use it just for a GUI, I could add a FEN prober to avoid this hassle.

I'd be interested in that

I have added this now, but egbbdll and code are not uploaded yet. You can just import a function probe_egbb_fen to look up with a fen string.

Code: Select all

DLLExport int  CDECL probe_egbb_fen(char* fen);

[hgm]

One more question:

Is there a minimum requirement for the cache size, when I don't care much about speed? Would, say, 64KB do?

[Daniel Shawul]

You can use as lows as 9k, which corresponds to a single cache entry (actually 8k + book-keeping items).

[hgm]

Does that mean that 65KB would be better? (8x8KB + 1KB book keeping)?

[Daniel Shawul]

Yes the data is 8k but there is a 4-bytes block index. So using multiples of 8k and adding N * 4 bytes to it is best.

[hgm]

OK, thanks. I tried to compile the probe library from source on Linux, using the included Makefile, but I got an error on memcpy. Some googling revealed that Volker Pittlik experienced the same problem in 2009, and that the solution was to add an #include "common.h" in cache.cpp. This worked, and provided me with an egbbso.so file. So beware that this still isn't fixed in the zip file I downloaded from your website!

Connecting to it from XBoard gives me some trouble, though. The probe code from Scorpio seems Windows-specific. After I added #defines to get rid of the error mesages for undeclared CDECL and HMODULE, it turns out that the routines LoadLibrary and GetProcAddress are unknown to the linker. Or is this because I need to give a special library flag to use .so files?

If this is really platform dependent, it would be a bad idea to put it in the back-end (where I have it now), and I'd better make separate versions of the load code in the respective front-ends.

Code: Select all

#define CDECL
#define HMODULE void *
#define EGBB_NAME "egbbso.so"

typedef int (CDECL *PPROBE_EGBB) (int player, int *piece, int *square);
typedef int (CDECL *PLOAD_EGBB) (char *path, int cache_size, int load_options);
static int egbbCode[] = { 6, 5, 4, 3, 2, 1 };

static int
BitbaseProbe ()
{
    int pieces[10], squares[10], cnt=0, r, f, res;
    static int loaded;
    static PPROBE_EGBB probeBB;
    if(!appData.testLegality) return 10;
    if(BOARD_HEIGHT != 8 || BOARD_RGHT-BOARD_LEFT != 8) return 12;
    if(loaded == 2 && forwardMostMove < 2) loaded = 0; // retry on new game
    for(r=0; r<BOARD_HEIGHT; r++) for(f=BOARD_LEFT; f<BOARD_RGHT; f++) {
	ChessSquare piece = boards[forwardMostMove][r][f];
	int black = (piece >= BlackPawn);
	int type = piece - black*BlackPawn;
	if(piece == EmptySquare) continue;
	if(type != WhiteKing && type > WhiteQueen) return 12; // unorthodox piece
	if(type == WhiteKing) type = WhiteQueen + 1;
	type = egbbCode[type];
	squares[cnt] = r*(BOARD_RGHT - BOARD_LEFT) + f - BOARD_LEFT;
        pieces[cnt] = piece + black*6;
	if(++cnt > 5) return 11;
    }
    pieces[cnt] = squares[cnt] = 0;
    // probe EGBB
    if(loaded == 2) return 13; // loading failed before
    if(loaded == 0) {
	loaded = 2; // prepare for failure
	char *p, *path = strstr(appData.egtFormats, "scorpio:"), buf[MSG_SIZ];
	HMODULE lib;
	PLOAD_EGBB loadBB;
	if(!path) return 13; // no egbb installed
	strncpy(buf, path + 8, MSG_SIZ);
	if(p = strchr(buf, ',')) *p = NULLCHAR; else p = buf + strlen(buf);
	snprintf(p, MSG_SIZ - strlen(buf), "%c%s", SLASH, EGBB_NAME);
	lib = LoadLibrary(buf);
	if(!lib) return 13;
	loadBB = (PLOAD_EGBB) GetProcAddress(lib, "load_egbb_xmen");
	probeBB = (PPROBE_EGBB) GetProcAddress(lib, "probe_egbb_xmen");
	*p = NULLCHAR; loadBB(buf, 64*1028, 2); // 2 = SMART_LOAD
	loaded = 1; // success!
    }
    res = probeBB(forwardMostMove & 1, pieces, squares);
    return res > 0 ? 1 : res < 0 ? -1 : 0;
}

[Daniel Shawul]

OK, thanks. I tried to compile the probe library from source on Linux, using the included Makefile, but I got an error on memcpy. Some googling revealed that Volker Pittlik experienced the same problem in 2009, and that the solution was to add an #include "common.h" in cache.cpp. This worked, and provided me with an egbbso.so file. So beware that this still isn't fixed in the zip file I downloaded from your website!

The egbb files at my website are old ones even the ones titled new. My github repository has the most recent that should be used for 6 men. Note that you don't have to reproduce egbbso.so files yourself as it will come along with the egbbs. Here are the 5 men egbbs that have the dlls you need inside. It is a 200mb file. I still do not have a 32 bit binary for linux, so if you can compile that I will add it to the package.

Connecting to it from XBoard gives me some trouble, though. The probe code from Scorpio seems Windows-specific. After I added #defines to get rid of the error messages for undeclared CDECL and HMODULE, it turns out that the routines LoadLibrary and GetProcAddress are unknown to the linker. Or is this because I need to give a special library flag to use .so files?

I usually think in Windows API and redefine them for linux, so CDECL is empty, and the rest are redefined as follows as shown in probe.cpp

Code: Select all

#ifndef _MSC_VER
#    define HMODULE void*
#    define LoadLibrary(x) dlopen(x,RTLD_LAZY)
#    define GetProcAddress dlsym
#endif

The linker needs an '-ldl' flag for linux.

[hgm]

Weird. Apparently FireFox is not able to download .7z files; it tries to display them in stead (giving gibberish, of course). In fact it crashed my network to the point where I could only recover by disconnecting the power to my router, as even the WiFi had disappeared from the air.

Isn't there another source? (Preferably with just the DLLs. I already have all the data files installed.)