Interesting machine

[bob]

Would be one hell of a machine to generate endgame tables..

Indeed...

Just for fun, how long does it normally take to generate through 5 pieces? and then 6 pieces? If it is reasonable, I can give it a whirl just to see how this thing works, if your generator is available?

5 pieces take about an hour on my machine (i7-3930K at 4.2Ghz, 6 cores / 12 threads). 6 pieces around 100x as long.

My generator is available (see signature), but it expects a little endian machine and it uses a bit of x86 inline assembly for atomically doing updates to the tablebase arrays.

The inline assembly can now be replaced with C11 atomics (if you have a relatively recent gcc). That should not take me much time.

The endianness might be a non-issue, but that is not clear to me. What does the machine / the OS expect? But it can in any case be fixed and I should probably do that anyway at some point.

OK, if you are interested, you could try this version:
https://github.com/syzygy1/tb/tree/c11
https://github.com/syzygy1/tb/archive/c11.zip
It requires a relatively recent gcc (with c11 atomics support).

- download the zip file and extract
- specify desired number of max threads in src/Makefile (-DMAX_THREADS=N)
- make all
- test, e.g. with 8 threads: ./rtbgen -t 8 KQvK
- see if probing of KQvK works: ./rtbgen -t 8 KQQvK

This should create four files:

Code: Select all

$ md5sum *.rtbw?
21070fc6c230cce12f6f6b615c9f1eb4  KQQvK.rtbw
fccaac715e6b2d5540db71ebe6284b49  KQQvK.rtbz
f06221548404795b6b33469e247b4560  KQvK.rtbw
ac866466e16eb19a4f8c796f8e1abd2b  KQvK.rtbz

File sizes should be 7056, 25936, 272, 5392.
(If the md5sums do not match but the file sizes do, my attempt to port the internal (non-md5) checksum calculation to big endian was not yet fully successful.)

rtbgen creates pawnless tables, rtbgenp creates pawnful tables.

Create all 5-piece TBs:

Code: Select all

$ ./run.pl --generate --threads 160 --min 3 --max 5

Hmmm, run.pl expects rtbgen and rtbgenp to be in your $PATH.

Don't bother with rtbver and rtbverp (or with ./run.pl --verify).

Of course untested on a power 8 machine, so the above is likely to break down at some point .

I am about 2/3 of the way through my SMP tests for Crafty on that machine. As soon as this finishes, I will give your code a try. I am going to attempt to compile it tonight if I have time, to make sure that the gcc compiler is OK. This is a RHEL system so it should be pretty current.

Note:

I just checked and the RHEL system included gcc 4.8.5... I am thinking that is not "recent enough" as it complains on a make all and mentions no "stdatomic.h"

[syzygy]

I just checked and the RHEL system included gcc 4.8.5... I am thinking that is not "recent enough" as it complains on a make all and mentions no "stdatomic.h"

Unfortunately C11 atomics is only supported since gcc 4.9:
https://gcc.gnu.org/gcc-4.9/changes.html

It should be possible to do the same with gcc-specific __sync_bool_compare_and_swap(). I'll have a look when I find some time.

[bob]

I just checked and the RHEL system included gcc 4.8.5... I am thinking that is not "recent enough" as it complains on a make all and mentions no "stdatomic.h"

Unfortunately C11 atomics is only supported since gcc 4.9:
https://gcc.gnu.org/gcc-4.9/changes.html

It should be possible to do the same with gcc-specific __sync_bool_compare_and_swap(). I'll have a look when I find some time.

I can install 4.9 tomorrow...

[EvgeniyZh]

Hi bob,
I have an old commadore 64 you can also

This is a bit newer. 20 cores, 1 terabyte of DRAM. Got anything close?

Should be enough to generate 7-man, shouldn't it? Would take tons of the time though

[Nordlandia]

Hi bob,
I have an old commadore 64 you can also

This is a bit newer. 20 cores, 1 terabyte of DRAM. Got anything close?

Should be enough to generate 7-man, shouldn't it? Would take tons of the time though

If Robert can test few 7-Men positions and inform us about the average time per position

[MikeB]

5 pieces take about an hour on my machine (i7-3930K at 4.2Ghz, 6 cores / 12 threads). 6 pieces around 100x as long.

Just an aside.. its awesome that we now live in a world where someone can generate 6-piece tablebases for chess in less than a week on a single machine costing only a few thousand dollars. (At least using a sufficiently-clever generator, which yours is)

How about 48 hours on a 2010 Mac Pro with 12 cores - 24 threads purchased used for about $2100. It was faster for me to build it then download it - only DSL available where I live.

[Nordlandia]

5 pieces take about an hour on my machine (i7-3930K at 4.2Ghz, 6 cores / 12 threads). 6 pieces around 100x as long.

Just an aside.. its awesome that we now live in a world where someone can generate 6-piece tablebases for chess in less than a week on a single machine costing only a few thousand dollars. (At least using a sufficiently-clever generator, which yours is)

How about 48 hours on a 2010 Mac Pro with 12 cores - 24 threads purchased used for about $2100. It was faster for me to build it then download it - only DSL available where I live.

Michael B: can you make some plausible time estimations about generating 7-Men Syzygy in case using i7-6950X overclocked, or Intel® Xeon® Processor E7-8890 v4
(60M Cache, 2.20 GHz)

http://ark.intel.com/products/94456/Int ... o-3_50-GHz

http://ark.intel.com/products/93790/Int ... e-2_20-GHz

[Vinvin]

IBM just donated a dual socket 10 cores per socket power-8 machine to the department.
...

Is there some conclusions about this machine ?

[bob]

IBM just donated a dual socket 10 cores per socket power-8 machine to the department.
...

Is there some conclusions about this machine ?

Really good box, but not as fast as any recent Intel processor...

[Dann Corbit]

It would be very interesting to run an instance of Stockfish (or other engine which uses LazySMP) to be able to put to bed the notion of whether or not it still works well with absurd levels of thread contention.

Some experiments apparently showed it worked well in that arena. It would be interesting to see what happens in an extreme case.