Interesting machine

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
bob
Posts: 20923
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Interesting machine

Post by bob » Sun May 22, 2016 9:27 pm

syzygy wrote:
syzygy wrote:
bob wrote:
syzygy wrote:
bob wrote:Would be one hell of a machine to generate endgame tables..
Indeed...
Just for fun, how long does it normally take to generate through 5 pieces? and then 6 pieces? If it is reasonable, I can give it a whirl just to see how this thing works, if your generator is available?
5 pieces take about an hour on my machine (i7-3930K at 4.2Ghz, 6 cores / 12 threads). 6 pieces around 100x as long.

My generator is available (see signature), but it expects a little endian machine and it uses a bit of x86 inline assembly for atomically doing updates to the tablebase arrays.

The inline assembly can now be replaced with C11 atomics (if you have a relatively recent gcc). That should not take me much time.

The endianness might be a non-issue, but that is not clear to me. What does the machine / the OS expect? But it can in any case be fixed and I should probably do that anyway at some point.
OK, if you are interested, you could try this version:
https://github.com/syzygy1/tb/tree/c11
https://github.com/syzygy1/tb/archive/c11.zip
It requires a relatively recent gcc (with c11 atomics support).

- download the zip file and extract
- specify desired number of max threads in src/Makefile (-DMAX_THREADS=N)
- make all
- test, e.g. with 8 threads: ./rtbgen -t 8 KQvK
- see if probing of KQvK works: ./rtbgen -t 8 KQQvK

This should create four files:

Code: Select all

$ md5sum *.rtbw?
21070fc6c230cce12f6f6b615c9f1eb4  KQQvK.rtbw
fccaac715e6b2d5540db71ebe6284b49  KQQvK.rtbz
f06221548404795b6b33469e247b4560  KQvK.rtbw
ac866466e16eb19a4f8c796f8e1abd2b  KQvK.rtbz
File sizes should be 7056, 25936, 272, 5392.
(If the md5sums do not match but the file sizes do, my attempt to port the internal (non-md5) checksum calculation to big endian was not yet fully successful.)

rtbgen creates pawnless tables, rtbgenp creates pawnful tables.

Create all 5-piece TBs:

Code: Select all

$ ./run.pl --generate --threads 160 --min 3 --max 5
Hmmm, run.pl expects rtbgen and rtbgenp to be in your $PATH.

Don't bother with rtbver and rtbverp (or with ./run.pl --verify).

Of course untested on a power 8 machine, so the above is likely to break down at some point ;-).
I am about 2/3 of the way through my SMP tests for Crafty on that machine. As soon as this finishes, I will give your code a try. I am going to attempt to compile it tonight if I have time, to make sure that the gcc compiler is OK. This is a RHEL system so it should be pretty current.

Note:

I just checked and the RHEL system included gcc 4.8.5... I am thinking that is not "recent enough" as it complains on a make all and mentions no "stdatomic.h"

syzygy
Posts: 4904
Joined: Tue Feb 28, 2012 10:56 pm

Re: Interesting machine

Post by syzygy » Sun May 22, 2016 10:59 pm

bob wrote:I just checked and the RHEL system included gcc 4.8.5... I am thinking that is not "recent enough" as it complains on a make all and mentions no "stdatomic.h"
Unfortunately C11 atomics is only supported since gcc 4.9:
https://gcc.gnu.org/gcc-4.9/changes.html

It should be possible to do the same with gcc-specific __sync_bool_compare_and_swap(). I'll have a look when I find some time.

bob
Posts: 20923
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Interesting machine

Post by bob » Sun May 22, 2016 11:56 pm

syzygy wrote:
bob wrote:I just checked and the RHEL system included gcc 4.8.5... I am thinking that is not "recent enough" as it complains on a make all and mentions no "stdatomic.h"
Unfortunately C11 atomics is only supported since gcc 4.9:
https://gcc.gnu.org/gcc-4.9/changes.html

It should be possible to do the same with gcc-specific __sync_bool_compare_and_swap(). I'll have a look when I find some time.
I can install 4.9 tomorrow...

EvgeniyZh
Posts: 42
Joined: Fri Sep 19, 2014 2:54 pm
Location: Israel
Contact:

Re: Interesting machine

Post by EvgeniyZh » Fri Jun 10, 2016 5:52 am

bob wrote:
lauriet wrote:Hi bob,
I have an old commadore 64 you can also

:D

:D
This is a bit newer. :) 20 cores, 1 terabyte of DRAM. Got anything close? :)
Should be enough to generate 7-man, shouldn't it? Would take tons of the time though

User avatar
Nordlandia
Posts: 2743
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: Interesting machine

Post by Nordlandia » Fri Jun 10, 2016 2:03 pm

EvgeniyZh wrote:
bob wrote:
lauriet wrote:Hi bob,
I have an old commadore 64 you can also

:D

:D
This is a bit newer. :) 20 cores, 1 terabyte of DRAM. Got anything close? :)
Should be enough to generate 7-man, shouldn't it? Would take tons of the time though
If Robert can test few 7-Men positions and inform us about the average time per position :wink:

User avatar
MikeB
Posts: 4614
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Interesting machine

Post by MikeB » Sat Jun 11, 2016 4:43 am

wgarvin wrote:
syzygy wrote:5 pieces take about an hour on my machine (i7-3930K at 4.2Ghz, 6 cores / 12 threads). 6 pieces around 100x as long.
Just an aside.. its awesome that we now live in a world where someone can generate 6-piece tablebases for chess in less than a week on a single machine costing only a few thousand dollars. (At least using a sufficiently-clever generator, which yours is)
How about 48 hours on a 2010 Mac Pro with 12 cores - 24 threads purchased used for about $2100. It was faster for me to build it then download it - only DSL available where I live.

User avatar
Nordlandia
Posts: 2743
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: Interesting machine

Post by Nordlandia » Sat Jun 11, 2016 8:47 am

MikeB wrote:
wgarvin wrote:
syzygy wrote:5 pieces take about an hour on my machine (i7-3930K at 4.2Ghz, 6 cores / 12 threads). 6 pieces around 100x as long.
Just an aside.. its awesome that we now live in a world where someone can generate 6-piece tablebases for chess in less than a week on a single machine costing only a few thousand dollars. (At least using a sufficiently-clever generator, which yours is)
How about 48 hours on a 2010 Mac Pro with 12 cores - 24 threads purchased used for about $2100. It was faster for me to build it then download it - only DSL available where I live.
Michael B: can you make some plausible time estimations about generating 7-Men Syzygy in case using i7-6950X overclocked, or Intel® Xeon® Processor E7-8890 v4
(60M Cache, 2.20 GHz)

http://ark.intel.com/products/94456/Int ... o-3_50-GHz

http://ark.intel.com/products/93790/Int ... e-2_20-GHz

Vinvin
Posts: 4911
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Interesting machine

Post by Vinvin » Thu Aug 11, 2016 11:04 am

bob wrote:IBM just donated a dual socket 10 cores per socket power-8 machine to the department.
...
Is there some conclusions about this machine ?

bob
Posts: 20923
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Interesting machine

Post by bob » Thu Aug 11, 2016 6:22 pm

Vinvin wrote:
bob wrote:IBM just donated a dual socket 10 cores per socket power-8 machine to the department.
...
Is there some conclusions about this machine ?
Really good box, but not as fast as any recent Intel processor...

Dann Corbit
Posts: 11979
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Interesting machine

Post by Dann Corbit » Thu Aug 11, 2016 8:02 pm

It would be very interesting to run an instance of Stockfish (or other engine which uses LazySMP) to be able to put to bed the notion of whether or not it still works well with absurd levels of thread contention.

Some experiments apparently showed it worked well in that arena. It would be interesting to see what happens in an extreme case.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Post Reply