Compiling Crafty

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

User avatar
Jim Ablett
Posts: 1383
Joined: Fri Jul 14, 2006 7:56 am
Location: London, England
Full name: Jim Ablett

Re: Compiling Crafty

Post by Jim Ablett »

Hi Nolan,

Here's the .bat file I use for compiling windows Crafty 2014 with intel compiler.




******* crafty.bat ********

Code: Select all

cls
rem 
rem  The next options are optimizations inside Crafty that you will have
rem   test to see if they help.  on some machines, these will slow things
rem   by up to 10%, while on other machines these options will result in
rem   improving search speed up to 20%.  NOTE:  if you are running Linux
rem   system, the default configurations below will use the hand-written
rem   assembly modules.  Typical performance improvement is 5%, but this
rem   only applies to X86 machines running Linux.
rem   
rem                default
rem   -DBOOKDIR      N  path to the directory containing the book binary files.
rem                     The default for all such path values is "." if you don't
rem                     specify a path with this macro definition.
rem   -DCPUS=n       N  defines the maximum number of CPUS Crafty will be able
rem                     to use in a SMP system.  Note that this is the max you
rem                     will be able to use.  You need to use the smpmt=n command
rem                     to make crafty use more than the default 1 process.
rem   -DDGT          N  This is a unix-only option to support the DGT board.
rem   -DEPD          Y  if you want full EPD support built in.
rem   -DHASHSTATS    N  This option compiles in some of the statistics 
rem                     gathering to provide hashing statistics.
rem   -DINLINE32     N  Compiles with the Intel assembly code for FirstOne(),
rem                     LastOne() and PopCnt().  This is for gcc-style inlining
rem                     and now works with the Intel C/C++ compiler as well.
rem   -DINLINE64     N  Compiles with the Intel assembly code for FirstOne(),
rem                     LastOne() and PopCnt() for the AMD opteron, only tested
rem                     with the 64-bit opteron GCC compiler / Intel ICC compiler.
rem   -DLIMITEXT     N  limit extensions as search gets deeper
rem   -DLOGDIR       N  path to the directory where Crafty puts the log.nnn and
rem                     game.nnn files.
rem   -DNOEGTB       N  eliminates the egtb code for compilers that can't deal
rem                     with the large egtb.cpp code/templates.
rem   -DNOFUTILITY   N  disables classic futility pruning
rem   -DNUMA         N  says this system is NUMA, which is mainly used for Linux
rem                     systems, and references libnuma, needed for the NUMA calls
rem                     (crafty doesn't use many of these, it does the memory
rem                     setup stuff itself)
rem   -DPOWERPC      N  enables PPC spinlock inline function for SMP boxes only
rem   -DRCDIR        N  path to the directory where we look for the .craftyrc or
rem                     crafty.rc (windows) file.
rem   -DSMP          N  enables SMP support (not clustering).  If you choose
rem                     this option, you need to look closely at the following
rem                     two options as well.
rem   -DTBDIR        N  path to the directory where the endgame tablebase files
rem                     are found.  default = "./TB"
rem   -DTRACE        N  This enables the "trace" command so that the search tree
rem                     can be dumped while running.


rem  add  '#define inline __inline' to 'chess.h'

rem  add 'typedef unsigned long DWORD_PTR;' to 'utility.c'

rem comment out  'dgt.c' 'epd.c' 'epdglue.c' from 'crafty.c' (for speedup)


rem  smp version below
rem icl -o crafty.exe crafty.c egtb.cpp /DNT_i386 /DWIN32 /DCPUS=2 /DSMP /G7 /Ox /Qipo /QxK /MT /Qparallel /Qprof-gen


rem  single cpu version below
rem icl -o crafty.exe crafty.c egtb.cpp /DNT_i386 /DWIN32 /G7 /Ox /Qipo /QxK /MT /Qparallel /Qprof-gen


cls
del *.o
del *.obj
cls


icl -o crafty.exe main.c attacks.c book.c boolean.c data.c draw.c drawn.c edit.c enprise.c epd.c epdglue.c evaluate.c give.c history.c init.c input.c interupt.c iterate.c lookup.c make.c movegen.c next.c nextc.c nexte.c resign.c root.c search.c setboard.c store.c swap.c test.c time.c utility.c valid.c validate.c /DNT_i386 /DWIN32 /DCPUS=2 /DSMP /G7 /Ox /Qipo /QxK /MT /Qparallel /Qopenmp /Qprof-gen

Read 'rem'med out comments for info on which files to edit before compiling.

Add the 'crafty.bat' + all crafty src files to 'Program Files\Intel\Compiler\C++\10.0.025\IA32\Bin' and run it.

Jim.
NJDenson
Posts: 75
Joined: Tue May 23, 2006 7:01 pm

Re: Compiling Crafty

Post by NJDenson »

Thanks Jim this might do it .. I will keep you all posted.
NJDenson
Posts: 75
Joined: Tue May 23, 2006 7:01 pm

Re: Compiling Crafty

Post by NJDenson »

I got crafty to compile but, its only using 4 cpu's ... I have changed options in the crafty.rc file as well as the makefile.xp

Is there something I am missing. Any assistance will be appreciated, to get it to compile for over 8 cpu's.

Is it in one of the c files?

Thank You


Initializing multiple threads.
System is SMP, not NUMA.
unable to open book file [./book.bin].
book is disabled
unable to open book file [./books.bin].
ERROR - Crafty was compiled with CPUS=4. mt can not exceed this value.
max threads set to 4
SMP search split at ply >= 1
maximum thread group size set to 8
adaptive estimated NPS = 1200K
adaptive minimum hsize = 12M
adaptive maximum hsize = 192M
adaptive minimum psize = 3M
adaptive maximum psize = 48M
pondering enabled.
book learning enabled
result learning enabled
usage: noise <n>
EGTB access enabled
using tbpath=F:\Chess\Tablebases
6 piece tablebase files found
12495kb of RAM used for TB indices and decompression tables
resign after 5 consecutive moves with score < -9.
pondering enabled.


Crafty v2x.x (4 cpus)

White(1):
User avatar
Jim Ablett
Posts: 1383
Joined: Fri Jul 14, 2006 7:56 am
Location: London, England
Full name: Jim Ablett

Re: Compiling Crafty

Post by Jim Ablett »

I got crafty to compile but, its only using 4 cpu's ... I have changed options in the crafty.rc file as well as the makefile.xp
Can you show modified makefile?
Jim.
NJDenson
Posts: 75
Joined: Tue May 23, 2006 7:01 pm

Re: Compiling Crafty

Post by NJDenson »

I was able to fix the problem. I had to del the obj file and rebuild.

Now I need to figure out how to make it 64 bit and make the profile optimization work. I gave up trying to get it to work in the studio settings .

# Makefile version 3 for Crafty 16.15
# Crafty v16.x makefile for Windows NT Intel
# Written by Jason Deines (jdeines@mindspring.com) April 1998
# Version modified by Gregor Overney (gregor_overney@hp.com) Nov 1998
# Version modified by Peter Karrer (pkarrer@active.ch) Dec 1998
# Version modified by Gregor Overney (gregor_overney@hp.com) Sep 1999
#
# This makefile is designed to be used from the command line with
# Microsoft's nmake. Either rename this # file to "Makefile" or name it
# explicitly when invoking nmake:
# nmake -f Makefile.xp
#
# The makefile is set up for Microsoft Visual C++ 6.0 Intel.
#
# The default target produces a file called "crafty.exe".

# Build target is defined here.
TARGET = NT_i386

# Command-line compiler and linker invocation commands:
CC = cl
LD = link

# Base compiler flags needed for build:
BFLAGS = /D_CONSOLE /DWIN32 /D_CRT_SECURE_NO_DEPRECATE

# Compiler flags:
# /O2 optimize for speed
# /Oa assume no aliasing (no good for VC 6 without SP3)
# /Gr fastcall calling convention
# /G5 target Pentium (but will run on all x86 architectures)
# /G6 target Pentium Pro (but will run on all x86 architectures)
# /Ob2 inline function calls if suitable
#
# For debugging use these flags instead:
# CFLAGS = /Od /Zi
# LDFLAGS = /DEBUG /DEBUGTYPE:CV
#

#CFLAGS = /O2 /G6 /Gr /Ob2
#CFLAGS = /Od /Zi
CFLAGS = /Ox /Gr /GL

# Linker flags, normally not needed except for debug builds:
LDFLAGS = /LTCG
#LDFLAGS = /DEBUG /DEBUGTYPE:CV

# See the default crafty makefile for a description of the options below.
# With VC++, defines like COMPACT_ATTACKS, etc, makes the code slower, so
# those # options are disabled by default. FAST is normally not defined
# so that hash statistics are reported -- for the fastest possible
# executable, define FAST below. for 6 piece EGTB support, add /DEGTB6.

#COPTS =

# For an SMP build use/add the following build options.
# NT_INTEREX is defined if you want to use the built-in InterlockedExchange()
# function for thread resource locking, instead of the inline asm routine.
# (This shouldn't be needed, at least on Intel.)
# /MT is a compiler flag needed for multithreaded builds.

#COPTS = /MT /DSMP /DCPUS=8 /DNT_INTEREX
COPTS = /MT /DSMP /DCPUS=8

# To enable assembly optimizations in vcinline.h, use /DVC_INLINE_ASM.

#AOPTS = /DVC_INLINE_ASM

ALLOPTS = $(COPTS) $(AOPTS) /D$(TARGET)

objects = crafty.obj egtb.obj

includes = chess.h data.h epd.h epddefs.h epdglue.h vcinline.h

crafty : $(objects)
$(LD) $(LDFLAGS) $(objects) /out:Crafty-21.6-win64.exe

$(objects) : $(includes)

.c.obj :
$(CC) $(BFLAGS) $(CFLAGS) $(ALLOPTS) /c $*.c

.cpp.obj :
$(CC) $(BFLAGS) $(CFLAGS) $(ALLOPTS) /Zm1000 /c $*.cpp

clean:
del /q $(objects)
del /q log.*
del /q game.*
del /q *.bak
NJDenson
Posts: 75
Joined: Tue May 23, 2006 7:01 pm

Re: Compiling Crafty (8 cpu slow down)

Post by NJDenson »

Now that i have a working crafty does anyone have an idea as to why it slows down when i use 8 cpu's. When playing on playchess with 4 cpu's i get around 7000 - 9000 knps when i compile with 8 cpu's i get 7 k.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Compiling Crafty (8 cpu slow down)

Post by bob »

NJDenson wrote:Now that i have a working crafty does anyone have an idea as to why it slows down when i use 8 cpu's. When playing on playchess with 4 cpu's i get around 7000 - 9000 knps when i compile with 8 cpu's i get 7 k.
Are all 8 cpus actually being used? I've been running on 8-way boxes for years and have not ever seen a case where 8 cpus did not almost double the NPS. that suggests that something is wrong, as in Crafty is not using all 8. The log.nnn file will tell a lot if you can send me one.

BTW, these are the "optimal options" if you are using a core/core2 type CPU (including opterons which are also 64 bits):

-DINLINE64 -DCPUS=8

If your machine is a 2 x quadcore, that is probably best. If it is an AMD box, which is NUMA, then -DNUMA and -DLIBNUMA will help further...

Bob
NJDenson
Posts: 75
Joined: Tue May 23, 2006 7:01 pm

Re: Compiling Crafty (8 cpu slow down)

Post by NJDenson »

Please note the two benches I ran for 4 and 8 cpu's. All is fine during this test. The slow down comes when i run it via ChessBase.

Crafty v21.6 (4 cpus)

White(1): bench
Running benchmark. . .
......
Total nodes: 98384585
Raw nodes per second: 6024775.566442
Total elapsed time: 16.330000
White(1):





Crafty v21.6 (8 cpus)

White(1): bench
Running benchmark. . .
......
Total nodes: 145724185
Raw nodes per second: 10154995.470383
Total elapsed time: 14.350000
White(1):
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Compiling Crafty (8 cpu slow down)

Post by bob »

NJDenson wrote:Please note the two benches I ran for 4 and 8 cpu's. All is fine during this test. The slow down comes when i run it via ChessBase.

Crafty v21.6 (4 cpus)

White(1): bench
Running benchmark. . .
......
Total nodes: 98384585
Raw nodes per second: 6024775.566442
Total elapsed time: 16.330000
White(1):

That I can't answer. If you turn logging on, and run it via chessbase and post the log.nnn file or email it to me, we might discover what is up. Could be chessbase is setting max cpus to 4 overriding your crafty.rc file....




Crafty v21.6 (8 cpus)

White(1): bench
Running benchmark. . .
......
Total nodes: 145724185
Raw nodes per second: 10154995.470383
Total elapsed time: 14.350000
White(1):
User avatar
beachknight
Posts: 3533
Joined: Tue Jan 09, 2007 8:33 pm
Location: Antalya, Turkey

Re: Compiling Crafty (8 cpu slow down)

Post by beachknight »

hi, merhaba, hallo HT