Compiling crafty 21.6

[Cardoso]

Hi,
I just gave up compiling crafty with Visual Studio 2008.
Could someone please send me a crafty 21.6 mod source code that compiles in Visual Studio 2008?

Thanks in advance,
Alvaro

my email is:
alvarojpcardoso@gmail.com

[thciobanu]

Create an empty project, add crafty.c and egtb.cpp to it.
In the Project property pages, you should:
- select "use multi-byte character set" instead of unicode
- define NT_i386 if it's not already defined for the preprocessor.

Then, in dgt.c move the line
#if defined(DGT)
to the top (it's only supported on Linux afaik, so the #includes are Linux specific)

Then in lock.h I did the following:
- I moved
#if defined(_WIN32) || defined(_WIN64)
# define pthread_attr_t HANDLE
# define pthread_t HANDLE
# define thread_t HANDLE
#endif
to the top of the file (not nested in the #if CPUS > 1)
- added
# define lock_t volatile int
before the #endif on the last line.

And that about gives you a compiling version of crafty 21.6 on VC2008 (or at least it worked for me). You can then play around with compiler&linker options to get a faster crafty.

[Denis P. Mendoza]

Hi,
I just gave up compiling crafty with Visual Studio 2008.
Could someone please send me a crafty 21.6 mod source code that compiles in Visual Studio 2008?

Thanks in advance,
Alvaro

my email is:
alvarojpcardoso@gmail.com

Alvaro,

There were so many threads that already answered this in the forums. Theodor gave you a good and detailed tips. But since I for one learned from the experts here, this was my experience compiling Crafty as a rule of thumb!

Basic approach c/o Jim Ablett: Works well on 20.x versions and above

1. Extract all files on a specific folder
2. add '#define inline __inline' to "chess.h"
3. add 'typedef unsigned long DWORD_PTR;" to 'utility.c"
4. comment out 'dgt.c' from "crafty.c"

But for Crafty 21.6, only step #4 will fix the problem

Here are my sample command lines for MSVC8 32-bit compile: similar to Dann Corbit Style

/Ox /Ob2 /Oi /Ot /Oy /GT /GL /D "WIN32" /D "NDEBUG" /D "INLINE32" /D "_CONSOLE" /D "EGTB6" /D "CPUS=8" /D "SMP" /D "_VC80_UPGRADE=0x0600" /D "_MBCS" /GF /FD /EHsc /MT /Zp16 /GS- /Gy /fp:fast /Fp".\Release/crafty.pch" /Fo".\Release/" /Fd".\Release/" /W3 /nologo /c /TP /wd4996 /errorReport:prompt

Do a profile guided optimization which includes the following:
Perform a 5 seconds run through WAC.
Perform a 10 minute analysis of WAC.230
Run 4 games at 5'+5"
Gather the .pgc files and rebuild.

For 64-bit: Just use 'WIN64' and 'INLINE64' on the switches above.
Note that there not much difference on command switches for MSVC8 and MSVC9

Or try Peter Skinner's style which he mentioned on some thread

Here are the build command lines that I use:

64bit (Intel):

cl /nologo /w /favor:EM64T /Ox /Gr /GL /D_CONSOLE /DWIN64 /Zm500 /DNT_i386 /DHASHSTATS /DFAST /DEPD /DEGTB6 /MT /DSMP /DCPUS=8 /DNT_INTEREX /DVC_INLINE_ASM crafty.c egtb.cpp

64.bit (AMD):

cl /nologo /w /favor:AMD64 /Ox /Gr /GL /D_CONSOLE /DWIN64 /Zm500 /DNT_i386 /DHASHSTATS /DFAST /DEPD /DEGTB6 /MT /DSMP /DCPUS=8 /DNT_INTEREX /DVC_INLINE_ASM crafty.c egtb.cpp

64bit (blended):

cl /nologo /w /favor:blend /Ox /Gr /GL /D_CONSOLE /DWIN64 /Zm500 /DNT_i386 /DHASHSTATS /DFAST /DEPD /DEGTB6 /MT /DSMP /DCPUS=8 /DNT_INTEREX /DVC_INLINE_ASM crafty.c egtb.cpp

32bit build:

cl /nologo /w /Ox /arch:SSE2 /Gr /GL /D_CONSOLE /DWIN32 /Zm500 /DNT_i386 /DFAST /DEPD /DEGTB6 /DHASHSTATS /MT /DSMP /DCPUS=8 /DNT_INTEREX /DVC_INLINE_ASM crafty.c egtb.cpp

Older CPU Build:

cl /nologo /w /Ox /Gr /GL /D_CONSOLE /DWIN32 /Zm500 /DNT_i386 /DFAST /DEGTB6 /DHASHSTATS /MT /DSMP /DCPUS=8 /DNT_INTEREX /DVC_INLINE_ASM crafty.c egtb.cpp

There is now a link to my PGO makefiles on the website as well.

Just invoke them with:
nmake -f pgi.xp
nmake -f pgo.xp

Visit Peter Skinner's Crafty site for the makefiles:
http://www.webkikr.net/

If you only want to make 32-bit compiles, use this fixed version by Jim Ablett.

Crafty 21.6 win32 Intel Compiler 10 p.g.o (single-cpu)

http://mihd.net/frxlg3

In case anyone prefers some speed-up using the Intel C++ Compiler, Jim again supplied some essential switches which typically produce well-optimized builds - using ICC (32 or 64-bit) command window.

1. Extract src, say at C:\Documents and Settings\Administrator\Desktop\crafty
2. Execute ICC compiler command window
3. Type: cd then 'space' and cut-and-paste location of src folder directory as stated above
4. Press 'enter' and type the sample switches stated below

for SMP

icl -o crafty.exe crafty.c egtb.cpp /DNT_i386 /DWIN32 /DCPUS=8 /DINLINE32 /DSMP /G7 /Ox /Qipo /QxK /MT /Qparallel

icl -o crafty.exe crafty.c egtb.cpp /DNT_i386 /DWIN64 /DCPUS=8 /DINLINE64 /DSMP /G7 /Ox /Qipo /QxK /MT /Qparallel


For single cpu

icl -o crafty.exe crafty.c egtb.cpp /DNT_i386 /DWIN32 /G7 /Ox /Qipo /QxK /MT /Qparallel
icl -o crafty.exe crafty.c egtb.cpp /DNT_i386 /DWIN64 /G7 /Ox /Qipo /QxW /MT /Qparallel

To compile PGO's add '/Qprof_gen' to command line, compile it, run it from the 'bin' directory - run the Crafty 'bench' command and also play a few games. This will generate the necessary 'profile'.

Now compile again but replace '/Qprof_gen' with '/Qprof_use'.
This will produce a 'profile guided optimized' binary.

You can also use Jim's sample crafty bat file:

Code: Select all

cls 
rem 
rem  The next options are optimizations inside Crafty that you will have 
rem   test to see if they help.  on some machines, these will slow things 
rem   by up to 10%, while on other machines these options will result in 
rem   improving search speed up to 20%.  NOTE:  if you are running Linux 
rem   system, the default configurations below will use the hand-written 
rem   assembly modules.  Typical performance improvement is 5%, but this 
rem   only applies to X86 machines running Linux. 
rem    
rem                default 
rem   -DBOOKDIR      N  path to the directory containing the book binary files. 
rem                     The default for all such path values is "." if you don't 
rem                     specify a path with this macro definition. 
rem   -DCPUS=n       N  defines the maximum number of CPUS Crafty will be able 
rem                     to use in a SMP system.  Note that this is the max you 
rem                     will be able to use.  You need to use the smpmt=n command 
rem                     to make crafty use more than the default 1 process. 
rem   -DDGT          N  This is a unix-only option to support the DGT board. 
rem   -DEPD          Y  if you want full EPD support built in. 
rem   -DHASHSTATS    N  This option compiles in some of the statistics 
rem                     gathering to provide hashing statistics. 
rem   -DINLINE32     N  Compiles with the Intel assembly code for FirstOne(), 
rem                     LastOne() and PopCnt().  This is for gcc-style inlining 
rem                     and now works with the Intel C/C++ compiler as well. 
rem   -DINLINE64     N  Compiles with the Intel assembly code for FirstOne(), 
rem                     LastOne() and PopCnt() for the AMD opteron, only tested 
rem                     with the 64-bit opteron GCC compiler / Intel ICC compiler. 
rem   -DLIMITEXT     N  limit extensions as search gets deeper 
rem   -DLOGDIR       N  path to the directory where Crafty puts the log.nnn and 
rem                     game.nnn files. 
rem   -DNOEGTB       N  eliminates the egtb code for compilers that can't deal 
rem                     with the large egtb.cpp code/templates. 
rem   -DNOFUTILITY   N  disables classic futility pruning 
rem   -DNUMA         N  says this system is NUMA, which is mainly used for Linux 
rem                     systems, and references libnuma, needed for the NUMA calls 
rem                     (crafty doesn't use many of these, it does the memory 
rem                     setup stuff itself) 
rem   -DPOWERPC      N  enables PPC spinlock inline function for SMP boxes only 
rem   -DRCDIR        N  path to the directory where we look for the .craftyrc or 
rem                     crafty.rc (windows) file. 
rem   -DSMP          N  enables SMP support (not clustering).  If you choose 
rem                     this option, you need to look closely at the following 
rem                     two options as well. 
rem   -DTBDIR        N  path to the directory where the endgame tablebase files 
rem                     are found.  default = "./TB" 
rem   -DTRACE        N  This enables the "trace" command so that the search tree 
rem                     can be dumped while running. 


rem  add  '#define inline __inline' to 'chess.h' 

rem  add 'typedef unsigned long DWORD_PTR;' to 'utility.c' 

rem comment out  'dgt.c' 'epd.c' 'epdglue.c' from 'crafty.c' (for speedup) 


rem  smp version below 
rem icl -o crafty.exe crafty.c egtb.cpp /DNT_i386 /DWIN32 /DCPUS=2 /DSMP /G7 /Ox /Qipo /QxK /MT /Qparallel /Qprof-gen 


rem  single cpu version below 
rem icl -o crafty.exe crafty.c egtb.cpp /DNT_i386 /DWIN32 /G7 /Ox /Qipo /QxK /MT /Qparallel /Qprof-gen 


cls 
del *.o 
del *.obj 
cls 


icl -o crafty.exe main.c attacks.c book.c boolean.c data.c draw.c drawn.c edit.c enprise.c epd.c epdglue.c evaluate.c give.c history.c init.c input.c interupt.c iterate.c lookup.c make.c movegen.c next.c nextc.c nexte.c resign.c root.c search.c setboard.c store.c swap.c test.c time.c utility.c valid.c validate.c /DNT_i386 /DWIN32 /DCPUS=2 /DSMP /G7 /Ox /Qipo /QxK /MT /Qparallel /Qopenmp /Qprof-gen 

Jim made some comments below to use your specified ICC compiler. You may use this procedure in ICC command prompt as well:

Read 'rem'med out comments for info on which files to edit before compiling.
Add the 'crafty.bat' + all crafty src files to 'Program Files\Intel\Compiler\C++\10.0.025\IA32\Bin' and run it.

Sorry for the long narration. I hope this helps.

[thciobanu]

You are most right, the hints I gave above are far from producing a competitive build. But they do give you a working version that you can tweak as you see fit. The reason I had to modify the header files is that there's no mention of a #define CPUS, as I've wanted just that, to have an .exe version for just one core (mostly for my personal curiosity to see if there was any difference between a SP crafty and a MP crafty with just one thread), and I stumbled upon those glitches in the preprocessor definitions.

[bob]

You are most right, the hints I gave above are far from producing a competitive build. But they do give you a working version that you can tweak as you see fit. The reason I had to modify the header files is that there's no mention of a #define CPUS, as I've wanted just that, to have an .exe version for just one core (mostly for my personal curiosity to see if there was any difference between a SP crafty and a MP crafty with just one thread), and I stumbled upon those glitches in the preprocessor definitions.

There is effectively no difference. If you look at the code, there is one if-test that is done for each node searched, and when I compare the speeds with SMP against no-SMP, the speed difference is so tiny as to be meaningless.....

I will try to fix this so that the next version compiles cleanly, Are there any other issues with compiling under windoze? I can get Tracy/Peter to confirm that whatever I fix will work... Main mistake is that I never compile with anything but CPUS=2 or more, and CPUS=1 makes a difference in some of the conditional compilation code that might be broken in that case...

[thciobanu]

There is effectively no difference.

Yes, I noticed that. When I first looked through the code I saw some for loops being left out for single CPU and was curious how much difference would they make in performance, but in my newbie test none were visible. Also no significant improvements were seen when comparing a SSE2 build with a "regular" one, but that again is probably just my noob-ness manifesting.

[bob]

Create an empty project, add crafty.c and egtb.cpp to it.
In the Project property pages, you should:
- select "use multi-byte character set" instead of unicode
- define NT_i386 if it's not already defined for the preprocessor.

Then, in dgt.c move the line
#if defined(DGT)
to the top (it's only supported on Linux afaik, so the #includes are Linux specific)

Then in lock.h I did the following:
- I moved
#if defined(_WIN32) || defined(_WIN64)
# define pthread_attr_t HANDLE
# define pthread_t HANDLE
# define thread_t HANDLE
#endif
to the top of the file (not nested in the #if CPUS > 1)
- added
# define lock_t volatile int
before the #endif on the last line.

And that about gives you a compiling version of crafty 21.6 on VC2008 (or at least it worked for me). You can then play around with compiler&linker options to get a faster crafty.

I believe I have all of these now correct in version 22.0... we will release this as soon as a couple of glitches are worked out...

[pedrox]

Hi Denis,

Thx for full details on compilation, are very useful, I was trying to find some messages of Jim and yours.

I have a question about pgo, I use microsoft visual c. I make "Profile Guided Optimization" --> "Instrument".

So I want to train the program, if I call the program from "Run Instumented/Optimized Application", it appears the console, at the console of danasah I have a test that I used to see 30 positions for 10 seconds, but I do like to do a test Positions epd or play several games? If I use Arena and install the engine that I have in the folder release, I have a message saying that "this application has failed to start becasuse PGORT90.DLL was not found.

Thx

Pedro

[Cardoso]

Thanks,
Alvaro

[Denis P. Mendoza]

Hi Denis,

Thx for full details on compilation, are very useful, I was trying to find some messages of Jim and yours.

I have a question about pgo, I use microsoft visual c. I make "Profile Guided Optimization" --> "Instrument".

So I want to train the program, if I call the program from "Run Instumented/Optimized Application", it appears the console, at the console of danasah I have a test that I used to see 30 positions for 10 seconds, but I do like to do a test Positions epd or play several games? If I use Arena and install the engine that I have in the folder release, I have a message saying that "this application has failed to start becasuse PGORT90.DLL was not found.

Thx

Pedro

Hello Pedro,

This problem you encountered maybe caused by too many MSVC compilers inside your pc. Maybe the default environmental variables, etc.. are not properly set for your new compiler. It also happened to me using MSVC8 and MSVC9. My MSVC6 is still active as I use it to check errors on ICC compiling (as errors are not completely seen in console mode for ICC).

Since I'm not so familiar with the compiler GUI instructions, here's a trial-and-error attempt which works at the moment. I'm not exactly sure if all the files I'll mention are needed. I had not time to verify!

1. For MSVC9 (2008), after creating an instrumented excutable (assuming it is a normal 32 bit compilation) copy the pgort90.dll and pgodb90.dll (from C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin) and paste it in the location of the executable. If it doesn't work, try also to include pgocvt.exe, pgomgr.exe, and pgosweep.exe. I'm sure it will collect profiling data and merge these datas to create optimized code in the end. It isn't a perfect procedure but it works.
2. For MSVC 2005, you only need the dlls

It's unusual, but it is the only procedure I know at the moment.

I hope this helps Pedro.

Denis