Stockfish Cluster MPI
Moderators: bob, hgm, Harvey Williamson
Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Stockfish Cluster MPI
I would like to test the stockfish cluster version. https://github.com/official-stockfish/S ... ee/cluster
Can someone provide an .exe file?
Can someone provide an .exe file?
-
- Posts: 10
- Joined: Fri Feb 01, 2019 10:56 am
- Full name: Jeremy Tyrania
Re: Stockfish Cluster MPI
Thank you for your effort, but unfortunately it does not work.
Did you use the compiler options in the readme file?
To build the cluster branch, it is sufficient to specify COMPILER=mpicxx on the make command line, and do a clean build:
make -j ARCH=x86-64-modern clean build COMPILER=mpicxx
If the name of the compiler wrapper (typically mpicxx, but sometimes e.g. CC) does not match mpi an edit to the Makefile is required. Make sure that the MPI installation is configured to support MPI_THREAD_MULTIPLE, this might require adding system specific compiler options to the Makefile. Stockfish employs non-blocking (asynchronous) communication, and benefits from an MPI implementation that efficiently supports this. Some MPI implentations might benefit from leaving 1 core/thread free for these asynchronous communications, and might require setting additional environment variables. Refer to your MPI documentation for more info.
Did you use the compiler options in the readme file?
To build the cluster branch, it is sufficient to specify COMPILER=mpicxx on the make command line, and do a clean build:
make -j ARCH=x86-64-modern clean build COMPILER=mpicxx
If the name of the compiler wrapper (typically mpicxx, but sometimes e.g. CC) does not match mpi an edit to the Makefile is required. Make sure that the MPI installation is configured to support MPI_THREAD_MULTIPLE, this might require adding system specific compiler options to the Makefile. Stockfish employs non-blocking (asynchronous) communication, and benefits from an MPI implementation that efficiently supports this. Some MPI implentations might benefit from leaving 1 core/thread free for these asynchronous communications, and might require setting additional environment variables. Refer to your MPI documentation for more info.
-
- Posts: 10
- Joined: Fri Feb 01, 2019 10:56 am
- Full name: Jeremy Tyrania
Re: Stockfish Cluster MPI
Ahh I tried to compile it like normal stockfish :p I turned on MPI in the makefile and also checked during the compile it said mpi: 'yes'. Hope this one works.
https://www.dropbox.com/s/ejmlrevz8nuzk ... r.zip?dl=0

https://www.dropbox.com/s/ejmlrevz8nuzk ... r.zip?dl=0
Re: Stockfish Cluster MPI
Question:
Why does it start like a normal engine?
should this engine start in consolemode only if MPI is installed ?
regards, C.K.
Why does it start like a normal engine?
should this engine start in consolemode only if MPI is installed ?
regards, C.K.
komodoslash001 wrote: ↑Sun Feb 03, 2019 4:39 pmAhh I tried to compile it like normal stockfish :p I turned on MPI in the makefile and also checked during the compile it said mpi: 'yes'. Hope this one works.![]()
https://www.dropbox.com/s/ejmlrevz8nuzk ... r.zip?dl=0
Re: Stockfish Cluster MPI
First of all: Thanks to komodoslash001 for compiling!
Unfortunately it doesn't work. If you start it with someting like: "mpiexec -hosts 2 host1 host2 /path/to/stockfish-executable" then Stockfish starts on both hosts but the two processes don't communicate with each other. Maybe there is still something missing in the compile or it does only work when compiling for Linux.
Best regards
Timo
Unfortunately it doesn't work. If you start it with someting like: "mpiexec -hosts 2 host1 host2 /path/to/stockfish-executable" then Stockfish starts on both hosts but the two processes don't communicate with each other. Maybe there is still something missing in the compile or it does only work when compiling for Linux.
Best regards
Timo
Re: Stockfish Cluster MPI
This is such an exciting project, it'd be great to get it working easily.
I really hope the SF team make this easily accessible with clear instructions of how to link multiple PCs. It could kick start new interest in alpa-betas.
I really hope the SF team make this easily accessible with clear instructions of how to link multiple PCs. It could kick start new interest in alpa-betas.
Re: Stockfish Cluster MPI
I have uploaded stockfish_cluster_win64.7z to this public dropbox folder:fastgm wrote: ↑Sun Feb 03, 2019 7:22 amI would like to test the stockfish cluster version. https://github.com/official-stockfish/S ... ee/cluster
Can someone provide an .exe file?
https://www.dropbox.com/sh/v16dye7ww33x ... Oxasa/test
Stockfish is compiled for the x86-64-modern architecture. There are some limitations though:
- Windows 8 or later is needed to support MPI_THREAD_MULTIPLE which stockfish is using.
- Version 10.0 of MS-MPI is needed for bug fixes in the MPI_THREAD_MULTIPLE handling. You can get it from here: https://www.microsoft.com/en-us/downloa ... x?id=57467
- I had to disable link time optimization (-flto) when compiling stockfish, otherwise the program crashes. Maybe this happens because I cross compile from linux to windows.
If there are two computers called host1 and host2 and MS-MPI is installed on both computers, proceed as follows:
- On all computers, log in as the same user.
- On all computers, add firewall exceptions to allow the programs mpiexec and smpd (located in C:\Program Files\Microsoft MPI\Bin) to communicate over the network.
- On all computers, start a command prompt and execute: smpd -d 0
- Make sure stockfish is installed in the same directory on all computers.
- On the host1 computer, start a command prompt and execute:
cd /directory/where/stockfish/is/installed
mpiexec -hosts 2 host1 host2 stockfish-cluster.exe
Re: Stockfish Cluster MPI
Are you going to let two Clusterfishes battle it out until depth 70?fastgm wrote: ↑Sun Feb 03, 2019 7:22 amI would like to test the stockfish cluster version. https://github.com/official-stockfish/S ... ee/cluster
Can someone provide an .exe file?