Chess Engine Compiler Script for Android

[Archimedes]

If you have already installed a working environment of clang or gcc on Termux and want to compile Stockfish (and may be other chess engines too) for arm64-v8a based CPUs, then, may be, this script makes your life easier.

Download CECSA (Chess Engine Compiler Script for Android), start Termux and copy the tar archive from the download directory of your smarthphone to the home directory of Termux (change the command line for your needs).

Code: Select all

cp /storage/emulated/0/Download/cecsa_2021.10.19.tar.gz ./

Extract the tar archive.

Code: Select all

tar -xf ./cecsa_2021.10.19.tar.gz

Now you have a directory cecsa in your home directory. Go to the cecsa directory and start the script.

Code: Select all

cd ./cecsa
./start.sh

What does the script do? When you start the script without any parameter, it download the sources from Stockfish (via "git clone"), copy a working makefile (etc/Stockfish/Makefile) to the source directory of Stockfish and start the compiliation process. When finished, the executable will be copied to the CECSA_HOME directory (/storage/emulated/0/Download) outside of Termux. The CECSA_HOME directory can be defined as an environment variable at the bottom of the script. With the CECSA_COMPILER environment variable can you define your preferred compiler (clang or gcc). On Android the gcc compiler is a lot faster than the clang compiler.

At the moment, the script supports the following parameters:

Code: Select all

./start.sh
./start.sh all
./start.sh corchess
./start.sh crystal
./start.sh stockfish

As you can see, currently CorChess, Crystal and Stockfish are supported, but can be extended to other chess engines too, for sure. With the parameter "all" all supported chess engines will be compiled (this is mainly for testing purposes). The changed makefiles (necessary for proper compiling) are in the etc directory. Changes to the makefiles were only made, when it was absolute necessary for proper compiling.

CECSA 2021.10.19:
https://app.box.com/s/2i7hcwadwq1ibz0n5ojqervla798vubu

[Archimedes]

I have added Demolito as new chess engine (builds made with Android NDK have poor performance).

When starting the script you will see now, what chess engine will be compiled. For example, when you start the script without any parameters, then "Compile Stockfish for Android?" will be appeared. If you answered this question with yes, then there will be a new question "With embedded network?". No more need to change the makefiles for that purpose. Just enter the information just in time.

The correct version number is always a part of the compilation tasks which i take care about. If you are not pleased with the automatic version numbering of Stockfish (when version string is empty, version number is the date of the compile), you can now enter your own version number (normally i'm using the date of the last commit in reverse order as version number). This can be done by adding an additional parameter when starting the script.

Example:

Code: Select all

./start.sh stockfish 211021

Stockfish now shows "Stockfish 211021" when starting with DroidFish.

This works with all included chess engines.

Example:

Code: Select all

./start.sh demolito 2021-10-21

Demolito, for example, shows "Demolito dev" in DroidFish when you start the script only with "./start.sh demolito".

And, last but not least, there are new environment variables related to the chess engines.

Code: Select all

CECSA_CORCHESS_MAKEFILE=Makefile
CECSA_CRYSTAL_MAKEFILE=Makefile
CECSA_DEMOLITO_MAKEFILE=makefile
CECSA_STOCKFISH_MAKEFILE=Makefile

With this new environment variables you can choose, what makefile you want to use for a specific chess engine. When you want to do some experiments with settings or want to use your own makefiles, just make a copy of the original makefile in the etc directory and work with this copy. No need to make backups of the original makefile when doing some tests or something like that. You can create several makefiles and choose them in the script.

For normal compilation tasks there is no need to edit the script (changing environment variables) or the makefiles, all tasks can be done in the command line, that is, choosing, whether to use an embedded network or not and entering a version number (which is optional). Things should be kept as simple as possible.

CECSA 2021.10.21:
https://app.box.com/s/0zr9zf8vipuvafz6gh53beimgd77j8tv

[Archimedes]

Support for Android NDK added. Now, you are able to compile although for armeabi-v7a, x86 and x86_64 based CPUs (needs an installed Android NDK).

But, before we begin, install zip in your Termux environment. It is needed for making the zip archive with all the executables at the end of compiling.

Code: Select all

apt-get install zip

Let's have a look in the environment variables in the script.

Code: Select all

CECSA_ARM64_V8A_ONLY=true
CECSA_COMPILER=gcc
CECSA_HOME=/storage/emulated/0/Download
CECSA_VERSION=2021.10.24
CECSA_WAKE_LOCK=true

CECSA_CORCHESS_MAKEFILE=Makefile
CECSA_CRYSTAL_MAKEFILE=Makefile
CECSA_DEMOLITO_MAKEFILE=makefile
CECSA_STOCKFISH_MAKEFILE=Makefile

CECSA now supports three different compilers: clang, gcc and ndk.

The environment variable CECSA_ARM64_V8A_ONLY is new and everybody should know, what it means. If true, only builds for arm64-v8a based CPUs will be generated. If false, then there will also be generated builds for armeabi-v7a, x86 and x86_64 based CPUs. This is the job for Android NDK. Android NDK should be properly installed (extracted to a directory of your choice) and the root directory should be in the path environment variable (the command "ndk-build" should work in the command line). With CECSA_ARM64_V8A_ONLY=false you enable Android NDK.

Example: If you start a compile for Stockfish with the following settings, then the build for arm64-v8a will be done with gcc and the rest with the help of Android NDK.

Code: Select all

CECSA_ARM64_V8A_ONLY=false
CECSA_COMPILER=gcc

Same, when you start a compilation with Clang.

Code: Select all

CECSA_ARM64_V8A_ONLY=false
CECSA_COMPILER=clang

When you start a compilation with Android NDK, then all builds will be made with Android NDK (although that for arm64-v8a).

Code: Select all

CECSA_ARM64_V8A_ONLY=false
CECSA_COMPILER=ndk

Example: If i want to distribute the current Stockfish with all available builds, then yo need the following settings in the script:

Code: Select all

CECSA_ARM64_V8A_ONLY=false
CECSA_COMPILER=gcc

After that, start the compiliation process with the following command line (which includes a version number).

Code: Select all

./start.sh stockfish 211023

CECSA 2021.10.24:
https://app.box.com/s/z9wxi4cjledxbas0wktuvedata2tk66v

[Archimedes]

Installing Android NDK on Termux is simple. Just donwload the latest version from https://github.com/Lzhiyong/termux-ndk (currently this is the file android-ndk-r23.tar.xz). Extract the file to a directory in Termux of your choice, e. g. /data/data/com.termux/files/usr/lib/android-ndk, and put that directory to your path environment variable. I used a hidden file named .bashrc in the home directory for that.

.bashrc:

Code: Select all

export PATH=/data/data/com.termux/files/cctools-toolchain/bin:$PATH
export PATH=$PATH:/data/data/com.termux/files/usr/lib/android-ndk

That's all.

[Archimedes]

I removed the environment variable CECSA_ARM64_V8A_ONLY. Hopefully the environment variables are more clear now.

Code: Select all

CECSA_ABI_ARMEABI_V7A=true
CECSA_ABI_X86=false
CECSA_ABI_X86_64=false
CECSA_COMPILER=gcc
CECSA_HOME=/storage/emulated/0/Download
CECSA_MAKE_JOBS=1
CECSA_README=true
CECSA_VERSION=2021.10.25
CECSA_WAKE_LOCK=true

The ABIs armeabi-v7a, x86 and x86_64 can now be activated or deactivated independently from each other. If one of them is activated, you need Android NDK.

With CECSA_MAKE_JOBS you can define how many jobs the make command will use. It is something like number of threads for the make command.

With CECSA_README you can define, wether a readme file will be included in the resulting zip archiv or not.

CECSA 2021.10.25:
https://app.box.com/s/zfz5wp85fz0wyeapazma2ucxru5rmcko

[Archimedes]

Although it was primary intended for making Stockfish compiles, the CECSA script can easiliy be extended for usage with other chess engines too die to its modular approach. In the meanwhile the following chess engines are supported: Berserk, BrainLearn, CorChess, Crystal, Demolito, ShashChess, Stockfish, SugaR AI and SugaR AI ICCF.

The main environemnt variables are as follow:

Code: Select all

CECSA_ABI_ARMEABI_V7A=true
CECSA_ABI_X86=false
CECSA_ABI_X86_64=false
CECSA_COMPILER=gcc
CECSA_PGO=true

CECSA_HOME=/storage/emulated/0/Download
CECSA_MAKE_JOBS=1
CECSA_ONLINE=true
CECSA_VERSION=2021.11.11
CECSA_WAKE_LOCK=true
CECSA_ZIP_JNI=false
CECSA_ZIP_NETWORK=false
CECSA_ZIP_README=false
CECSA_ZIP_SOURCE=false

The most essential environment variables are at the bottom of the script. As you can see, there is a new option "CECSA_PGO". With CECSA_PGO=true, profile guided optimization will be used (if available).

CECSA_ONLINE=true means, that the source code of an chess engine will download first from GitHub before the compilation process starts (this is the default behavior). If CECSA_ONLINE=false the script doesn't download any sources from GitHub. The script assumes that the source code is already in the src directory. With this feature you can do now an offline compilation. Download the source code manually, extract it to src directory (take care of the directory structure and names, must be identical with GitHub), set CECSA_ONLINE to false and start the script. CECSA will not donwload any sources from GitHub now, instead it starts the compile immediately. Stockfish for example needs the directory name "Stockfish" in the src directory (as you can see it, after downloading from GitHub), otherwise the script will not find the sources.

With CECSA_ZIP_JNI=true the files for Android NDK will be included in the zip archive.

With CECSA_ZIP_NETWORK=true the current network file will be included in the zip archive (only works, when you have choosen non embedded network at the begin of compile).

With CECSA_ZIP_SOURCE=true the source code (may be changed from the script itself for proper compiling) will be included in the zip archive.

The most essential environment variables in the script can be overwritten by engine settings (could be helpful in some situations).

Example:

Code: Select all

# Berserk
CECSA_BERSERK_ABI_ARMEABI_V7A=
CECSA_BERSERK_ABI_X86=
CECSA_BERSERK_ABI_X86_64=
CECSA_BERSERK_COMPILER=
CECSA_BERSERK_PGO=
CECSA_BERSERK_MAKEFILE=makefile

Always keep in mind, when there is a compilation error during the compilation process, the script may not responsible for that. It is only executing the make command as you will do it manually. Crystal and Sugar AI ICCF for example can't be compiled for armeabi-v7a based CPUs (need changes in the source code). And so on.

And as a compiler, you must always take care of libraries which have to be present on your environment. For example, to compile BrainLearn and ShashChess you need at least the follwowing packages on your system (if not already present). Assuming, you have installed the GCC compiler correctly. However, when a library is missing, you will see that in an error message.

Code: Select all

apt-get install curl, libcurl-static libnghttp2-static libssh2-static openssl-static zlib-static
apt-get install ndk-multilib

CECSA 2021.11.11:
https://app.box.com/s/prvc6k62m9f4qcq8vs4r8xqy4qswg2tz

[Archimedes]

CECSA 2022.02.17:
https://app.box.com/s/11ljb29ywal4ohbq3a0pbs452lwbbzgz

[Archimedes]

Updated the makefiles from BrainLearn and ShashChess.

CECSA 2022.03.07:
https://app.box.com/s/it7efdz5wchedslmz6nwipleghezxduc

[jshriver]

This is an interesting project and surprised these can be done on the android device itself.

Once compiled are these engines exposed to OEX GUIs?

[Archimedes]

This is an interesting project and surprised these can be done on the android device itself.

Once compiled are these engines exposed to OEX GUIs?

No, you only get a zip archive with all executables. Works fine with DroidFish and Chess for Android 6.2.1. For OEX you need something like Android Studio.