Stockfish NN release (NNUE)

[zullil]

Hello guys!
I have a couple questions regarding android engines. I've made engines armv7 and armv8 (from nodchip's source code), and when i start the engine in the Droidfish, engine's behavior is identical to normal SF (although nps are much lower). Also in Droidfish/uci directory i've created a folder "eval" with "nn.bin" file inside, So here is the questions, how can i be sure that engine is using nn.bin?
Maybe because i use NDK's Clang for Android, engine must to have support for Clang, or i need to make some changes in source code?
Any help is much appreciated!
Here is the engines if you want to try. https://www.mirrored.to/files/1JXPZSMW/

Regards.
Alex.

start the engine in terminal and type "ucinewgame" - you should see a message confirming the nnfile was loaded.

If you do not, you may have that missed that the build command was changed for building NN

e.g., make ARCH=armv7 build COMP=gcc
would become
make -ARCH=armv7 nnue COMP=gcc

I think that should be

make nnue ARCH=armv7

(COMP=gcc is the default)

[MikeB]

Hello guys!
I have a couple questions regarding android engines. I've made engines armv7 and armv8 (from nodchip's source code), and when i start the engine in the Droidfish, engine's behavior is identical to normal SF (although nps are much lower). Also in Droidfish/uci directory i've created a folder "eval" with "nn.bin" file inside, So here is the questions, how can i be sure that engine is using nn.bin?
Maybe because i use NDK's Clang for Android, engine must to have support for Clang, or i need to make some changes in source code?
Any help is much appreciated!
Here is the engines if you want to try. https://www.mirrored.to/files/1JXPZSMW/

Regards.
Alex.

start the engine in terminal and type "ucinewgame" - you should see a message confirming the nnfile was loaded.

If you do not, you may have that missed that the build command was changed for building NN

e.g., make ARCH=armv7 build COMP=gcc
would become
make -ARCH=armv7 nnue COMP=gcc

I think that should be

make nnue ARCH=armv7

(COMP=gcc is the default)

Indeed, you are correct . That is what happens when a CPA starts giving out coding advice ;>) Thanks!

[Aleksei]

start the engine in terminal and type "ucinewgame" - you should see a message confirming the nnfile was loaded.

If you do not, you may have that missed that the build command was changed for building NN

e.g., make ARCH=armv7 build COMP=gcc
would become
make -ARCH=armv7 nnue COMP=gcc

Hi Mike, thanks for the tip, but still it doesn't work for me. I'm using NDK's toolchain with Clang (not GCC) for android engines, on my PC (Windows 7 x64). Since android's Clang doesn't have a shell (the shell appears only once, when clang generate engines). I'm using a batch file with follow commands (eg for normal Stockfish arm7).

Code: Select all

echo Generating 32 bit engine for ARM-Android.
set "compiler_options=-m32 -march=armv7-a -fPIE -Wl,-pie -mfloat-abi=softfp -mfpu=vfpv3-d16 -mthumb -Wl,--fix-cortex-a8 -lm -DUSE_POPCNT -O3 -flto -static-libstdc++ -std=c++11 -fno-strict-aliasing -fno-strict-overflow -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,-s"

So it would be nice if someone could explain which flags i have to write in batch file for SF NNUE, i've tried -nnue and -fnnue but compiler doesn't understand it. I might be wrong, but i think the source code need to have a better support for Clang.
Thank you!

[MikeB]

start the engine in terminal and type "ucinewgame" - you should see a message confirming the nnfile was loaded.

If you do not, you may have that missed that the build command was changed for building NN

e.g., make ARCH=armv7 build COMP=gcc
would become
make -ARCH=armv7 nnue COMP=gcc

Hi Mike, thanks for the tip, but still it doesn't work for me. I'm using NDK's toolchain with Clang (not GCC) for android engines, on my PC (Windows 7 x64). Since android's Clang doesn't have a shell (the shell appears only once, when clang generate engines). I'm using a batch file with follow commands (eg for normal Stockfish arm7).

Code: Select all

echo Generating 32 bit engine for ARM-Android.
set "compiler_options=-m32 -march=armv7-a -fPIE -Wl,-pie -mfloat-abi=softfp -mfpu=vfpv3-d16 -mthumb -Wl,--fix-cortex-a8 -lm -DUSE_POPCNT -O3 -flto -static-libstdc++ -std=c++11 -fno-strict-aliasing -fno-strict-overflow -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,-s"

So it would be nice if someone could explain which flags i have to write in batch file for SF NNUE, i've tried -nnue and -fnnue but compiler doesn't understand it. I might be wrong, but i think the source code need to have a better support for Clang.
Thank you!

Try -"nnue" without the dash ...just "nnue' - no quotes of course ..

[MikeB]

start the engine in terminal and type "ucinewgame" - you should see a message confirming the nnfile was loaded.

If you do not, you may have that missed that the build command was changed for building NN

e.g., make ARCH=armv7 build COMP=gcc
would become
make -ARCH=armv7 nnue COMP=gcc

Hi Mike, thanks for the tip, but still it doesn't work for me. I'm using NDK's toolchain with Clang (not GCC) for android engines, on my PC (Windows 7 x64). Since android's Clang doesn't have a shell (the shell appears only once, when clang generate engines). I'm using a batch file with follow commands (eg for normal Stockfish arm7).

Code: Select all

echo Generating 32 bit engine for ARM-Android.
set "compiler_options=-m32 -march=armv7-a -fPIE -Wl,-pie -mfloat-abi=softfp -mfpu=vfpv3-d16 -mthumb -Wl,--fix-cortex-a8 -lm -DUSE_POPCNT -O3 -flto -static-libstdc++ -std=c++11 -fno-strict-aliasing -fno-strict-overflow -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,-s"

So it would be nice if someone could explain which flags i have to write in batch file for SF NNUE, i've tried -nnue and -fnnue but compiler doesn't understand it. I might be wrong, but i think the source code need to have a better support for Clang.
Thank you!

Try -"nnue" without the dash ...just "nnue' - no quotes of course ..

also, just fyi, we did it get it to compile on a 32 bit pi, but it did require gcc 10 and I don't believe on the pi, clang would compile this -did not even try a as I recognized gcc 10 was going to be required .. and gcc 10 is not the standard on pi - not sure I can help you if the "nnue" does not work without the quotes - we are beyond my level of what limited expertise I have

[MikeB]

start the engine in terminal and type "ucinewgame" - you should see a message confirming the nnfile was loaded.

If you do not, you may have that missed that the build command was changed for building NN

e.g., make ARCH=armv7 build COMP=gcc
would become
make -ARCH=armv7 nnue COMP=gcc

Hi Mike, thanks for the tip, but still it doesn't work for me. I'm using NDK's toolchain with Clang (not GCC) for android engines, on my PC (Windows 7 x64). Since android's Clang doesn't have a shell (the shell appears only once, when clang generate engines). I'm using a batch file with follow commands (eg for normal Stockfish arm7).

Code: Select all

echo Generating 32 bit engine for ARM-Android.
set "compiler_options=-m32 -march=armv7-a -fPIE -Wl,-pie -mfloat-abi=softfp -mfpu=vfpv3-d16 -mthumb -Wl,--fix-cortex-a8 -lm -DUSE_POPCNT -O3 -flto -static-libstdc++ -std=c++11 -fno-strict-aliasing -fno-strict-overflow -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,-s"

So it would be nice if someone could explain which flags i have to write in batch file for SF NNUE, i've tried -nnue and -fnnue but compiler doesn't understand it. I might be wrong, but i think the source code need to have a better support for Clang.
Thank you!

Try -"nnue" without the dash ...just "nnue' - no quotes of course ..

also, just fyi, we did it get it to compile on a 32 bit pi, but it did require gcc 10 and I don't believe on the pi, clang would compile this -did not even try a as I recognized gcc 10 was going to be required .. and gcc 10 is not the standard on pi - not sure I can help you if the "nnue" does not work without the quotes - we are beyond my level of what limited expertise I have

You might want to grab the source from my github site - for the Pi , I had to add one system header manually
- it was not for the faint of heart. I will post the link if you want to try ,I have separate branch for the Pi - somebody, a while back submitted a patch to my fork compile for the andorid - but I do not know if it will work here. Again - I'm a retied CPA, I'm not a programmer - had one course in programming -APL in 1979. APL was used primarily in IBM mainframes starting in the 60'/70'ss, it hung around for while since so many Iof BM mainframes hung around for awhile. My formal training education in programming is VERY LOW. I only do this because I like to do it, not because I'm good at it.

[Scipion]

New, Blitz 1min+3sec-1 2020

1 2 3 4 5
1 SF NNUE halfkp-256 090720,x64 bmi2 ********** ½0½½½½½½½½ ½½1½½½½½½½ ½½½½½1½½½½ 1½1½½1½1½1 23.0/40
2 SF200620MZ 64 BMI2 ½1½½½½½½½½ ********** ½½0½½½½½½½ ½½½½½½½½½½ ½½11½1½½½1 22.0/40
3 BrainLearn 9.1.1 64 BMI2 ½½0½½½½½½½ ½½1½½½½½½½ ********** ½½½½½½½½½½ 10½11½½½1½ 21.5/40
4 Stockfish 110720 64 BMI2 ½½½½½0½½½½ ½½½½½½½½½½ ½½½½½½½½½½ ********** ½½1½½½½½1½ 20.5/40
5 Komodo 14 64-bit 0½0½½0½0½0 ½½00½0½½½0 01½00½½½0½ ½½0½½½½½0½ ********** 13.0/40

Little tournament 2 weeks ago, 2 threads each, no book .... Just wanted to share it ... seems times where SF was always alone on top of all engines tournaments, and far ahead, are over ...

[petero2]

I'm using NDK's toolchain with Clang (not GCC) for android engines, on my PC (Windows 7 x64). Since android's Clang doesn't have a shell (the shell appears only once, when clang generate engines). I'm using a batch file with follow commands (eg for normal Stockfish arm7).

Code: Select all

echo Generating 32 bit engine for ARM-Android.
set "compiler_options=-m32 -march=armv7-a -fPIE -Wl,-pie -mfloat-abi=softfp -mfpu=vfpv3-d16 -mthumb -Wl,--fix-cortex-a8 -lm -DUSE_POPCNT -O3 -flto -static-libstdc++ -std=c++11 -fno-strict-aliasing -fno-strict-overflow -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,-s"

So it would be nice if someone could explain which flags i have to write in batch file for SF NNUE

The Makefile contains the following:

Code: Select all

nnue: config-sanity
	$(MAKE) CXXFLAGS='$(CXXFLAGS) -DEVAL_NNUE -DENABLE_TEST_CMD -fopenmp' LDFLAGS='$(LDFLAGS) -fopenmp' build

So I think you need to add "-DEVAL_NNUE -DENABLE_TEST_CMD -fopenmp" to your bat file.

[Aleksei]

I'm using NDK's toolchain with Clang (not GCC) for android engines, on my PC (Windows 7 x64). Since android's Clang doesn't have a shell (the shell appears only once, when clang generate engines). I'm using a batch file with follow commands (eg for normal Stockfish arm7).

Code: Select all

echo Generating 32 bit engine for ARM-Android.
set "compiler_options=-m32 -march=armv7-a -fPIE -Wl,-pie -mfloat-abi=softfp -mfpu=vfpv3-d16 -mthumb -Wl,--fix-cortex-a8 -lm -DUSE_POPCNT -O3 -flto -static-libstdc++ -std=c++11 -fno-strict-aliasing -fno-strict-overflow -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,-s"

So it would be nice if someone could explain which flags i have to write in batch file for SF NNUE

The Makefile contains the following:

Code: Select all

nnue: config-sanity
	$(MAKE) CXXFLAGS='$(CXXFLAGS) -DEVAL_NNUE -DENABLE_TEST_CMD -fopenmp' LDFLAGS='$(LDFLAGS) -fopenmp' build

So I think you need to add "-DEVAL_NNUE -DENABLE_TEST_CMD -fopenmp" to your bat file.

Hi Peter, please check your PM!

[petero2]

I'm using NDK's toolchain with Clang (not GCC) for android engines, on my PC (Windows 7 x64). Since android's Clang doesn't have a shell (the shell appears only once, when clang generate engines). I'm using a batch file with follow commands (eg for normal Stockfish arm7).

Code: Select all

echo Generating 32 bit engine for ARM-Android.
set "compiler_options=-m32 -march=armv7-a -fPIE -Wl,-pie -mfloat-abi=softfp -mfpu=vfpv3-d16 -mthumb -Wl,--fix-cortex-a8 -lm -DUSE_POPCNT -O3 -flto -static-libstdc++ -std=c++11 -fno-strict-aliasing -fno-strict-overflow -ffunction-sections -fdata-sections -Wl,--gc-sections -Wl,-s"

So it would be nice if someone could explain which flags i have to write in batch file for SF NNUE

The Makefile contains the following:

Code: Select all

nnue: config-sanity
	$(MAKE) CXXFLAGS='$(CXXFLAGS) -DEVAL_NNUE -DENABLE_TEST_CMD -fopenmp' LDFLAGS='$(LDFLAGS) -fopenmp' build

So I think you need to add "-DEVAL_NNUE -DENABLE_TEST_CMD -fopenmp" to your bat file.

Hi Peter, please check your PM!

Hi Aleksei,

I don't like using PM for things that could be of interest for more than one person, but here are some more things I found out:

I also got crashes when using NDK r21d, so I used version r20b instead. I suspect this is related to the -fopenmp flag and I also suspect you only need this flag to run the training code, but I have not tested this.

For recent NDK versions you no longer need to create a standalone toolchain. You can run the compiler directly from the bin directory created when unpacking the NDK zip file, e.g. something like d:\android-ndk-r20b\toolchains\llvm\prebuilt\windows_something\bin. It may still work to create a standalone toolchain though, I have not tested that.

You don't need to change the Makefile when using your bat file because it is invoking the compiler directly so the Makefile is not used.