Zahak, a GoLang based chess engine

[amanjpro]

BTW, my linux-arm-{32,64 bits} both work just fine on Android, not sure if you do anything more in your binaries, if there is some option I should provide to make your life easier, I'll

I compile all go engines directly for Android (e. g. GOOS=android GOARCH=arm64). Therefore you need Android NDK.

Ah I see, then I'll add it to my CI to build it for Android target too.

A naiive question, does the binary that is compiled for Android yield better nps than the one for generic Linux arm64?

[Archimedes]

A naiive question, does the binary that is compiled for Android yield better nps than the one for generic Linux arm64?

Yes, a little bit. I would estimate around 4 % more nps.

Assuming the following libs directory already exists and Android NDK is already installed, then the following shell script (in the root directory of the project) will create Android builds.

Code: Select all

go run netgen/nn.go
cd zahak
export CGO_ENABLED=1
GOOS=android GOARCH=arm64         CC=/usr/lib/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang    go build -o ../libs/arm64-v8a/Zahak
GOOS=android GOARCH=arm   GOARM=7 CC=/usr/lib/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi16-clang go build -o ../libs/armeabi-v7a/Zahak
GOOS=android GOARCH=386           CC=/usr/lib/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android16-clang       go build -o ../libs/x86/Zahak
GOOS=android GOARCH=amd64         CC=/usr/lib/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/x86_64-linux-android21-clang     go build -o ../libs/x86_64/Zahak

This short example should give you the idea how to do that. I'm still using the latest LTS version r21e of Android NDK for that purpose. As you can see, 64 bit builds are targeting API level 21, 32 bit builds are targeting API level 16. I'm using the libs directory structure, because it is the default in Android NDK, when compiling C and C++ sources for Android.

[amanjpro]

A naiive question, does the binary that is compiled for Android yield better nps than the one for generic Linux arm64?

Yes, a little bit. I would estimate around 4 % more nps.

Assuming the following libs directory already exists and Android NDK is already installed, then the following shell script (in the root directory of the project) will create Android builds.

Code: Select all

go run netgen/nn.go
cd zahak
export CGO_ENABLED=1
GOOS=android GOARCH=arm64         CC=/usr/lib/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang    go build -o ../libs/arm64-v8a/Zahak
GOOS=android GOARCH=arm   GOARM=7 CC=/usr/lib/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi16-clang go build -o ../libs/armeabi-v7a/Zahak
GOOS=android GOARCH=386           CC=/usr/lib/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android16-clang       go build -o ../libs/x86/Zahak
GOOS=android GOARCH=amd64         CC=/usr/lib/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/x86_64-linux-android21-clang     go build -o ../libs/x86_64/Zahak

This short example should give you the idea how to do that. I'm still using the latest LTS version r21e of Android NDK for that purpose. As you can see, 64 bit builds are targeting API level 21, 32 bit builds are targeting API level 16. I'm using the libs directory structure, because it is the default in Android NDK, when compiling C and C++ sources for Android.

Noted, thanks a lot

[Wolfgang]

CEGT-40/4: https://cegt.forumieren.com/t1588-testi ... 7-0nn#3074

[lithander]

Meet the latest and greatest Zahak, version 7.0 with NNUE:

https://github.com/amanjpro/zahak/releases/tag/7.0

More details are here in this article: https://zahak.amanj.me/posts/meet-zahak-7/

TL; DR; the trainer is written from scratch, the data is from self-play, and you can train different nets with provided tools if you like

Elo estimation: +190 over Zahak 6.2

Oh, I missed this news! Congratz on the release!

I wonder, how much of the 190 ELO gain is due to the NNUE and how much to the other changes? How strong is Zahak 7.0 when using BitGenie's latest net? (after you ensured compatibility I'm sure you tested that, too?)

I liked your write up on how to train nets but to be able to try it we'd need massive amounts of training data first. Do you have your training data ("training-dataset.epd") available for download somewhere?

[amanjpro]

Meet the latest and greatest Zahak, version 7.0 with NNUE:

https://github.com/amanjpro/zahak/releases/tag/7.0

More details are here in this article: https://zahak.amanj.me/posts/meet-zahak-7/

TL; DR; the trainer is written from scratch, the data is from self-play, and you can train different nets with provided tools if you like

Elo estimation: +190 over Zahak 6.2

Oh, I missed this news! Congratz on the release!

I wonder, how much of the 190 ELO gain is due to the NNUE and how much to the other changes? How strong is Zahak 7.0 when using BitGenie's latest net? (after you ensured compatibility I'm sure you tested that, too?)

I liked your write up on how to train nets but to be able to try it we'd need massive amounts of training data first. Do you have your training data ("training-dataset.epd") available for download somewhere?

Thanks a lot Thomas

The net alone added nothing, I had to disable Futility Pruning for it to work, as FP is very much dependant on eval. Together it was worth 100 points

AVX added another 40 elos, draw pattern recognition another 20, reverse futility pruning tweaks the rest

My net is only compatible with BitGenie dev, not 9. And BG net in dev is fifty elo above v9. And somehow Zahak was better with Zahak net, and BG was better with BG net. But I believe BG net is stronger

As for the dataset, I'm thinking of a repository to host nets, list their authors and purpose, with datasets too... I already have four nets hhh. Some are meant for limited strength for human play for example.

Issue with putting the data publicly is that if we all use the same data to train a net for Zahak, we all produce more or less the same net... I might add some data to bootstrap the process. But I also change my fengen:

- To add ability to ignore moves by certain engines, that way Zahak vs another engine is also possible to be used (already in fengen)

- To ignore the scores in pgn and use Zahak to evaluate the moves. That way you can use human games, or minimal chess self play games to produce nets for Zahak

But yeah, it all starts with adding a repository and index for the trained nets

[Wolfgang]

Meet the latest and greatest Zahak, version 7.0 with NNUE:

https://github.com/amanjpro/zahak/releases/tag/7.0

More details are here in this article: https://zahak.amanj.me/posts/meet-zahak-7/

TL; DR; the trainer is written from scratch, the data is from self-play, and you can train different nets with provided tools if you like

Elo estimation: +190 over Zahak 6.2

CEGT 40/4:
https://cegt.forumieren.com/t1588-testing-zahak-7-0nn

CEGT 5+3Ponder:
https://cegt.forumieren.com/t1589-new-tournament

[amanjpro]

Meet the latest and greatest Zahak, version 7.0 with NNUE:

https://github.com/amanjpro/zahak/releases/tag/7.0

More details are here in this article: https://zahak.amanj.me/posts/meet-zahak-7/

TL; DR; the trainer is written from scratch, the data is from self-play, and you can train different nets with provided tools if you like

Elo estimation: +190 over Zahak 6.2

CEGT 40/4:
https://cegt.forumieren.com/t1588-testing-zahak-7-0nn

CEGT 5+3Ponder:
https://cegt.forumieren.com/t1589-new-tournament

Super, thanks a lot for testing Zahak

I'm super excited for the pondering one

[Werner]

Is there a difference between the included network and the network provided on guthub:
Use a bigger network of size 768 x 245 x 1 (#130)

Werner

[amanjpro]

Is there a difference between the included network and the network provided on guthub:
Use a bigger network of size 768 x 245 x 1 (#130)

Werner

They are different, the one in GitHub is 60+ elo stronger than the included one. But Zahak 7.1 cannot read that other one. Just before including that network, I also changed the probing code in Zahak to be able to read different network sizes