Sapeli 1.0 - New chess engine

[Gabor Szots]

Trying PEXT stuff to get rid of Magic index lookup tables.

This makes it impossible I think. I tried to compile with -m64 instead of -march=native but the compiler complained of some PEXT stuff.

[Terje]

Trying PEXT stuff to get rid of Magic index lookup tables.

This makes it impossible I think. I tried to compile with -m64 instead of -march=native but the compiler complained of some PEXT stuff.

If he has made pext the default and only version then there's probably nothing you can do to run it on a machine without pext

[JohnWoe]

Trying PEXT stuff to get rid of Magic index lookup tables.

This makes it impossible I think. I tried to compile with -m64 instead of -march=native but the compiler complained of some PEXT stuff.

Yes.
Like Terje said: If you don't have that instruction available then it's not gonna work.
pext is a neat way to get unique index to magic moves. I made Sapeli 1.91 purposely only BMI2. It's the fastest available.

I made some fixes to 1.92 (https://github.com/SamuraiDangyo/Sapeli ... r/Sapeli.c) as my AMD processor doesn't support pext.
So pext is optional. But preferred! Compile with -DPEXT.
If Windows compile with -DWINDOWS.
I don't know about PopCount(). It's well supported.

[Gabor Szots]

Trying PEXT stuff to get rid of Magic index lookup tables.

This makes it impossible I think. I tried to compile with -m64 instead of -march=native but the compiler complained of some PEXT stuff.

Yes.
Like Terje said: If you don't have that instruction available then it's not gonna work.
pext is a neat way to get unique index to magic moves. I made Sapeli 1.91 purposely only BMI2. It's the fastest available.

I made some fixes to 1.92 (https://github.com/SamuraiDangyo/Sapeli ... r/Sapeli.c) as my AMD processor doesn't support pext.
So pext is optional. But preferred! Compile with -DPEXT.
If Windows compile with -DWINDOWS.
I don't know about PopCount(). It's well supported.

Makefile did not work but gcc *.c -O3 -march=native -DWINDOWS -DNDEBUG -DPEXT -oSapeli_1.92-x64-SzG.exe did.

1.91 Makefile was OK.

[mvanthoor]

Yes.
Like Terje said: If you don't have that instruction available then it's not gonna work.
pext is a neat way to get unique index to magic moves. I made Sapeli 1.91 purposely only BMI2. It's the fastest available.

I made some fixes to 1.92 (https://github.com/SamuraiDangyo/Sapeli ... r/Sapeli.c) as my AMD processor doesn't support pext.
So pext is optional. But preferred! Compile with -DPEXT.
If Windows compile with -DWINDOWS.
I don't know about PopCount(). It's well supported.

If you PEXT, or compile with BMI2 (or both), your engine won't run at all on CPU's that do not have these instructions. On AMD cpu's, the engine will run very slow, compared to a non-PEXT version that is compiled with POPCNT only. (This has been proven in many benchmark, and numerous times it's been discussed here.)

To cover all bases, you'd need to create a conditional compilation that either uses 'normal' magic bit boards, or pext bit boards on request; and additionally, compile with POPCNT for AMD. That gives a whole list of executables:

BMI2 (and PEXT if you use this) for newer Intel CPU's since 2014 (Newer Haswells, Skylake, and up; it seems BMI2 and PEXT go together with Intel; there is no CPU with only one of them, as far as I know.) This compilation is often called "BMI2" or "Ultra"

POPCNT (for Intel CPU's since 2006, and all AMD's) This one is often callled "POPCNT" or "Modern".

Generic x64 (older x64 CPU's from before 2006.) Often called "x64", or "generic"

32-bit compilation (For 32-bit cpu's obviously, since the P2 MMX, 1997). Often called "x86", "mmx", or "old".

If you want... i586 or even i386. This will run on everything; either Pentium 1 or newer, or 80386 or newer. Often called "ancient". Personally, I wouldn't bother with this. Nobody is going to run your engine on a CPU 25 years old (or older)... and if they want to, they can compile it themselves.

If you compile for POPCNT (and no PEXT), you'd already cover any CPU that was released in the last 14 years. If you compile for generic x64, you're covering everything back to 2003/2004.

[Guenther]

Yes.
Like Terje said: If you don't have that instruction available then it's not gonna work.
pext is a neat way to get unique index to magic moves. I made Sapeli 1.91 purposely only BMI2. It's the fastest available.

I made some fixes to 1.92 (https://github.com/SamuraiDangyo/Sapeli ... r/Sapeli.c) as my AMD processor doesn't support pext.
So pext is optional. But preferred! Compile with -DPEXT.
If Windows compile with -DWINDOWS.
I don't know about PopCount(). It's well supported.

If you compile for POPCNT (and no PEXT), you'd already cover any CPU that was released in the last 14 years. If you compile for generic x64, you're covering everything back to 2003/2004.

Well, I bought my quadcore new in 2009 and it had no popcount, but it is still running fine, with a few hardware changes ;-)
(new graphics card, more ram, new power supply)

'The price I pay' now with Sapeli 1.92, compiled a few minutes ago seems too high though.
This was compiled now w/o pext and w/o popcount, the older 1.90 version is around 4 times faster here!

Code: Select all

Sapeli 1.90 by Toni Helminen
...
go infinite
info depth 1 nodes 40 time 0 nps 40000 score cp 31 pv e2e4
info depth 2 nodes 264 time 0 nps 264000 score cp 5 pv e2e4
info depth 3 nodes 2638 time 10 nps 239818 score cp 21 pv d2d4
info depth 4 nodes 10568 time 30 nps 340903 score cp 5 pv e2e4
info depth 5 nodes 46388 time 120 nps 383371 score cp 11 pv e2e4
info depth 6 nodes 206797 time 330 nps 624764 score cp 5 pv e2e3
info depth 7 nodes 551460 time 680 nps 809779 score cp 12 pv d2d4
info depth 8 nodes 2592612 time 2490 nps 1040791 score cp 7 pv b1c3
info depth 9 nodes 4704876 time 4280 nps 1099013 score cp 14 pv b1c3
info depth 10 nodes 8662803 time 7590 nps 1141193 score cp 11 pv b1c3
info depth 11 nodes 39035488 time 35140 nps 1110824 score cp 8 pv b1c3

Code: Select all

Sapeli 1.92 by Toni Helminen
...
go infinite
info depth 1 nodes 40 time 0 nps 40000 score cp 29 pv e2e4
info depth 2 nodes 264 time 10 nps 24000 score cp 5 pv e2e4
info depth 3 nodes 2537 time 30 nps 81838 score cp 20 pv d2d4
info depth 4 nodes 13442 time 100 nps 133089 score cp 5 pv e2e4
info depth 5 nodes 43040 time 250 nps 171474 score cp 14 pv d2d4
info depth 6 nodes 169805 time 760 nps 223134 score cp 5 pv d2d4
info depth 7 nodes 383004 time 1590 nps 240731 score cp 13 pv d2d4
info depth 8 nodes 1919219 time 7250 nps 264683 score cp 5 pv b1c3
info depth 9 nodes 4233266 time 15394 nps 274976 score cp 9 pv b1c3
info depth 10 nodes 11756140 time 42856 nps 274310 score cp 10 pv b1c3
info depth 11 nodes 59522553 time 229133 nps 259771 score cp 11 pv b1c3

[Terje]

Yes.
Like Terje said: If you don't have that instruction available then it's not gonna work.
pext is a neat way to get unique index to magic moves. I made Sapeli 1.91 purposely only BMI2. It's the fastest available.

I made some fixes to 1.92 (https://github.com/SamuraiDangyo/Sapeli ... r/Sapeli.c) as my AMD processor doesn't support pext.
So pext is optional. But preferred! Compile with -DPEXT.
If Windows compile with -DWINDOWS.
I don't know about PopCount(). It's well supported.

If you compile for POPCNT (and no PEXT), you'd already cover any CPU that was released in the last 14 years. If you compile for generic x64, you're covering everything back to 2003/2004.

Well, I bought my quadcore new in 2009 and it had no popcount, but it is still running fine, with a few hardware changes
(new graphics card, more ram, new power supply)

'The price I pay' now with Sapeli 1.92, compiled a few minutes ago seems too high though.
This was compiled now w/o pext and w/o popcount, the older 1.90 version is around 4 times faster here!

Code: Select all

Sapeli 1.90 by Toni Helminen
...
go infinite
info depth 1 nodes 40 time 0 nps 40000 score cp 31 pv e2e4
info depth 2 nodes 264 time 0 nps 264000 score cp 5 pv e2e4
info depth 3 nodes 2638 time 10 nps 239818 score cp 21 pv d2d4
info depth 4 nodes 10568 time 30 nps 340903 score cp 5 pv e2e4
info depth 5 nodes 46388 time 120 nps 383371 score cp 11 pv e2e4
info depth 6 nodes 206797 time 330 nps 624764 score cp 5 pv e2e3
info depth 7 nodes 551460 time 680 nps 809779 score cp 12 pv d2d4
info depth 8 nodes 2592612 time 2490 nps 1040791 score cp 7 pv b1c3
info depth 9 nodes 4704876 time 4280 nps 1099013 score cp 14 pv b1c3
info depth 10 nodes 8662803 time 7590 nps 1141193 score cp 11 pv b1c3
info depth 11 nodes 39035488 time 35140 nps 1110824 score cp 8 pv b1c3

Code: Select all

Sapeli 1.92 by Toni Helminen
...
go infinite
info depth 1 nodes 40 time 0 nps 40000 score cp 29 pv e2e4
info depth 2 nodes 264 time 10 nps 24000 score cp 5 pv e2e4
info depth 3 nodes 2537 time 30 nps 81838 score cp 20 pv d2d4
info depth 4 nodes 13442 time 100 nps 133089 score cp 5 pv e2e4
info depth 5 nodes 43040 time 250 nps 171474 score cp 14 pv d2d4
info depth 6 nodes 169805 time 760 nps 223134 score cp 5 pv d2d4
info depth 7 nodes 383004 time 1590 nps 240731 score cp 13 pv d2d4
info depth 8 nodes 1919219 time 7250 nps 264683 score cp 5 pv b1c3
info depth 9 nodes 4233266 time 15394 nps 274976 score cp 9 pv b1c3
info depth 10 nodes 11756140 time 42856 nps 274310 score cp 10 pv b1c3
info depth 11 nodes 59522553 time 229133 nps 259771 score cp 11 pv b1c3

I assume the compiler is emitting code to do pext 'by hand' instead of using hardware specifically made for it (as you don't have it), which is very slow.

[JohnWoe]

Yes.
Like Terje said: If you don't have that instruction available then it's not gonna work.
pext is a neat way to get unique index to magic moves. I made Sapeli 1.91 purposely only BMI2. It's the fastest available.

I made some fixes to 1.92 (https://github.com/SamuraiDangyo/Sapeli ... r/Sapeli.c) as my AMD processor doesn't support pext.
So pext is optional. But preferred! Compile with -DPEXT.
If Windows compile with -DWINDOWS.
I don't know about PopCount(). It's well supported.

If you compile for POPCNT (and no PEXT), you'd already cover any CPU that was released in the last 14 years. If you compile for generic x64, you're covering everything back to 2003/2004.

Well, I bought my quadcore new in 2009 and it had no popcount, but it is still running fine, with a few hardware changes
(new graphics card, more ram, new power supply)

'The price I pay' now with Sapeli 1.92, compiled a few minutes ago seems too high though.
This was compiled now w/o pext and w/o popcount, the older 1.90 version is around 4 times faster here!

Code: Select all

Sapeli 1.90 by Toni Helminen
...
go infinite
info depth 1 nodes 40 time 0 nps 40000 score cp 31 pv e2e4
info depth 2 nodes 264 time 0 nps 264000 score cp 5 pv e2e4
info depth 3 nodes 2638 time 10 nps 239818 score cp 21 pv d2d4
info depth 4 nodes 10568 time 30 nps 340903 score cp 5 pv e2e4
info depth 5 nodes 46388 time 120 nps 383371 score cp 11 pv e2e4
info depth 6 nodes 206797 time 330 nps 624764 score cp 5 pv e2e3
info depth 7 nodes 551460 time 680 nps 809779 score cp 12 pv d2d4
info depth 8 nodes 2592612 time 2490 nps 1040791 score cp 7 pv b1c3
info depth 9 nodes 4704876 time 4280 nps 1099013 score cp 14 pv b1c3
info depth 10 nodes 8662803 time 7590 nps 1141193 score cp 11 pv b1c3
info depth 11 nodes 39035488 time 35140 nps 1110824 score cp 8 pv b1c3

Code: Select all

Sapeli 1.92 by Toni Helminen
...
go infinite
info depth 1 nodes 40 time 0 nps 40000 score cp 29 pv e2e4
info depth 2 nodes 264 time 10 nps 24000 score cp 5 pv e2e4
info depth 3 nodes 2537 time 30 nps 81838 score cp 20 pv d2d4
info depth 4 nodes 13442 time 100 nps 133089 score cp 5 pv e2e4
info depth 5 nodes 43040 time 250 nps 171474 score cp 14 pv d2d4
info depth 6 nodes 169805 time 760 nps 223134 score cp 5 pv d2d4
info depth 7 nodes 383004 time 1590 nps 240731 score cp 13 pv d2d4
info depth 8 nodes 1919219 time 7250 nps 264683 score cp 5 pv b1c3
info depth 9 nodes 4233266 time 15394 nps 274976 score cp 9 pv b1c3
info depth 10 nodes 11756140 time 42856 nps 274310 score cp 10 pv b1c3
info depth 11 nodes 59522553 time 229133 nps 259771 score cp 11 pv b1c3

Thanks for testing.
On Linux > sudo cat /proc/cpuinfo | grep bmi : Should tell if your CPU has bmi2 instructions.

I added MODERN flag because popcnt/ctz are both ABM sets so they are both available or not.
Without modern the compile will be really slow. But I wanted Sapeli to work on any Linux machine.

Sapeli 1.90:

Code: Select all

...
info depth 7 nodes 551460 time 338 nps 1626725 score cp 12 pv d2d4
info depth 8 nodes 2592612 time 1100 nps 2354779 score cp 7 pv b1c3
info depth 9 nodes 4704876 time 1879 nps 2502593 score cp 14 pv b1c3
info depth 10 nodes 8662803 time 3304 nps 2621120 score cp 11 pv b1c3
info depth 11 nodes 39035488 time 14828 nps 2632374 score cp 8 pv b1c3

Sapeli 1.92:

Code: Select all

...
info depth 7 nodes 383011 time 256 nps 1490315 score cp 13 pv d2d4
info depth 8 nodes 1919226 time 791 nps 2423265 score cp 5 pv b1c3
info depth 9 nodes 4233276 time 1526 nps 2772282 score cp 9 pv b1c3
info depth 10 nodes 11756024 time 4003 nps 2936069 score cp 10 pv b1c3
info depth 11 nodes 59571526 time 20817 nps 2861539 score cp 11 pv b1c3

Sapeli No modern/pext

Code: Select all

....
info depth 8 nodes 1919105 time 2594 nps 739539 score cp 5 pv b1c3
info depth 9 nodes 4233513 time 5406 nps 782968 score cp 9 pv b1c3
info depth 10 nodes 11757964 time 15188 nps 774110 score cp 10 pv b1c3

So 2.9 Mnps vs 0.8 Mnps. A massive difference.

I released Sapeli 1.92: https://github.com/SamuraiDangyo/Sapeli ... /tag/v1.92
On lichess: https://lichess.org/@/SapeliEngine

This game hurts: https://lichess.org/oPwy24gk

Sapeli missed an easy win because I removed KPK tables to shrink code

[Gabor Szots]

I released Sapeli 1.92: https://github.com/SamuraiDangyo/Sapeli ... /tag/v1.92

But you already had a Sapeli 1.92 a week ago. I can see the source is a bit different (in size), does it play the same moves? Would be useful to know as I have already submitted my games for our list and I would not want a mixup in version numbers.

[JohnWoe]

I released Sapeli 1.92: https://github.com/SamuraiDangyo/Sapeli ... /tag/v1.92

But you already had a Sapeli 1.92 a week ago. I can see the source is a bit different (in size), does it play the same moves? Would be useful to know as I have already submitted my games for our list and I would not want a mixup in version numbers.

I released v1.91 a few weeks ago. If it works then there's no big difference to 1.92. 1.91 is BMI2 only version.

1.92 plays a bit more passively It used to bring his queen out too early etc. I tested against 1.90 and 1.92 was much better. I released 1.92 since Guenther started testing it. No big deal. This new 1.93 contains a few speedup tricks.

I'm still playing with prefetch to get more speedups. But couldn't measure any effect. It's asynchronous I like it. But on Linux it's considered harmful?