Senpai 2.0

[Guenther]

Hi all,

After a few years in computer draughts, I am back to chess for a little while. I see too many draws at top level, and am afraid that it will be too late in a couple of years. Big hardware and large opening books are also not helping IMO.

So I'm here to announce Senpai 2.0.

...

Fabien.

Thanks a lot Fabien and all who helped with this release!

Guenther

Oops, now I realized I cannot even use it :(
Any chance for a non popcount build for older 64 bit hardware?

[Xann]

Oops, now I realized I cannot even use it
Any chance for a non popcount build for older 64 bit hardware?

Yes, when someone builds one.

[tomitank]

Thank you very much and Congratulations!

-Tamas

[BrendanJNorman]

Hi all,

After a few years in computer draughts, I am back to chess for a little while. I see too many draws at top level, and am afraid that it will be too late in a couple of years. Big hardware and large opening books are also not helping IMO.

So I'm here to announce Senpai 2.0. The code looks all different because I started from my draughts program, but it's actually still Senpai inside. I prefer having a consistent codebase for multiple games (also Shogi and Othello).

As a rewrite it's hard to list changes, but here is an attempt:
- Chess960 is baked in, not an add-on; Senpai 2 plays chess only as a special case!
- search: restricted singular extensions
- search: additional reduction/pruning of "losing" moves (SEE < 0)
- eval: tempo bonus (it seems I forgot in 1.0 ...)
- eval: "space" (glorified pawn-chain PST); this changes playing style a lot
- eval: scoring by logistic regression; this gives me more freedom for eval features
- board: copy/make (customary in games with fewer piece types than chess)
- board: optional PEXT bitboards (BMI2)

As I recall, no single change brought more than 10 Elo. So it's more a sum of small (IMO) things.

This release has been rushed a little, so it's still lacking UCI options and multi-PV; sorry about that.

Official website: http://www.amateurschach.de/main/_senpai.htm
Direct link: http://www.amateurschach.de/download/senpai_20.zip
Expected level: Texel/Protector (again)

Thanks to everybody who helped with this release:
- Graham Banks
- Michael Byrne
- Wilhelm Hudetz
- Daniel Jose
- Steve Maughan (https://www.chessprogramming.net)
- Frank Quisinsky (http://www.amateurschach.de)
- XXX (<- this line for people I forgot, sorry in advance).

That's all for now,

Fabien.

Here are some early results from my current tournament.

First, a breathtaking win by Rodent III Shirov as he again sets the board on fire.

Senpai 2.0 vs Rodent III Shirov

[pgn][Event "GrandMasters Blitz (SlverSuite)"]
[Site "BRENDANNORMD8A2"]
[Date "2017.11.10"]
[Round "1"]
[White "Senpai 2.0"]
[Black "Rodent III Shirov"]
[Result "0-1"]
[ECO "D05"]
[WhiteElo "2200"]
[BlackElo "2200"]
[PlyCount "92"]
[EventDate "2017.??.??"]

1. Nf3 Nf6 2. d4 d5 3. e3 e6 4. c3 c5 5. Bd3 Bd6 6. dxc5 Bxc5 7. O-O O-O 8.
Nbd2 Qc7 9. e4 Nc6 10. h3 Bd7 11. Re1 Qg3 12. Qe2 Ne5 13. Kh1 Bxf2 14. Nf1 Nxf3
15. Nxg3 Nxe1 16. e5 Nxd3 17. exf6 Bxg3 18. Qxd3 Be5 19. fxg7 Bxg7 20. Qg3 Kh8
21. Be3 e5 22. Qh4 Bc6 23. Bc5 Rg8 24. Qh5 d4 25. Qxf7 d3 26. Kg1 Rgd8 27. Rf1
e4 28. Qf4 Rd5 29. Bxa7 Rf8 30. Qc7 d2 31. Rd1 Rd3 32. Bc5 Re8 33. Kf2 e3+ 34.
Ke2 Rxc3 35. Qxc6 bxc6 36. bxc3 Bxc3 37. Bb6 Bb4 38. a4 c5 39. Rb1 Re6 40. Bc7
Bc3 41. a5 c4 42. Bb6 Bf6 43. Bxe3 Bd4 44. Kxd2 Rxe3 45. Rb8+ Kg7 46. Rb7+ Kf6
0-1[/pgn]

and second, a smooth win by Senpai vs Deep Gandalf 7.

The game essentially goes like this...

1. Senpai gets a typical Catalan space advantage
2. Senpai provokes a weakness on the black kingside (dark squares)
3. Senpai opens the h-file via h2-h4-h5xg6
4. Senpai plays a very human-like Kg2/Rh1 idea
5. Senpai breaks through on the h-file.

Great to have another strong human-like engine around 2900-3000.

A match against Gaviota or Wasp will be very interesting soon for me.

Senpai 2.0 vs Deep Gandalf 7

[pgn][Event "GrandMasters Blitz (SlverSuite)"]
[Site "BRENDANNORMD8A2"]
[Date "2017.11.10"]
[Round "1"]
[White "Senpai 2.0"]
[Black "Deep Gandalf 7"]
[Result "1-0"]
[ECO "E09"]
[WhiteElo "2200"]
[BlackElo "2200"]
[PlyCount "81"]
[EventDate "2017.??.??"]

1. d4 Nf6 2. c4 e6 3. Nf3 d5 4. g3 Be7 5. Bg2 O-O 6. O-O c6 7. Qc2 Nbd7 8. Nbd2
b5 9. c5 a5 10. e4 dxe4 11. Nxe4 Nxe4 12. Qxe4 Bb7 13. Qc2 Nf6 14. Bd2 a4 15.
a3 Qc8 16. Rfe1 Rd8 17. Rad1 Qd7 18. Ne5 Qe8 19. Bg5 Nd5 20. Bxe7 Qxe7 21. Be4
g6 22. Qd3 Kg7 23. Qd2 Rab8 24. Bf3 Kg8 25. Ng4 Kh8 26. h4 Rbc8 27. Ne5 Kg8 28.
h5 Qf6 29. hxg6 hxg6 30. Kg2 Qg7 31. Rh1 Ne7 32. Qf4 Nf5 33. Rh2 g5 34. Qe4 Rc7
35. Rh5 Kf8 36. Bg4 Rd5 37. Bxf5 exf5 38. Qe2 Re7 39. Rdh1 Rexe5 40. dxe5 Qxe5
41. Rh8+ 1-0[/pgn]

Thanks again Fabian, a very interesting engine you've given us.

[Marek Soszynski]

Oops, now I realized I cannot even use it :(
Any chance for a non popcount build for older 64 bit hardware?

Yes, when someone builds one.

Thanks for the engine, Fabien. I require a non-PC compile too.

[carldaman]

Thanks, Fabien. A pleasant surprise!

Cheers,
CL

[Xann]

What I can say regarding compilation is:
* indeed gcc is significantly slower than clang, which is very unusual (normally it's the other way around). I discovered a gcc bug recently, perhaps it's the same. ...

No bug. I think it's LTO, more precisely cross-module inlining.

Here's what happened. During the development of my draughts program Scan, I discovered Clang's -flto. For the first time ever, I could move "critical" code from the include files to where they belong: with related functions. And not lose performance, or very little (at least with Clang/LLVM). Having code in the include files also triggers a lot of useless recompilation during development. By contrast, the cost of LTO is slower linking; something I can live with. During the release, I moved a selected set back to the include files. It took some time in itself, and is not fun at all. I also couldn't do profiling on my machine, so a lot of guesswork was needed. At least, I still remembered which functions used to be there.

But since then, I've been used to not thinking about it anymore. After all it's the compiler's job to inline functions, regardless of their placement in the source code. The Senpai 2 release was already taking a lot of time and, among other things, I skipped the "manual inlining" step.

GCC most likely has vastly inferior inlining ability than LLVM, perhaps due to its design at a time when separate compilation was a must. Thinking about it, the Windows compilers might have similar shortcomings. So it's not a bug, and it's also partly my fault. The question is, do we do something about it?

[lucasart]

What I can say regarding compilation is:
* indeed gcc is significantly slower than clang, which is very unusual (normally it's the other way around). I discovered a gcc bug recently, perhaps it's the same. ...

No bug. I think it's LTO, more precisely cross-module inlining.

Here's what happened. During the development of my draughts program Scan, I discovered Clang's -flto. For the first time ever, I could move "critical" code from the include files to where they belong: with related functions. And not lose performance, or very little (at least with Clang/LLVM). Having code in the include files also triggers a lot of useless recompilation during development. By contrast, the cost of LTO is slower linking; something I can live with. During the release, I moved a selected set back to the include files. It took some time in itself, and is not fun at all. I also couldn't do profiling on my machine, so a lot of guesswork was needed. At least, I still remembered which functions used to be there.

But since then, I've been used to not thinking about it anymore. After all it's the compiler's job to inline functions, regardless of their placement in the source code. The Senpai 2 release was already taking a lot of time and, among other things, I skipped the "manual inlining" step.

GCC most likely has vastly inferior inlining ability than LLVM, perhaps due to its design at a time when separate compilation was a must. Thinking about it, the Windows compilers might have similar shortcomings. So it's not a bug, and it's also partly my fault. The question is, do we do something about it?

Totally agree with you regarding inlining. I never use the word "inline" in my code. I simply declare (externally linked) functions in headers, and implement them in source files, even the most trivial functions. And I rely on LTO to make the magic happen (code generating is done while linking, allowing cross module optimization, including inlining). But I use GCC and it works perfectly. Last I checked, my code compiles to a slower executable with Clang compared to GCC.

But then my code is in C, yours is in C++. And with C++ (at least the way you use it) come lots of do nothing wrappers, making performance more sensitive to LTO inlining capacity. Perhaps that's why.

[lucasart]

What I can say regarding compilation is:
* indeed gcc is significantly slower than clang, which is very unusual (normally it's the other way around). I discovered a gcc bug recently, perhaps it's the same. ...

No bug. I think it's LTO, more precisely cross-module inlining.

Here's what happened. During the development of my draughts program Scan, I discovered Clang's -flto. For the first time ever, I could move "critical" code from the include files to where they belong: with related functions. And not lose performance, or very little (at least with Clang/LLVM). Having code in the include files also triggers a lot of useless recompilation during development. By contrast, the cost of LTO is slower linking; something I can live with. During the release, I moved a selected set back to the include files. It took some time in itself, and is not fun at all. I also couldn't do profiling on my machine, so a lot of guesswork was needed. At least, I still remembered which functions used to be there.

But since then, I've been used to not thinking about it anymore. After all it's the compiler's job to inline functions, regardless of their placement in the source code. The Senpai 2 release was already taking a lot of time and, among other things, I skipped the "manual inlining" step.

GCC most likely has vastly inferior inlining ability than LLVM, perhaps due to its design at a time when separate compilation was a must. Thinking about it, the Windows compilers might have similar shortcomings. So it's not a bug, and it's also partly my fault. The question is, do we do something about it?

Totally agree with you regarding inlining. I never use the word "inline" in my code. I simply declare (externally linked) functions in headers, and implement them in source files, even the most trivial functions. And I rely on LTO to make the magic happen (code generating is done while linking, allowing cross module optimization, including inlining). But I use GCC and it works perfectly. Last I checked, my code compiles to a slower executable with Clang compared to GCC.

But then my code is in C, yours is in C++. And with C++ (at least the way you use it) come lots of do nothing wrappers, making performance more sensitive to LTO inlining capacity. Perhaps that's why.

I tried again with a recent Clang, and confirm your finding. Indeed Clang produces faster compiles than GCC (by 4.5%) on my code (which relies heavily on cross-module inlining, making zero use of "static inline" implementation of function in header files).

[BrendanJNorman]

Hi Brendan,

and on other legend made the graphics (Wilhelm Hudetz, Austria).

Both, Fabien & John on one site ...
A good day for computer chess in my humble opinion!

Of course, not important that it's the own site!

After a longer time slower downloads on www.amateurschach.de it's solved since yesterday. My provider found & fixed a defect patch cable.

Downloads now with full speed ...

Best
Frank

I had here again a lot of fun with Fabien ...
Fabien is really a very interesting person.

I wish me to have 0.1% from his programming knowledge only and I am an expert. But unfortunately ... maybe 0.001% ...

Indeed, but it seems that you too Frank, as a non-programmer did a lot for CC over the years too.

BTW...any word on a release of Wasp 2.5?