Senpai 2.0

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

carldaman
Posts: 1541
Joined: Sat Jun 02, 2012 12:13 am

Re: Senpai 2.0

Post by carldaman » Fri Nov 10, 2017 10:13 pm

Thanks, Fabien. A pleasant surprise! :D

Cheers,
CL

Xann
Posts: 125
Joined: Sat Jan 22, 2011 6:14 pm
Location: Lille, France

Re: Senpai 2.0

Post by Xann » Sat Nov 11, 2017 3:58 am

lucasart wrote:What I can say regarding compilation is:
* indeed gcc is significantly slower than clang, which is very unusual (normally it's the other way around). I discovered a gcc bug recently, perhaps it's the same. ...
No bug. I think it's LTO, more precisely cross-module inlining.

Here's what happened. During the development of my draughts program Scan, I discovered Clang's -flto. For the first time ever, I could move "critical" code from the include files to where they belong: with related functions. And not lose performance, or very little (at least with Clang/LLVM). Having code in the include files also triggers a lot of useless recompilation during development. By contrast, the cost of LTO is slower linking; something I can live with. During the release, I moved a selected set back to the include files. It took some time in itself, and is not fun at all. I also couldn't do profiling on my machine, so a lot of guesswork was needed. At least, I still remembered which functions used to be there.

But since then, I've been used to not thinking about it anymore. After all it's the compiler's job to inline functions, regardless of their placement in the source code. The Senpai 2 release was already taking a lot of time and, among other things, I skipped the "manual inlining" step.

GCC most likely has vastly inferior inlining ability than LLVM, perhaps due to its design at a time when separate compilation was a must. Thinking about it, the Windows compilers might have similar shortcomings. So it's not a bug, and it's also partly my fault. The question is, do we do something about it?

User avatar
lucasart
Posts: 2957
Joined: Mon May 31, 2010 11:29 am
Contact:

Re: Senpai 2.0

Post by lucasart » Sat Nov 11, 2017 6:37 am

Xann wrote:
lucasart wrote:What I can say regarding compilation is:
* indeed gcc is significantly slower than clang, which is very unusual (normally it's the other way around). I discovered a gcc bug recently, perhaps it's the same. ...
No bug. I think it's LTO, more precisely cross-module inlining.

Here's what happened. During the development of my draughts program Scan, I discovered Clang's -flto. For the first time ever, I could move "critical" code from the include files to where they belong: with related functions. And not lose performance, or very little (at least with Clang/LLVM). Having code in the include files also triggers a lot of useless recompilation during development. By contrast, the cost of LTO is slower linking; something I can live with. During the release, I moved a selected set back to the include files. It took some time in itself, and is not fun at all. I also couldn't do profiling on my machine, so a lot of guesswork was needed. At least, I still remembered which functions used to be there.

But since then, I've been used to not thinking about it anymore. After all it's the compiler's job to inline functions, regardless of their placement in the source code. The Senpai 2 release was already taking a lot of time and, among other things, I skipped the "manual inlining" step.

GCC most likely has vastly inferior inlining ability than LLVM, perhaps due to its design at a time when separate compilation was a must. Thinking about it, the Windows compilers might have similar shortcomings. So it's not a bug, and it's also partly my fault. The question is, do we do something about it?
Totally agree with you regarding inlining. I never use the word "inline" in my code. I simply declare (externally linked) functions in headers, and implement them in source files, even the most trivial functions. And I rely on LTO to make the magic happen (code generating is done while linking, allowing cross module optimization, including inlining). But I use GCC and it works perfectly. Last I checked, my code compiles to a slower executable with Clang compared to GCC.

But then my code is in C, yours is in C++. And with C++ (at least the way you use it) come lots of do nothing wrappers, making performance more sensitive to LTO inlining capacity. Perhaps that's why.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

User avatar
lucasart
Posts: 2957
Joined: Mon May 31, 2010 11:29 am
Contact:

Re: Senpai 2.0

Post by lucasart » Sat Nov 11, 2017 12:53 pm

lucasart wrote:
Xann wrote:
lucasart wrote:What I can say regarding compilation is:
* indeed gcc is significantly slower than clang, which is very unusual (normally it's the other way around). I discovered a gcc bug recently, perhaps it's the same. ...
No bug. I think it's LTO, more precisely cross-module inlining.

Here's what happened. During the development of my draughts program Scan, I discovered Clang's -flto. For the first time ever, I could move "critical" code from the include files to where they belong: with related functions. And not lose performance, or very little (at least with Clang/LLVM). Having code in the include files also triggers a lot of useless recompilation during development. By contrast, the cost of LTO is slower linking; something I can live with. During the release, I moved a selected set back to the include files. It took some time in itself, and is not fun at all. I also couldn't do profiling on my machine, so a lot of guesswork was needed. At least, I still remembered which functions used to be there.

But since then, I've been used to not thinking about it anymore. After all it's the compiler's job to inline functions, regardless of their placement in the source code. The Senpai 2 release was already taking a lot of time and, among other things, I skipped the "manual inlining" step.

GCC most likely has vastly inferior inlining ability than LLVM, perhaps due to its design at a time when separate compilation was a must. Thinking about it, the Windows compilers might have similar shortcomings. So it's not a bug, and it's also partly my fault. The question is, do we do something about it?
Totally agree with you regarding inlining. I never use the word "inline" in my code. I simply declare (externally linked) functions in headers, and implement them in source files, even the most trivial functions. And I rely on LTO to make the magic happen (code generating is done while linking, allowing cross module optimization, including inlining). But I use GCC and it works perfectly. Last I checked, my code compiles to a slower executable with Clang compared to GCC.

But then my code is in C, yours is in C++. And with C++ (at least the way you use it) come lots of do nothing wrappers, making performance more sensitive to LTO inlining capacity. Perhaps that's why.
I tried again with a recent Clang, and confirm your finding. Indeed Clang produces faster compiles than GCC (by 4.5%) on my code (which relies heavily on cross-module inlining, making zero use of "static inline" implementation of function in header files).
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

BrendanJNorman
Posts: 961
Joined: Sun Feb 07, 2016 11:43 pm

Re: Senpai 2.0

Post by BrendanJNorman » Sat Nov 11, 2017 3:58 pm

Frank Quisinsky wrote:Hi Brendan,

and on other legend made the graphics (Wilhelm Hudetz, Austria).

Both, Fabien & John on one site ...
A good day for computer chess in my humble opinion!

Of course, not important that it's the own site!

After a longer time slower downloads on www.amateurschach.de it's solved since yesterday. My provider found & fixed a defect patch cable.

Downloads now with full speed ...

Best
Frank

I had here again a lot of fun with Fabien ...
Fabien is really a very interesting person.

I wish me to have 0.1% from his programming knowledge only and I am an expert. But unfortunately ... maybe 0.001% ...
Indeed, but it seems that you too Frank, as a non-programmer did a lot for CC over the years too.

BTW...any word on a release of Wasp 2.5?
Check my site for engine reviews and other chess stuff :)

www.chessncognac.com

Frank Quisinsky
Posts: 4819
Joined: Wed Nov 18, 2009 6:16 pm
Location: Trier, Germany
Contact:

Re: Senpai 2.0 ... BTW: Wasp (other things).

Post by Frank Quisinsky » Sat Nov 11, 2017 5:23 pm

Hi Brendan,

yes, I do so many things all the years in helping others oder promotion things. Now I am 50 years old and have more interest on testing 1-2 engines (Wasp and Senpai ... perfect for me), to do other things. Testing Wasp, Senpai in detail if FEOBOS is over. Here I have interest to create a new test-set with problematic positions for both engines. 100% beta test is my idea with many graphics with chessboards on my site ... OK, OK after FEOBOS. Furthermore, the work with Klaus Wlotzka is great ... sure I will start with Klaus a new Project ... we have different ideas but we Need different things the CuteChess programmer must do for us.

I will create a new selection to older chess Computer. A tourney with 6 older interesting chess Computer, myself and Zarkov 2.51 on an 166Mhz Pentium I Notebook. In free time the games with tournament rules can be running.

I think end of the year I have here all complete. The 6 chess Computer will be ...

1. Mephisto Modular MMV with HG550 and 10Mhz
2. Mephisto Modular MMIIa with HG240 and 7.4Mhz
3. Mephisto Modular Magellan (no tuning)
4. Mephisto Modena 8Mhz
5. Mephisto Nigel Short 10Mhz
6. unclear ... I have the Mephisto Modular board and I am searching here g good London 68000 Modul set.

Sure here that MMIIa will be not on the last place.
I am thinking I am older and more and more crazy.
My wish is that Wasp or Senpai can be running on Modular with a modul from Ruud and FEOBOS opening book.

To your question:
John is working on the update.
Possible that Wasp 2.6 or 3.0 (no ideas what he create for a Version number) will be available in the next weeks this year. Playing style is great but John will see more Elo.

So nice what John do in Wasp.
Please have partience ... will be available a bit later (think so).

Best
Frank
I like computer chess!

User avatar
fern
Posts: 8745
Joined: Sun Feb 26, 2006 3:07 pm

Re: Senpai 2.0

Post by fern » Sat Nov 11, 2017 5:51 pm

Comme vous etez, fabien?
Seulement 64 bits? Quel domage.....

Fernand

Peter Berger
Posts: 311
Joined: Thu Mar 09, 2006 1:56 pm

Re: Senpai 2.0

Post by Peter Berger » Mon Nov 13, 2017 10:23 pm

I have a problem with Senpai 2.0 on my computer.

I am using the latest free version of Avira Free Antivirus with Windows 10.

Avira insists on identifying Senpai as a virus and it thus gets deleted. I can work around this by deactivating live protection for my computer, but as I don't want to do this permanently, using Senpai becomes quite a pain. Am I the only one?

Peter

Rein Halbersma
Posts: 672
Joined: Tue May 22, 2007 9:13 am

Re: Senpai 2.0

Post by Rein Halbersma » Mon Nov 13, 2017 10:40 pm

lucasart wrote: I tried again with a recent Clang, and confirm your finding. Indeed Clang produces faster compiles than GCC (by 4.5%) on my code (which relies heavily on cross-module inlining, making zero use of "static inline" implementation of function in header files).
For my draughts program (template heavy, header-only C++ code), GCC consistently produces 10% faster binaries than Clang. I suspect that Clang is not aggressive enough with inlining all my thin wrappers. I am too lazy to figure out the exact threshold parameter that will trigger Clang to inline as well as GCC.

tttony
Posts: 254
Joined: Sat Apr 23, 2011 10:33 pm
Contact:

Re: Senpai 2.0

Post by tttony » Mon Nov 13, 2017 11:44 pm

Cool!

I see that you are not using one source file anymore :D

Hope you keep developing to see more impovement

Post Reply