Senpai 2.0

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
lucasart
Posts: 3232
Joined: Mon May 31, 2010 1:29 pm
Full name: lucasart

Re: Senpai 2.0

Post by lucasart »

Xann wrote:
lucasart wrote:What I can say regarding compilation is:
* indeed gcc is significantly slower than clang, which is very unusual (normally it's the other way around). I discovered a gcc bug recently, perhaps it's the same. ...
No bug. I think it's LTO, more precisely cross-module inlining.

Here's what happened. During the development of my draughts program Scan, I discovered Clang's -flto. For the first time ever, I could move "critical" code from the include files to where they belong: with related functions. And not lose performance, or very little (at least with Clang/LLVM). Having code in the include files also triggers a lot of useless recompilation during development. By contrast, the cost of LTO is slower linking; something I can live with. During the release, I moved a selected set back to the include files. It took some time in itself, and is not fun at all. I also couldn't do profiling on my machine, so a lot of guesswork was needed. At least, I still remembered which functions used to be there.

But since then, I've been used to not thinking about it anymore. After all it's the compiler's job to inline functions, regardless of their placement in the source code. The Senpai 2 release was already taking a lot of time and, among other things, I skipped the "manual inlining" step.

GCC most likely has vastly inferior inlining ability than LLVM, perhaps due to its design at a time when separate compilation was a must. Thinking about it, the Windows compilers might have similar shortcomings. So it's not a bug, and it's also partly my fault. The question is, do we do something about it?
Totally agree with you regarding inlining. I never use the word "inline" in my code. I simply declare (externally linked) functions in headers, and implement them in source files, even the most trivial functions. And I rely on LTO to make the magic happen (code generating is done while linking, allowing cross module optimization, including inlining). But I use GCC and it works perfectly. Last I checked, my code compiles to a slower executable with Clang compared to GCC.

But then my code is in C, yours is in C++. And with C++ (at least the way you use it) come lots of do nothing wrappers, making performance more sensitive to LTO inlining capacity. Perhaps that's why.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.
User avatar
lucasart
Posts: 3232
Joined: Mon May 31, 2010 1:29 pm
Full name: lucasart

Re: Senpai 2.0

Post by lucasart »

lucasart wrote:
Xann wrote:
lucasart wrote:What I can say regarding compilation is:
* indeed gcc is significantly slower than clang, which is very unusual (normally it's the other way around). I discovered a gcc bug recently, perhaps it's the same. ...
No bug. I think it's LTO, more precisely cross-module inlining.

Here's what happened. During the development of my draughts program Scan, I discovered Clang's -flto. For the first time ever, I could move "critical" code from the include files to where they belong: with related functions. And not lose performance, or very little (at least with Clang/LLVM). Having code in the include files also triggers a lot of useless recompilation during development. By contrast, the cost of LTO is slower linking; something I can live with. During the release, I moved a selected set back to the include files. It took some time in itself, and is not fun at all. I also couldn't do profiling on my machine, so a lot of guesswork was needed. At least, I still remembered which functions used to be there.

But since then, I've been used to not thinking about it anymore. After all it's the compiler's job to inline functions, regardless of their placement in the source code. The Senpai 2 release was already taking a lot of time and, among other things, I skipped the "manual inlining" step.

GCC most likely has vastly inferior inlining ability than LLVM, perhaps due to its design at a time when separate compilation was a must. Thinking about it, the Windows compilers might have similar shortcomings. So it's not a bug, and it's also partly my fault. The question is, do we do something about it?
Totally agree with you regarding inlining. I never use the word "inline" in my code. I simply declare (externally linked) functions in headers, and implement them in source files, even the most trivial functions. And I rely on LTO to make the magic happen (code generating is done while linking, allowing cross module optimization, including inlining). But I use GCC and it works perfectly. Last I checked, my code compiles to a slower executable with Clang compared to GCC.

But then my code is in C, yours is in C++. And with C++ (at least the way you use it) come lots of do nothing wrappers, making performance more sensitive to LTO inlining capacity. Perhaps that's why.
I tried again with a recent Clang, and confirm your finding. Indeed Clang produces faster compiles than GCC (by 4.5%) on my code (which relies heavily on cross-module inlining, making zero use of "static inline" implementation of function in header files).
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.
BrendanJNorman
Posts: 2526
Joined: Mon Feb 08, 2016 12:43 am
Full name: Brendan J Norman

Re: Senpai 2.0

Post by BrendanJNorman »

Frank Quisinsky wrote:Hi Brendan,

and on other legend made the graphics (Wilhelm Hudetz, Austria).

Both, Fabien & John on one site ...
A good day for computer chess in my humble opinion!

Of course, not important that it's the own site!

After a longer time slower downloads on www.amateurschach.de it's solved since yesterday. My provider found & fixed a defect patch cable.

Downloads now with full speed ...

Best
Frank

I had here again a lot of fun with Fabien ...
Fabien is really a very interesting person.

I wish me to have 0.1% from his programming knowledge only and I am an expert. But unfortunately ... maybe 0.001% ...
Indeed, but it seems that you too Frank, as a non-programmer did a lot for CC over the years too.

BTW...any word on a release of Wasp 2.5?
Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: Senpai 2.0 ... BTW: Wasp (other things).

Post by Frank Quisinsky »

Hi Brendan,

yes, I do so many things all the years in helping others oder promotion things. Now I am 50 years old and have more interest on testing 1-2 engines (Wasp and Senpai ... perfect for me), to do other things. Testing Wasp, Senpai in detail if FEOBOS is over. Here I have interest to create a new test-set with problematic positions for both engines. 100% beta test is my idea with many graphics with chessboards on my site ... OK, OK after FEOBOS. Furthermore, the work with Klaus Wlotzka is great ... sure I will start with Klaus a new Project ... we have different ideas but we Need different things the CuteChess programmer must do for us.

I will create a new selection to older chess Computer. A tourney with 6 older interesting chess Computer, myself and Zarkov 2.51 on an 166Mhz Pentium I Notebook. In free time the games with tournament rules can be running.

I think end of the year I have here all complete. The 6 chess Computer will be ...

1. Mephisto Modular MMV with HG550 and 10Mhz
2. Mephisto Modular MMIIa with HG240 and 7.4Mhz
3. Mephisto Modular Magellan (no tuning)
4. Mephisto Modena 8Mhz
5. Mephisto Nigel Short 10Mhz
6. unclear ... I have the Mephisto Modular board and I am searching here g good London 68000 Modul set.

Sure here that MMIIa will be not on the last place.
I am thinking I am older and more and more crazy.
My wish is that Wasp or Senpai can be running on Modular with a modul from Ruud and FEOBOS opening book.

To your question:
John is working on the update.
Possible that Wasp 2.6 or 3.0 (no ideas what he create for a Version number) will be available in the next weeks this year. Playing style is great but John will see more Elo.

So nice what John do in Wasp.
Please have partience ... will be available a bit later (think so).

Best
Frank
User avatar
fern
Posts: 8755
Joined: Sun Feb 26, 2006 4:07 pm

Re: Senpai 2.0

Post by fern »

Comme vous etez, fabien?
Seulement 64 bits? Quel domage.....

Fernand
Peter Berger
Posts: 653
Joined: Thu Mar 09, 2006 2:56 pm

Re: Senpai 2.0

Post by Peter Berger »

I have a problem with Senpai 2.0 on my computer.

I am using the latest free version of Avira Free Antivirus with Windows 10.

Avira insists on identifying Senpai as a virus and it thus gets deleted. I can work around this by deactivating live protection for my computer, but as I don't want to do this permanently, using Senpai becomes quite a pain. Am I the only one?

Peter
Rein Halbersma
Posts: 741
Joined: Tue May 22, 2007 11:13 am

Re: Senpai 2.0

Post by Rein Halbersma »

lucasart wrote: I tried again with a recent Clang, and confirm your finding. Indeed Clang produces faster compiles than GCC (by 4.5%) on my code (which relies heavily on cross-module inlining, making zero use of "static inline" implementation of function in header files).
For my draughts program (template heavy, header-only C++ code), GCC consistently produces 10% faster binaries than Clang. I suspect that Clang is not aggressive enough with inlining all my thin wrappers. I am too lazy to figure out the exact threshold parameter that will trigger Clang to inline as well as GCC.
tttony
Posts: 268
Joined: Sun Apr 24, 2011 12:33 am

Re: Senpai 2.0

Post by tttony »

Cool!

I see that you are not using one source file anymore :D

Hope you keep developing to see more impovement
User avatar
Eelco de Groot
Posts: 4561
Joined: Sun Mar 12, 2006 2:40 am
Full name:   

Re: Senpai 2.0

Post by Eelco de Groot »

Peter Berger wrote:I have a problem with Senpai 2.0 on my computer.

I am using the latest free version of Avira Free Antivirus with Windows 10.

Avira insists on identifying Senpai as a virus and it thus gets deleted. I can work around this by deactivating live protection for my computer, but as I don't want to do this permanently, using Senpai becomes quite a pain. Am I the only one?

Peter
Hi Peter,

I think you'll just have to add Senpai to the 'real time' exceptions, or upload it to Avira with the request to test it, if that is possible.

I found this post here:

Istvan Tamas
October 24, 2017 at 3:01 pm

Hi Elcaset,

Avira Antivirus received a new interface and therefore many things changed. Here we go with the new steps:
Open Avira Antivirus
On the bottom left you’ll find a small gear which will lead you to the settings.
In “Real-time Protection” you’ll find the option to set up your exceptions.

Hope this helps! If you’ve any further question please let us know.

Best,
Istvan
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
Ras
Posts: 2487
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: Senpai 2.0

Post by Ras »

lucasart wrote:I never use the word "inline" in my code.
Fun fact, I'm using the "noinline" attribute way more often than the inline one with GCC. :-)