Compiling glaurung, stockfish, smaug

[sje]

Yes, I think GCC (at least until version 4.2) is one of those compilers which can't inline function calls across module boundaries.

My way of thinking about this kind of problems is that the 10-20% performance penalty (of course the numbers could be different for my program, but I'm too lazy to test) is a weakness of GCC, and not of my program. It's therefore the GCC developers' responsibility to fix it, and not mine, and I'm not willing to do any strange hacks to work around it.

Why not explicitly declare inline functions and have them appear in header files? The g++ option -finline-limit is useful here for tuning purposes.

[bob]

Yes, I think GCC (at least until version 4.2) is one of those compilers which can't inline function calls across module boundaries.

My way of thinking about this kind of problems is that the 10-20% performance penalty (of course the numbers could be different for my program, but I'm too lazy to test) is a weakness of GCC, and not of my program. It's therefore the GCC developers' responsibility to fix it, and not mine, and I'm not willing to do any strange hacks to work around it.

Why not explicitly declare inline functions and have them appear in header files? The g++ option -finline-limit is useful here for tuning purposes.

This can be a bit risky in terms of performance. Now you force it to inline the things, where normally it can make a rational decision on the tradeoff of speed vs space and not make the program executable bloat to the point it blows cache out...

I use the "crafty.c" approach which has the added advantage that it works for _all_compilers, even though I do all my work using Intel's...

[mcostalba]

I use the "crafty.c" approach which has the added advantage that it works for _all_compilers, even though I do all my work using Intel's...

If you code taking in account this way of compile then you can risk some clash, but you know it in advance and you'll be careful.

But if an engine is NOT foreseen to be used in that way (and the copious use of anonymous namespaces in Glaurung and friends is a strong hint that it is not) it is very risky to just blindly sum all files up.

Just my opinion

Marco

[Sven]

I use the "crafty.c" approach which has the added advantage that it works for _all_compilers, even though I do all my work using Intel's...

If you code taking in account this way of compile then you can risk some clash, but you know it in advance and you'll be careful.

But if an engine is NOT foreseen to be used in that way (and the copious use of anonymous namespaces in Glaurung and friends is a strong hint that it is not) it is very risky to just blindly sum all files up.

Just my opinion

Marco

I see no risk in doing so. Either it compiles (so you're fine), or it does not (so you fix it).

I also see an advantage of putting all, or at least many, *.c/*.cpp files into one "include all modules" source file. As others have already mentioned, some compilers can optimize better than others when given such a combined source file. My development philosophy includes to prepare my code for different compile environments, not just the one that I currently have installed and use mainly for my own development. Even for closed-source code, it may help in the future to have portable code, at least to some extent, who can tell today? I do not want to anticipate today what I will do with my code next year.

A second reason for having a combined "all modules" file (provided we stay within our nice small computer chess world of <100k lines of code) is that it can greatly simplify things for someone else taking your code and trying to compile it on a different system. It may be a secondary argument but some people, including myself, like small or medium-sized software that can easily be compiled with "g++ -o MYPROGRAM MYPROGRAM_all.cpp" or "cl /FeMYPROGRAM.exe MYPROGRAM_all.cpp" (omitting optimization and such).

Third point: preparing your code to be compiled in larger units may be helpful for reusing that code in another context, whether in another project of your own or in someone else's project (in case of open source). Go and try what happens if you copy some variables and functions from some file in project A in order to reuse them in project B. Everything that is "global" and not scoped somehow may cause headache for you.

Regarding anonymous namespaces, Marco is right that this may result in getting name clashes when compiling several *.cpp files at once. In case of Glaurung and its derivatives, actually there seems to be no problem at the moment, although I would suggest to assign names.

The compile problem of this thread was related to "mersenne.cpp" which is obviously an exception since it is derived from open source code written by someone else, and contains a global macro "N"

So I agree with Marco that the original author (Tord) at least did not intend to have all its "own" cpp sources compiled together with "mersenne.cpp" in one large compile unit. One possible solution would be (disregarding here that moving "mersenne.cpp" behind "tt.cpp" already works - this may cause other problems in theory, there might be no correct order) to exclude "mersenne.cpp" from the "all modules" file and compile it separately.

But I would not call it "risky", either you find a solution or you don't. If it compiles with all files at once then the program will work the same way as it does when compiling modules separately.

Sven

[jwes]

Might it better to only combine those functions that are called in the search, as these are the only ones that will help significantly by being better optimized ?

[michiguel]

The problem is in the previous line, please post some lines before the 188.

I really don't understund the meaning of that <engine_name>.cpp file with all that includes of *.cpp...anyway...the point is...I also don't want to understand

I think it's an ugly "optimization" some people use as a workaround for compilers which are not able to inline function calls across file boundaries. Of course it isn't guaranteed to work, because identifier names in different modules may clash. I suspect that this is the reason for Bernhard's problem, but I also don't have the source code available right now, so I can't check.

Tord

I never tried but GCC can do that with --fwhole-program --combine

Has anybody tried?

Miguel

[Tord Romstad]

I never tried but GCC can do that with --fwhole-program --combine

Only for C (not C++) code, I think, but perhaps this has changed in the most recent versions.

Tord

[mcostalba]

But I would not call it "risky", either you find a solution or you don't. If it compiles with all files at once then the program will work the same way as it does when compiling modules separately.

This could be true in C, but C++ it is a completly diffrent beast!

Here (one of) the problem is function overloading:

void myFunc(long);
void myFunc(int);

now

myFunc(7); // what will be the called function?

It is perfectly legal to define BOTH functions, compiler even does not warns, so if the two overloads of myFunc are defined in two different anonymous namespaces then a call to myFunc() will work by accident if you mean calling the 'int' one and will silently break if you want to call the 'long' one.

And this is only a very basic example. Name lookup in C++ is a VERY complex subject at least an order of magnitude bigger then in C and there are also some really esotic cases (not in the above example of course). So there is a lot of stuff that could silently break.

There is a reason why people invented namespaces

[wgarvin]

But I would not call it "risky", either you find a solution or you don't. If it compiles with all files at once then the program will work the same way as it does when compiling modules separately.

This could be true in C, but C++ it is a completly diffrent beast!

Here (one of) the problem is function overloading:

void myFunc(long);
void myFunc(int);

now

myFunc(7); // what will be the called function?

It is perfectly legal to define BOTH functions, compiler even does not warns, so if the two overloads of myFunc are defined in two different anonymous namespaces then a call to myFunc() will work by accident if you mean calling the 'int' one and will silently break if you want to call the 'long' one.

And this is only a very basic example. Name lookup in C++ is a VERY complex subject at least an order of magnitude bigger then in C and there are also some really esotic cases (not in the above example of course). So there is a lot of stuff that could silently break.

There is a reason why people invented namespaces

That is all true, but... in practice, you can often get away with it.

Most of the time when there is a name conflict, the resulting code won't compile. If it still compiles, then it may work just fine (for example, if there were colliding function names but the parameter lists gave the compiler enough distinguishing information to pick the right overloads in each case).

Another reason to use this sort of "all in one file" compilation is that with some large projects, you can get significantly faster build times. The tradeoff is having to recompile more stuff whenever you make a change. Also, if you still want to be able to compile modules separately, then you have to be careful not to write code that relies on include files which are included by some other module, but not included by the module in which you wrote your code. Because the "all in one file" will still compile, but your edited source file won't.