Using LLVM over GCC

[matthewlai]

Just wondering, has anyone else been using LLVM?

I have been developing on OSX, and want my engine to be compilable with both GCC and LLVM, and it's pretty easy.

I find that LLVM's compilation speed is MUCH faster than GCC (~2.5x), and the resulting binary (my engine) is also slightly faster (both -O3 -march=native). With my engine, LLVM is even faster than ICC. Though to be fair, that could be because I did most of my testing with LLVM, so I could have optimized for it. That said, it's general consensus that LLVM produces faster binaries than GCC, and compiles a lot faster.

LLVM also had better C++11 support than GCC 4.8. For example, GCC 4.8 libstdc++'s <regex> is completely broken even though it is included. Even simplest examples will give wrong results. It took me quite a while to figure out, and I am very disappointed in libstdc++'s decision to ship something so broken instead of just not including it, or at least add "#warn <regex> support is horribly broken, use at your own risk" or something to that effect.

Yes, the release note does say <regex> is broken. But who reads release notes for every single compiler update? A lot of times I don't even KNOW my compiler has been updated by APT when I do a system-wide update. No warnings at all when it's used. Really?

I also independently discovered another libstdc++ bug a while ago that was also very hard to narrow down. More than a year old now. Problem is simple and well understood. Clean fix available and patch submitted. Still not fixed after many months of sitting there.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58038

When I first learned to program I learned to always assume bugs are in my code instead of the compiler/standard library. It has served me well for a good few years, but libstdc++ is now constantly challenging that assumption.

At this point I am thinking about switching to LLVM/libc++ on Linux as well. What do you think?

[lucasart]

Your mileage may vary. But in my experience, DiscoCheck and Stockfish are still faster when compiled with GCC. At least on Linux (Lubuntu 13.10). What GCC are you using? The real one? Or the Apple version?

One of the main issues ot Clang is that you can't use link time optimization on Debian/Ubuntu, because it's mis-packaged. And the packagers don't seem to care, as the issue has been going on for years and is still not fixed.

That being said, the difference is small, and once the link time optimization issue is fixed (another 10 years at this rate?) Clang/LLVM will be a good option on Linux. I'm sure Clang/LLVM will beat GCC in the long run, but not yet on my configuration.

[matthewlai]

Your mileage may vary. But in my experience, DiscoCheck and Stockfish are still faster when compiled with GCC. At least on Linux (Lubuntu 13.10). What GCC are you using? The real one? Or the Apple version?

One of the main issues ot Clang is that you can't use link time optimization on Debian/Ubuntu, because it's mis-packaged. And the packagers don't seem to care, as the issue has been going on for years and is still not fixed.

That being said, the difference is very small, and once the link time optimization issue is fixed (another 10 years at this rate?) Clang/LLVM will be a good option on Linux. I'm sure Clang/LLVM will beat GCC in the long run, but not yet on my configuration.

Apple doesn't supply GCC anymore. I am using standard GCC 4.9 from Homebrew (one of the third party repositories for OSX).

Both of the libstdc++ bugs were discovered on Linux though. Standard 4.8.

Yeah if LTO doesn't work it's going to be slow. It does work on OSX, though (which is not surprising since LLVM is OSX's official toolchain now).

[hgm]

I heard there is some very weird behavior of Clang w.r.t. inlined functions. XBoard had two funcitions dedined as 'inline', and used only in the same file as they were defined. This then produces linker errors, because Clang does use real calls rather than inlining, but does not provide any real code to be called because it assumes they will be inlined...

I could cure this weirdnes by declaring them 'static inline'.

[matthewlai]

I heard there is some very weird behavior of Clang w.r.t. inlined functions. XBoard had two funcitions dedined as 'inline', and used only in the same file as they were defined. This then produces linker errors, because Clang does use real calls rather than inlining, but does not provide any real code to be called because it assumes they will be inlined...

I could cure this weirdnes by declaring them 'static inline'.

Hmm that's strange. I have a bunch of inline functions only used in the same file, too, and they compile fine.

[stegemma]

In my limited experience, i'm finally decided to use the native compiler for any Platform. In Windows 64 bit, installing and configuring Eclipse/gcc was a pain (for me) and i haven't get a stable and usable environment. Debugging multi-threaded c++ application was almost impossible. I know that someone could hate it but Visual C++ is the best compiler that i've found, for Windows 64 bit. I come from Borland C++ Builder, so i'm not a MS fan but this seems to me to be the truth. I've tried gcc with Eclipse IDE, code::blocks and CodeLite, on Windows, with different obstacle to get a fast and really professional environment.

On Linux, Eclipse/gcc instead seems to works fine but i have done limited testing.

Same lot of problems to install and configure Eclipse/gcc on Mac OS X (for me) but it works. Still, xCode on Mac OS X appears to be the best choice.

I think that Windows porting of Eclipse/gcc environment (with MingW) is not a stable solution, if you have to handle professional software. This ide/compiler are Linux native and should be used only in Linux (IMHO, of course).

I've seen only this morning the Clang site and it seems a good compiler, i haven't tryed yet maybe i'll take a look on it.

The problem is that today, in 2014, we can't use a good compiler not fully integrated in a good ide, in _OUR_ operating system. Is not important if the operating system itself is not good: the whole environment should works well, fast and with no pain to configure all that we need.

In Satana development i've loose a lot of time only to try making Eclipse/gcc works in a decent way... this is not acceptable, for me.

[matthewlai]

In my limited experience, i'm finally decided to use the native compiler for any Platform. In Windows 64 bit, installing and configuring Eclipse/gcc was a pain (for me) and i haven't get a stable and usable environment. Debugging multi-threaded c++ application was almost impossible. I know that someone could hate it but Visual C++ is the best compiler that i've found, for Windows 64 bit. I come from Borland C++ Builder, so i'm not a MS fan but this seems to me to be the truth. I've tried gcc with Eclipse IDE, code::blocks and CodeLite, on Windows, with different obstacle to get a fast and really professional environment.

On Linux, Eclipse/gcc instead seems to works fine but i have done limited testing.

Same lot of problems to install and configure Eclipse/gcc on Mac OS X (for me) but it works. Still, xCode on Mac OS X appears to be the best choice.

I think that Windows porting of Eclipse/gcc environment (with MingW) is not a stable solution, if you have to handle professional software. This ide/compiler are Linux native and should be used only in Linux (IMHO, of course).

I've seen only this morning the Clang site and it seems a good compiler, i haven't tryed yet maybe i'll take a look on it.

The problem is that today, in 2014, we can't use a good compiler not fully integrated in a good ide, in _OUR_ operating system. Is not important if the operating system itself is not good: the whole environment should works well, fast and with no pain to configure all that we need.

In Satana development i've loose a lot of time only to try making Eclipse/gcc works in a decent way... this is not acceptable, for me.

That is the reason why I never do any portable development/debugging on Windows.

I like to use the same toolchain on all platforms because in reality, supporting multiple compilers does take some work, even if you try very hard to be totally standard-compliant. GCC (and hopefully later LLVM) is a natural target since it's available on just about all platforms under the sun.

The problem with MSVC is of course that it's not available on anything other than Windows.

GNU toolchain on Windows is always a little glitchy, so I do all my development on Linux or (now) OSX, and just switch to Windows to make a build when I need to. I can probably even set up a mingw cross-compiling toolchain to never have to boot into Windows, but I never got around to that.

That way, I don't have to deal with all the toolchain stuff on Windows, as long as calling g++ works, and I can still make Windows builds.

[jdart]

I build with LLVM on MacOS because it is the platform compiler there.

I also support compilation under: gcc, Visual C++, clang, Intel cc, and mingw, although the last 3 of these are not thoroughly tested. You will get different warnings/errors from the various compilers so at least for that reason it is beneficial to try multiple ones.

Personally I have not seen a significant runtime speed advantage from LLVM.

I think gcc is pretty good now. They have made really significant architectural changes over the past decade or so.

--Jon

[flok]

Just wondering, has anyone else been using LLVM?
I find that LLVM's compilation speed is MUCH faster than GCC (~2.5x), and the resulting binary (my engine) is also slightly faster (both -O3 -march=native). With my engine, LLVM is even faster than ICC.

Code: Select all

Rank Name                                                   Elo    +    - games score oppo. draws 
   1 Fairy-Max 4.8Q                                         529  207  124    72   99%   -84    1% 
   2 QueenBee -gnu                                           39   68   65    65   60%    11   34% 
   3 QueenBee -clang                                         36   76   73    53   55%    42   30% 
   4 QueenBee-MonteCarlo                                    -92   70   71    63   37%    50   17% 
   5 DeepBrutePos/2.2/Sun Dec 22 00:24:54 2013/1134/nm-wb  -124   72   74    57   34%    74   30% 
   6 QueenBee -gnu-tpt                                     -182   75   80    57   25%    69   18% 
   7 QueenBee -clang-tpt                                   -208   80   87    45   21%    41   29%

The ones with "clang" are llvm 3.5, "gnu" is indeed g++. DeepBrutePos is a java application and the ones ending with "-tpt" are with transposition table.
So without tt it looks like llvm gives faster binaries, but with the (albeit broken) tt it is slower.

[matthewlai]

Just wondering, has anyone else been using LLVM?
I find that LLVM's compilation speed is MUCH faster than GCC (~2.5x), and the resulting binary (my engine) is also slightly faster (both -O3 -march=native). With my engine, LLVM is even faster than ICC.

Code: Select all

Rank Name                                                   Elo    +    - games score oppo. draws 
   1 Fairy-Max 4.8Q                                         529  207  124    72   99%   -84    1% 
   2 QueenBee -gnu                                           39   68   65    65   60%    11   34% 
   3 QueenBee -clang                                         36   76   73    53   55%    42   30% 
   4 QueenBee-MonteCarlo                                    -92   70   71    63   37%    50   17% 
   5 DeepBrutePos/2.2/Sun Dec 22 00:24:54 2013/1134/nm-wb  -124   72   74    57   34%    74   30% 
   6 QueenBee -gnu-tpt                                     -182   75   80    57   25%    69   18% 
   7 QueenBee -clang-tpt                                   -208   80   87    45   21%    41   29%

The ones with "clang" are llvm 3.5, "gnu" is indeed g++. DeepBrutePos is a java application and the ones ending with "-tpt" are with transposition table.
So without tt it looks like llvm gives faster binaries, but with the (albeit broken) tt it is slower.

That's way too few games to draw any conclusion. If you look at the error ranges ("+" and "-" columns), they are all much bigger than any difference in ELO.

A 100% increase in speed corresponds to about 75 ELO.

Increase from changing compiler will probably be something like 5% or ~3 ELO.

You'd need at least many thousand games to see a difference like that.

Just have the engine analyze the same position and see how much time it takes to reach a certain depth. Tree size shouldn't change, assuming you use the same seed for the random number generator on every run.