Just wondering, has anyone else been using LLVM?
I have been developing on OSX, and want my engine to be compilable with both GCC and LLVM, and it's pretty easy.
I find that LLVM's compilation speed is MUCH faster than GCC (~2.5x), and the resulting binary (my engine) is also slightly faster (both -O3 -march=native). With my engine, LLVM is even faster than ICC. Though to be fair, that could be because I did most of my testing with LLVM, so I could have optimized for it. That said, it's general consensus that LLVM produces faster binaries than GCC, and compiles a lot faster.
LLVM also had better C++11 support than GCC 4.8. For example, GCC 4.8 libstdc++'s <regex> is completely broken even though it is included. Even simplest examples will give wrong results. It took me quite a while to figure out, and I am very disappointed in libstdc++'s decision to ship something so broken instead of just not including it, or at least add "#warn <regex> support is horribly broken, use at your own risk" or something to that effect.
Yes, the release note does say <regex> is broken. But who reads release notes for every single compiler update? A lot of times I don't even KNOW my compiler has been updated by APT when I do a system-wide update. No warnings at all when it's used. Really?
I also independently discovered another libstdc++ bug a while ago that was also very hard to narrow down. More than a year old now. Problem is simple and well understood. Clean fix available and patch submitted. Still not fixed after many months of sitting there.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58038
When I first learned to program I learned to always assume bugs are in my code instead of the compiler/standard library. It has served me well for a good few years, but libstdc++ is now constantly challenging that assumption.
At this point I am thinking about switching to LLVM/libc++ on Linux as well. What do you think?
Using LLVM over GCC
Moderators: hgm, Rebel, chrisw
-
- Posts: 793
- Joined: Sun Aug 03, 2014 4:48 am
- Location: London, UK
Using LLVM over GCC
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
-
- Posts: 3232
- Joined: Mon May 31, 2010 1:29 pm
- Full name: lucasart
Re: Using LLVM over GCC
Your mileage may vary. But in my experience, DiscoCheck and Stockfish are still faster when compiled with GCC. At least on Linux (Lubuntu 13.10). What GCC are you using? The real one? Or the Apple version?
One of the main issues ot Clang is that you can't use link time optimization on Debian/Ubuntu, because it's mis-packaged. And the packagers don't seem to care, as the issue has been going on for years and is still not fixed.
That being said, the difference is small, and once the link time optimization issue is fixed (another 10 years at this rate?) Clang/LLVM will be a good option on Linux. I'm sure Clang/LLVM will beat GCC in the long run, but not yet on my configuration.
One of the main issues ot Clang is that you can't use link time optimization on Debian/Ubuntu, because it's mis-packaged. And the packagers don't seem to care, as the issue has been going on for years and is still not fixed.
That being said, the difference is small, and once the link time optimization issue is fixed (another 10 years at this rate?) Clang/LLVM will be a good option on Linux. I'm sure Clang/LLVM will beat GCC in the long run, but not yet on my configuration.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.
-
- Posts: 793
- Joined: Sun Aug 03, 2014 4:48 am
- Location: London, UK
Re: Using LLVM over GCC
Apple doesn't supply GCC anymore. I am using standard GCC 4.9 from Homebrew (one of the third party repositories for OSX).lucasart wrote:Your mileage may vary. But in my experience, DiscoCheck and Stockfish are still faster when compiled with GCC. At least on Linux (Lubuntu 13.10). What GCC are you using? The real one? Or the Apple version?
One of the main issues ot Clang is that you can't use link time optimization on Debian/Ubuntu, because it's mis-packaged. And the packagers don't seem to care, as the issue has been going on for years and is still not fixed.
That being said, the difference is very small, and once the link time optimization issue is fixed (another 10 years at this rate?) Clang/LLVM will be a good option on Linux. I'm sure Clang/LLVM will beat GCC in the long run, but not yet on my configuration.
Both of the libstdc++ bugs were discovered on Linux though. Standard 4.8.
Yeah if LTO doesn't work it's going to be slow. It does work on OSX, though (which is not surprising since LLVM is OSX's official toolchain now).
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
-
- Posts: 27869
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Using LLVM over GCC
I heard there is some very weird behavior of Clang w.r.t. inlined functions. XBoard had two funcitions dedined as 'inline', and used only in the same file as they were defined. This then produces linker errors, because Clang does use real calls rather than inlining, but does not provide any real code to be called because it assumes they will be inlined...
I could cure this weirdnes by declaring them 'static inline'.
I could cure this weirdnes by declaring them 'static inline'.
-
- Posts: 793
- Joined: Sun Aug 03, 2014 4:48 am
- Location: London, UK
Re: Using LLVM over GCC
Hmm that's strange. I have a bunch of inline functions only used in the same file, too, and they compile fine.hgm wrote:I heard there is some very weird behavior of Clang w.r.t. inlined functions. XBoard had two funcitions dedined as 'inline', and used only in the same file as they were defined. This then produces linker errors, because Clang does use real calls rather than inlining, but does not provide any real code to be called because it assumes they will be inlined...
I could cure this weirdnes by declaring them 'static inline'.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
-
- Posts: 859
- Joined: Mon Aug 10, 2009 10:05 pm
- Location: Italy
- Full name: Stefano Gemma
Re: Using LLVM over GCC
In my limited experience, i'm finally decided to use the native compiler for any Platform. In Windows 64 bit, installing and configuring Eclipse/gcc was a pain (for me) and i haven't get a stable and usable environment. Debugging multi-threaded c++ application was almost impossible. I know that someone could hate it but Visual C++ is the best compiler that i've found, for Windows 64 bit. I come from Borland C++ Builder, so i'm not a MS fan but this seems to me to be the truth. I've tried gcc with Eclipse IDE, code::blocks and CodeLite, on Windows, with different obstacle to get a fast and really professional environment.
On Linux, Eclipse/gcc instead seems to works fine but i have done limited testing.
Same lot of problems to install and configure Eclipse/gcc on Mac OS X (for me) but it works. Still, xCode on Mac OS X appears to be the best choice.
I think that Windows porting of Eclipse/gcc environment (with MingW) is not a stable solution, if you have to handle professional software. This ide/compiler are Linux native and should be used only in Linux (IMHO, of course).
I've seen only this morning the Clang site and it seems a good compiler, i haven't tryed yet maybe i'll take a look on it.
The problem is that today, in 2014, we can't use a good compiler not fully integrated in a good ide, in _OUR_ operating system. Is not important if the operating system itself is not good: the whole environment should works well, fast and with no pain to configure all that we need.
In Satana development i've loose a lot of time only to try making Eclipse/gcc works in a decent way... this is not acceptable, for me.
On Linux, Eclipse/gcc instead seems to works fine but i have done limited testing.
Same lot of problems to install and configure Eclipse/gcc on Mac OS X (for me) but it works. Still, xCode on Mac OS X appears to be the best choice.
I think that Windows porting of Eclipse/gcc environment (with MingW) is not a stable solution, if you have to handle professional software. This ide/compiler are Linux native and should be used only in Linux (IMHO, of course).
I've seen only this morning the Clang site and it seems a good compiler, i haven't tryed yet maybe i'll take a look on it.
The problem is that today, in 2014, we can't use a good compiler not fully integrated in a good ide, in _OUR_ operating system. Is not important if the operating system itself is not good: the whole environment should works well, fast and with no pain to configure all that we need.
In Satana development i've loose a lot of time only to try making Eclipse/gcc works in a decent way... this is not acceptable, for me.
-
- Posts: 793
- Joined: Sun Aug 03, 2014 4:48 am
- Location: London, UK
Re: Using LLVM over GCC
That is the reason why I never do any portable development/debugging on Windows.stegemma wrote:In my limited experience, i'm finally decided to use the native compiler for any Platform. In Windows 64 bit, installing and configuring Eclipse/gcc was a pain (for me) and i haven't get a stable and usable environment. Debugging multi-threaded c++ application was almost impossible. I know that someone could hate it but Visual C++ is the best compiler that i've found, for Windows 64 bit. I come from Borland C++ Builder, so i'm not a MS fan but this seems to me to be the truth. I've tried gcc with Eclipse IDE, code::blocks and CodeLite, on Windows, with different obstacle to get a fast and really professional environment.
On Linux, Eclipse/gcc instead seems to works fine but i have done limited testing.
Same lot of problems to install and configure Eclipse/gcc on Mac OS X (for me) but it works. Still, xCode on Mac OS X appears to be the best choice.
I think that Windows porting of Eclipse/gcc environment (with MingW) is not a stable solution, if you have to handle professional software. This ide/compiler are Linux native and should be used only in Linux (IMHO, of course).
I've seen only this morning the Clang site and it seems a good compiler, i haven't tryed yet maybe i'll take a look on it.
The problem is that today, in 2014, we can't use a good compiler not fully integrated in a good ide, in _OUR_ operating system. Is not important if the operating system itself is not good: the whole environment should works well, fast and with no pain to configure all that we need.
In Satana development i've loose a lot of time only to try making Eclipse/gcc works in a decent way... this is not acceptable, for me.
I like to use the same toolchain on all platforms because in reality, supporting multiple compilers does take some work, even if you try very hard to be totally standard-compliant. GCC (and hopefully later LLVM) is a natural target since it's available on just about all platforms under the sun.
The problem with MSVC is of course that it's not available on anything other than Windows.
GNU toolchain on Windows is always a little glitchy, so I do all my development on Linux or (now) OSX, and just switch to Windows to make a build when I need to. I can probably even set up a mingw cross-compiling toolchain to never have to boot into Windows, but I never got around to that.
That way, I don't have to deal with all the toolchain stuff on Windows, as long as calling g++ works, and I can still make Windows builds.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
-
- Posts: 4368
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
Re: Using LLVM over GCC
I build with LLVM on MacOS because it is the platform compiler there.
I also support compilation under: gcc, Visual C++, clang, Intel cc, and mingw, although the last 3 of these are not thoroughly tested. You will get different warnings/errors from the various compilers so at least for that reason it is beneficial to try multiple ones.
Personally I have not seen a significant runtime speed advantage from LLVM.
I think gcc is pretty good now. They have made really significant architectural changes over the past decade or so.
--Jon
I also support compilation under: gcc, Visual C++, clang, Intel cc, and mingw, although the last 3 of these are not thoroughly tested. You will get different warnings/errors from the various compilers so at least for that reason it is beneficial to try multiple ones.
Personally I have not seen a significant runtime speed advantage from LLVM.
I think gcc is pretty good now. They have made really significant architectural changes over the past decade or so.
--Jon
Re: Using LLVM over GCC
matthewlai wrote:Just wondering, has anyone else been using LLVM?
I find that LLVM's compilation speed is MUCH faster than GCC (~2.5x), and the resulting binary (my engine) is also slightly faster (both -O3 -march=native). With my engine, LLVM is even faster than ICC.
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Fairy-Max 4.8Q 529 207 124 72 99% -84 1%
2 QueenBee -gnu 39 68 65 65 60% 11 34%
3 QueenBee -clang 36 76 73 53 55% 42 30%
4 QueenBee-MonteCarlo -92 70 71 63 37% 50 17%
5 DeepBrutePos/2.2/Sun Dec 22 00:24:54 2013/1134/nm-wb -124 72 74 57 34% 74 30%
6 QueenBee -gnu-tpt -182 75 80 57 25% 69 18%
7 QueenBee -clang-tpt -208 80 87 45 21% 41 29%
So without tt it looks like llvm gives faster binaries, but with the (albeit broken) tt it is slower.
-
- Posts: 793
- Joined: Sun Aug 03, 2014 4:48 am
- Location: London, UK
Re: Using LLVM over GCC
That's way too few games to draw any conclusion. If you look at the error ranges ("+" and "-" columns), they are all much bigger than any difference in ELO.flok wrote:matthewlai wrote:Just wondering, has anyone else been using LLVM?
I find that LLVM's compilation speed is MUCH faster than GCC (~2.5x), and the resulting binary (my engine) is also slightly faster (both -O3 -march=native). With my engine, LLVM is even faster than ICC.The ones with "clang" are llvm 3.5, "gnu" is indeed g++. DeepBrutePos is a java application and the ones ending with "-tpt" are with transposition table.Code: Select all
Rank Name Elo + - games score oppo. draws 1 Fairy-Max 4.8Q 529 207 124 72 99% -84 1% 2 QueenBee -gnu 39 68 65 65 60% 11 34% 3 QueenBee -clang 36 76 73 53 55% 42 30% 4 QueenBee-MonteCarlo -92 70 71 63 37% 50 17% 5 DeepBrutePos/2.2/Sun Dec 22 00:24:54 2013/1134/nm-wb -124 72 74 57 34% 74 30% 6 QueenBee -gnu-tpt -182 75 80 57 25% 69 18% 7 QueenBee -clang-tpt -208 80 87 45 21% 41 29%
So without tt it looks like llvm gives faster binaries, but with the (albeit broken) tt it is slower.
A 100% increase in speed corresponds to about 75 ELO.
Increase from changing compiler will probably be something like 5% or ~3 ELO.
You'd need at least many thousand games to see a difference like that.
Just have the engine analyze the same position and see how much time it takes to reach a certain depth. Tree size shouldn't change, assuming you use the same seed for the random number generator on every run.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.