The best compiler for chess, Intel or gcc or something else?

[Don]

P.S. PGO was useful before 4.6 - at least for me. But since then it has worked extremely well for me.

In my experience, nothing beats GCC 4.7. As for PGO, I have never found that to be faster: maybe it used to in earlier versions, but with -O4 -flto, it's as fast w/o PGO

Who needs ICC or Mickeysoft VC++ anymore

That is pretty odd, I found major benefit's for PGO with Komodo. Maybe it is very program specific then.

[bob]

ICC used to have an edge even when PGO was not used. In my experience this is not the case anymore since gcc 4.5.

When PGO is used, it's possible that ICC still has an edge. Probably a small one, if anything (something like 5-10%).

GCC has an edge in being able to generate hardware-agnostic binaries, whereas ICC creates executables that either don't run on AMD at all, or run with a crappy codepath which deliberately cripples performance. This is well documented and was proven in court:

http://www.agner.org/optimize/blog/read.php?i=49

On this web page is noted

In my tests, the Gnu compiler for Linux has an optimizing performance similar to the Intel compiler, but the Gnu function library (glibc) is inferior.

Is this correct ? Here's an old post by Linus http://ecos.sourceware.org/ml/libc-alph ... 00079.html

This goes back to early 2002 and the point was not that it was inferior, but Linus felt it was bloated - too many functions.

In either case, I don't think this applies to execution speed of a chess program.

He was pointing out that it can have an unseen effect on ANY program. A big library, loaded into memory, needs a lot of pages. If the functions you use (and you do use quite a few even if you don't realize it) are scattered around in a bloated shared library, the number of pages you access goes up. And that spills over into TLB thrashing, which turns into slower virtual-to-real addresses for lots of instructions.

Lots of people don't look beyond the ends of their noses when they are writing code like that and just think "it is big, but it isn't used much, so nobody cares" when they have no idea what they are talking about....

There are usually two angles to these arguments, which is easier to support, and which is most efficient in terms of system resource usage. Many decisions are based on the first question only. Linus is always interested in BOTH.

[bob]

P.S. PGO was useful before 4.6 - at least for me. But since then it has worked extremely well for me.

In my experience, nothing beats GCC 4.7. As for PGO, I have never found that to be faster: maybe it used to in earlier versions, but with -O4 -flto, it's as fast w/o PGO

Who needs ICC or Mickeysoft VC++ anymore

That is pretty odd, I found major benefit's for PGO with Komodo. Maybe it is very program specific then.

For well-written programs, PGO is not going to produce huge improvements. But it will improve things significantly. 10% to as much as 20% is certainly possible. But this is mainly about optimizing the direct instruction path of a program so that the cache doesn't load blocks of code that are rarely used. There's very little gain elsewhere. But with a lot of if statements, particularly if-then-else type structures, it will move the uncommon path out of the primary execution stream and cause cache prefetching (filling an entire block) to work better since it won't prefetch the blocks of code that are infrequently used.

[Don]

P.S. PGO was useful before 4.6 - at least for me. But since then it has worked extremely well for me.

In my experience, nothing beats GCC 4.7. As for PGO, I have never found that to be faster: maybe it used to in earlier versions, but with -O4 -flto, it's as fast w/o PGO

Who needs ICC or Mickeysoft VC++ anymore

That is pretty odd, I found major benefit's for PGO with Komodo. Maybe it is very program specific then.

For well-written programs, PGO is not going to produce huge improvements. But it will improve things significantly. 10% to as much as 20% is certainly possible. But this is mainly about optimizing the direct instruction path of a program so that the cache doesn't load blocks of code that are rarely used. There's very little gain elsewhere. But with a lot of if statements, particularly if-then-else type structures, it will move the uncommon path out of the primary execution stream and cause cache prefetching (filling an entire block) to work better since it won't prefetch the blocks of code that are infrequently used.

I actually meant that before 4.6 it was USELESS - I could see no advantage at all. By mistake I said it was "useful" but that is not what I meant.

I do take some care to avoid conditional instructions as much as possible but any chess program is relatively heavy on logic. But complex nested if/then statements - are you saying that PGO does the best with them? I could easily believe that.

[jdart]

As I have reported before, my experience is that, even though GCC PGO is broken, recent GCC versions w/o PGO are at least as good as Intel compiles with PGO. That is on my code, and results are likely to vary depending on what your code is like.

On Windows I do not use GCC, I use MSVC with PGO. But there too I have not seen better results with the Intel compiler.

--Jon

[jdart]

Personally I have not found clang to give better runtime performance than GCC.

--Jon

[diep]

Jim Ablett would surely have something to say on this. Someone should make him aware of the thread.

I don't know about MSVC but let's compare to the Intel compiler. When Jim Ablett was making binaries for me, they were faster that anything I could produce but they were nothing like what is being reported here. To believe some of these posts gcc is a completely worthless compiler and huge numbers are being bandied about that do not have the ring of truth. Jim Ablett used the Intel compiler and could get about 5% more than I could get with GCC, but this was before gcc 4.6 which has improved a lot.

So I'm asking people here not be compiler advocates, this thread is not about what your favorite compiler is but to get some objective facts based on recent data. That may be asking too much however because it's easy to use the "wrong" compiler options, the wrong compiler versions and so on. So anytime I hear anything too much over 5% (comparing gcc to Intel or MSVC) I start to believe something is being exaggerated or made up. In fact my experience says that 5% is a rough upper bound on what is credible in either direction. I do think it's likely one compiler may be 2-5 percent better. The other possibility is that this is highly dependent on how you write code. I have heard people claim that llvm or other compilers compile their code to run faster than gcc does, but each time I try it is around 5% weaker.

GCC needs more code and a lot for Diep.

As a result it has a much higher L1i missrate, that translates to around a 10% difference in speed. It's actually 11% when i accurately compared.

That's single core when running on the same L5420 hardware.

I did not compare with a recent intel c++ nor with a recent visual studio in fact, that's how huge the difference is.

The executable size of GCC is a lot larger.

I compared in both cases of course a PGO'ed version. I pgo'ed for exactly 15 minutes in both cases.

Not sure why you say anything is not believable above 5%.

It's easy to prove WHY gcc 4.7.0 is slower. It needs MORE instructions to get the SAME thing done and a LOT.

Diep is C code.

So GCC improved a lot for intel core2/i7 cpu's, as difference was really 30% or so, yet it's still a lot behind.

[diep]

Personally I have not found clang to give better runtime performance than GCC.

--Jon

I'll retest clang again real soon. interesting to know their progress.

Last time i tested it - it wasn't impressive - yet they had just started then...

[diep]

In another thread someone reports that "MSVC version without PGO was 10% faster then the GCC version with the PGO."

But Dragan, who reports this, does not say what compiler options are used.

In the past I've noticed that MSVC without PGO was slightly faster than Intel C++ without PGO, but with PGO enabled Intel C++ was still on top.

Today I've made a short comparison with my program between MSVC 10.0, MSVC 11.0 and Intel C++ 13.0 using a single thread on a core i7:

MSVC 10.0 (-O2) 2627007 nps.
MSVC 11.0 (-O2) 2615285 nps.
Intel C++ 13.0 (-O2) 2629834 nps.
Intel C++ 13.0 (-O3) 2630472 nps.
Intel C++ 13.0 (PGO) 2707104 nps.
MSVC 11.0 (PGO) 2825527 nps.

It seems MSVC 11.0 with PGO has a slight edge here.

I never tried with GCC because (in the past) it had a bad reputation, and I'll probably have to make a lot of changes to my source to get it compiled without errors.

I also found that the use of 64 bit ints instead of 32 bit ints gives a very slight improvement on both Intel and MSVC. Under Linux both compilers seem to default to 64 bit ints while under Windows this is 32 bit.

Time to try msvc 11.0, this is an impressive step forward.

Now of course i'm not bitboards so tricky vectorisation in Diep is a lot tougher, but still...

[Don]

Jim Ablett would surely have something to say on this. Someone should make him aware of the thread.

I don't know about MSVC but let's compare to the Intel compiler. When Jim Ablett was making binaries for me, they were faster that anything I could produce but they were nothing like what is being reported here. To believe some of these posts gcc is a completely worthless compiler and huge numbers are being bandied about that do not have the ring of truth. Jim Ablett used the Intel compiler and could get about 5% more than I could get with GCC, but this was before gcc 4.6 which has improved a lot.

So I'm asking people here not be compiler advocates, this thread is not about what your favorite compiler is but to get some objective facts based on recent data. That may be asking too much however because it's easy to use the "wrong" compiler options, the wrong compiler versions and so on. So anytime I hear anything too much over 5% (comparing gcc to Intel or MSVC) I start to believe something is being exaggerated or made up. In fact my experience says that 5% is a rough upper bound on what is credible in either direction. I do think it's likely one compiler may be 2-5 percent better. The other possibility is that this is highly dependent on how you write code. I have heard people claim that llvm or other compilers compile their code to run faster than gcc does, but each time I try it is around 5% weaker.

GCC needs more code and a lot for Diep.

As a result it has a much higher L1i missrate, that translates to around a 10% difference in speed. It's actually 11% when i accurately compared.

That's single core when running on the same L5420 hardware.

I did not compare with a recent intel c++ nor with a recent visual studio in fact, that's how huge the difference is.

The executable size of GCC is a lot larger.

I compared in both cases of course a PGO'ed version. I pgo'ed for exactly 15 minutes in both cases.

Not sure why you say anything is not believable above 5%.

It's easy to prove WHY gcc 4.7.0 is slower. It needs MORE instructions to get the SAME thing done and a LOT.

Diep is C code.

The number of instructions to accomplish something can easily be explained by many things but it's clear that the shortest code is not guaranteed to be the fastest. For example loop unrolling can benefit a program and that is often controlled by simple compiler options. If Deep has a lot of code, then your experience could be much different due to instruction cache issues.

The reason it's difficult to believe any claim much than 5% is that it simply is not what I'm observing from my own experiences - the ones I can verify. I know that Vida uses MSVC and GCC and there is a 6% difference in favor of GCC - but Linux is always faster if both compiles are good.

I have had people make compiles for me using Intel and MSVC in the past and they only beat my own cross compiled mingw stuff by a few percent - and that was with the inferior gcc versions.

Anyway, what I am hearing is a lot of numbers that do not add up and are in serious conflict. The only thing I am sure of is that gcc is not 15% slower than everything else as has been suggested.

I think this discussion is a lot like comparing test results, different compilers, different compiler versions, different platforms, different compiler options - the truth gets obfuscated somewhere in the noise.