The best compiler for chess, Intel or gcc or something else?

[ZirconiumX]

First of all, I would like to point out that I have been playing devil's advocate throughout this - and with good reason as you shall read.

Conditions: all compiles -O3 - ICC 13.0, GCC 4.4.2, Clang 3.1. Durandal Oct 12 2012 source. Benchmarked to depth 14 from the standard chess starting position.

I shall read these out from worst to best.

GCC

The bog standard benchmark. I really need to get a better version - but it isn't the most straightforward of things to compile.

Depth: 14
Time: 1m 36.930sec
Nodes: 123795582
NPS: 1277164

ICC

Apparently ICC makes the best compiles - it also makes INCORRECT ones as you can tell from the node count.

Depth: 14
Time: 1m 20.146sec
Nodes: 127549048
NPS: 1591458

LLVM

No one seems to have paid much in the way of attention to LLVM. It is something of a shame then - because it is the fastest.

Depth: 14
Time: 1m 14.419sec
Nodes: 123795582
NPS: 1663494

So, we can conclude that the bog standard compiler is bog standard, the best one is incorrect and the worst compiler is the best.

Matthew:out

[syzygy]

Apparently ICC makes the best compiles - it also makes INCORRECT ones as you can tell from the node count.

Before you blame the compiler, blame the code.

[diep]

Apparently ICC makes the best compiles - it also makes INCORRECT ones as you can tell from the node count.

Before you blame the compiler, blame the code.

Pathetic compare anyway - he's not using PGO.
ICC gets a LOT out of pgo.

[bob]

First of all, I would like to point out that I have been playing devil's advocate throughout this - and with good reason as you shall read.

Conditions: all compiles -O3 - ICC 13.0, GCC 4.4.2, Clang 3.1. Durandal Oct 12 2012 source. Benchmarked to depth 14 from the standard chess starting position.

I shall read these out from worst to best.

GCC

The bog standard benchmark. I really need to get a better version - but it isn't the most straightforward of things to compile.

Depth: 14
Time: 1m 36.930sec
Nodes: 123795582
NPS: 1277164

ICC

Apparently ICC makes the best compiles - it also makes INCORRECT ones as you can tell from the node count.

Very poor assumption. Most likely, the PROGRAM has a bug. Most common form is an uninitialized variable, but that is not the only potential bug, as one can do bad things with pointers.

The intel compiler is used in commercial enterprises all over the world. A compiler bug is always possible, but extremely unlikely.

Depth: 14
Time: 1m 20.146sec
Nodes: 127549048
NPS: 1591458

LLVM

No one seems to have paid much in the way of attention to LLVM. It is something of a shame then - because it is the fastest.

Depth: 14
Time: 1m 14.419sec
Nodes: 123795582
NPS: 1663494

So, we can conclude that the bog standard compiler is bog standard, the best one is incorrect and the worst compiler is the best.

Matthew:out

[ZirconiumX]

Vincent - I purposely didn't use PGO - as that would introduce yet more variables.

To the others - If my code is at fault - why do GCC and Clang both produce the same node count? It is possible that one of the intel optimizations has optimized out something very important in my search - I don't know. And I'm not going to trawl through x86 assembler to find out.

I have a feeling that if the intel compile produced the correct node count - it would be fastest.

Matthew:out

[rbarreira]

Vincent - I purposely didn't use PGO - as that would introduce yet more variables.

To the others - If my code is at fault - why do GCC and Clang both produce the same node count? It is possible that one of the intel optimizations has optimized out something very important in my search - I don't know. And I'm not going to trawl through x86 assembler to find out.

I have a feeling that if the intel compile produced the correct node count - it would be fastest.

Matthew:out

The only way to find out if it's a bug in your code or a compiler bug is to narrow down the problem until you identify the piece of code which is behaving differently in the two executables.

[diep]

Vincent - I purposely didn't use PGO

then test it WITH pgo.

intel c++ wins 20-30% here with pgo and GCC just 7%.

You could of course also make a comparition between the compilers just using -O instead of -O2 or -O3.

[jdart]

Re icc, you can try the -fno-strict-aliasing flag, which enables code that breaks the strict aliasing rules to compile/run (this is one of the most common "bad stuff with pointers" that Bob is referring to).

--Jon

[Tom Likens]

For Diep -O3 is not faster than -O2 when using PGO with GCC 4.7.0

Hey Vincent,

How did Diep compiled w/ gcc 4.7.0 compare to ICC? Did it make up any of the advantage
that Intel had previously?

regards,
--tom

[Karlo Bala]

To the others - If my code is at fault - why do GCC and Clang both produce the same node count?

Matthew:out

There could be a number of reasons. Some compilers initialize uninitialized variables some other - not. Different compilers may place data in different order, then writing data beyond the array bounds could damage some other data, etc.