The Best Stockfish Version and compile

[PaulieD]

My tests from Immortal site:

i3 M380 Dual Core @2.53 GHz - 6.0GB RAM
W7 Home Premium X.64 SP-1
FRITZ BENCH: Speed 4.82/Kns 2313
GUI: Deep Fritz 14 (64Bit)
HASH: 1024MB
BOOK: GM 8 move (Stockfish Testing Framework)
TABLEBASES: None
PONDER: Off
TIME: 1 Min

This is my first test with the new Deep fritz 14 (64 bit) GUI. I like it, it is smoother and more powerful than DF 13 was.

I wanted, once again, to test for the strongest SF version. I used only Ipman complies. I also enabled Large Pages only and not TB's although all versions were TB capable.

I did this based on differing test results from a variety of sources...Ipman's site, my own testing and the testing of the other major testers on this site.

The goal was to run 133 head to head matches (total 399) each amongst what has been the 3 versions most often identified as the strongest SF versions to date.

The winner is 021113SL, this agrees with the results at Ipmans site and it is 11 ELO stronger than 241113SL and 12 ELO stronger than my old champ 111113SL. The error bars are 24-25 Elo on this many games, as you can see in the Elostat below, so you could get different results. However, this is the largest test of it's kind in this thread at the current time.

Code: Select all

EloStat
Program                              Elo   +   -   Games   Score   Av.Op.  Draws

  1 Stockfish 021113SL 64 SSE4.2   : 3208  25  24   266    51.7 %   3196   65.8 %
  2 Stockfish 241113SL 64 SSE4.2   : 3197  25  25   266    49.2 %   3202   63.9 %
  3 Stockfish 111113SL 64 SSE4.2   : 3196  24  24   266    49.1 %   3202   68.0 %

PGN: https://mega.co.nz/#!DcBkzIbS!B20RBVbhX ... fD78BEr_g0

[ouachita]

where can I download 021113SL?

[PaulieD]

https://mega.co.nz/#!TFp2SKDS!JwXkqjd5I ... q4Da9oZR6s

or try here:
http://immortalchess.net/forum/showpost ... count=3714

[ouachita]

thx Paul, keeping up with the latest SF versions and their relative strength seems to be a full time job

[gleperlier]

Thanks !

Gab

[Houdini]

The winner is 021113SL, this agrees with the results at Ipmans site and it is 11 ELO stronger than 241113SL and 12 ELO stronger than my old champ 111113SL. The error bars are 24-25 Elo on this many games, as you can see in the Elostat below, so you could get different results. However, this is the largest test of it's kind in this thread at the current time.

Good test, but wrong conclusion.
Based on your test it's impossible to say which version is stronger, the error margins are a lot larger than the measured difference. On another day you could easily find a different order.

The bottom-line is that if you want to measure a 10 Elo difference you need to play several thousand games, otherwise you're just generating statistical noise.

Robert

[ouachita]

Based on your test it's impossible to say which version is stronger, . . . Robert

. . . which is of course is one of the problems with daily compiles

[PaulieD]

The winner is 021113SL, this agrees with the results at Ipmans site and it is 11 ELO stronger than 241113SL and 12 ELO stronger than my old champ 111113SL. The error bars are 24-25 Elo on this many games, as you can see in the Elostat below, so you could get different results. However, this is the largest test of it's kind in this thread at the current time.

Good test, but wrong conclusion.
Based on your test it's impossible to say which version is stronger, the error margins are a lot larger than the measured difference. On another day you could easily find a different order.

The bottom-line is that if you want to measure a 10 Elo difference you need to play several thousand games, otherwise you're just generating statistical noise.

Robert

I said that already?!
"The error bars are 24-25 Elo on this many games, as you can see in the Elostat below, so you could get different results."

[PaulieD]

thx Paul, keeping up with the latest SF versions and their relative strength seems to be a full time job

You are right!

There is not enough time to run enough games in between versions, so we do the best we can...