Threads test incl. Stockfish 7

[fastgm]

I have included the new Stockfish 7 with Lazy SMP implemantation in my threads test.

Here the data of the test and the graphical presentation, see also: http://www.fastgm.de/threads5.html including the test conditions.

Andreas

[Jouni]

I was waiting for this already thanks. Nice progress, if not at Komodo level? SF7 gets +228 from 16 cores, but Komodo 8 in older test 265. Seems to be same conditions.

[Nordlandia]

Nice representation and graphics

[bob]

As a note, and you will probably hear this from others, very fast searches have always been a weakness of normal parallel searches. They begin to "bear down" as time stretches out. It would be much more interesting to stretch the time to 60 seconds per move rather than about 1 second since that is a more useful / common time control limit. Also it would be nice to plot X vs 2x where 2x is the single-cpu version gives 2x as much time per move, to get a "optimal" potential parallel speedup. Ditto for 4x, 8x and 16x. Be nice to see the actual upper bound in addition to the smp numbers...

[Jouni]

And how consistent this is with framework test? There 15cpu SF7 got +131 vs SF6. Here it is 62 + 44 = 106. In any case homepage is way too modest with "This release is around 60 ELO stronger than Stockfish 6 in self play." AT LEAST 100 ELO STRONGER!

[bob]

And how consistent this is with framework test? There 15cpu SF7 got +131 vs SF6. Here it is 62 + 44 = 106. In any case homepage is way too modest with "This release is around 60 ELO stronger than Stockfish 6 in self play." AT LEAST 100 ELO STRONGER!

You do realize SF uses self-play for testing? And you should remember that a rating computed from playing yourself is usually exaggerated significantly compared to playing a gauntlet of other programs. As much as double in fact. So I think their estimate is pretty reasonable based on scaling it back to reflect games against others rather than against an older version of their own program...

[Jouni]

Definitely. SF7 got +62 in selfplay (framework) and +58 against other engines (IPON). "usually exaggerated significantly" seems to be history .

[APassionForCriminalJustic]

Definitely. SF7 got +62 in selfplay (framework) and +58 against other engines (IPON). "usually exaggerated significantly" seems to be history .

If you really believe that Stockfish 7 is like 100+ Elo over Stockfish 6 with a lot of cores then you are living in lala land. At standard match play I would seriously doubt 100+ Elo. Lazy smp will probably fail in the end given that it is a belief amongst some of the experts that it is not a very good smp implementation. Lazy didn't do well for TCEC nor was it successful in Clemens Keck's latest tournament versus Komodo. Who really cares about bullet games where lazy apparently shines. Nobody cares about these fast, pointless games.

[Dann Corbit]

I am really keen to see CCRL 40/40 results for many threads for both SF and Crafty. They both have shiny new threading implementations with high hopes.

The only thing that will reveal the truth is measurement. Someone has a sig about theory verses practice. Let me say this about that. You gotta measure.

[mhull]

I am really keen to see CCRL 40/40 results for many threads for both SF and Crafty. They both have shiny new threading implementations with high hopes.

CCRL has an 8-engine match going which includes Stockfish:
http://www.talkchess.com/forum/viewtopi ... 421#654421

And an 8-engine swiss of mass proportions including Crafty 25.0:
http://www.talkchess.com/forum/viewtopi ... 099#653099

But Crafty won't get truly interesting, parallel-wise until Bob completes his conversion to DTS later this year.

The only thing that will reveal the truth is measurement. Someone has a sig about theory verses practice. Let me say this about that. You gotta measure.

...theory versus practice...