Stockfish 1.7

[swami]

So, Is Stockfish 1.7 really stronger than Rybka 3?

Can somebody confirm this?

Well done, the team.

[sockmonkey]

So, Is Stockfish 1.7 really stronger than Rybka 3?

Can somebody confirm this?

Well done, the team.

Code: Select all

STOCK17_1

Stockfish 1.7 JA - Rybka 3 mp (2907)               41.0  -  36.0        53.25%      Perf=2929
Stockfish 1.7 JA - Naum 4.2 (2823)                 46.0  -  31.0        59.74%      Perf=2891
Stockfish 1.7 JA - Deep Shredder 12 (2801)         52.5  -  24.5        68.18%      Perf=2933
Stockfish 1.7 JA - Komodo64 1.0 JA (2781)          51.0  -  25.0        67.11%      Perf=2904
Stockfish 1.7 JA - Zappa Mexico II (2709)          58.0  -  19.0        75.32%      Perf=2902
Stockfish 1.7 JA - Protector 1.3.2 JA (2698)       60.5  -  16.5        78.57%      Perf=2923
Stockfish 1.7 JA - Onno-1-1-1 (2682)               54.5  -  22.5        70.78%      Perf=2835
Stockfish 1.7 JA - Spark-0.3 VC(a) (2672)          60.5  -  16.5        78.57%      Perf=2897
Stockfish 1.7 JA - Deep Sjeng WC2008 (2671)        60.0  -  16.0        78.95%      Perf=2900
Stockfish 1.7 JA - Toga II 1.4 beta5c BB (2661)    61.0  -  16.0        79.22%      Perf=2893
                                                   545.0 -  223.0       70.96%      Perf=2895



768 out of 1000 games played

It's close, but Stockfish is at definitely holding. Great work!

Jeremy

[Lusakan]

"Close" doesnt explain much. The point is how do we interpret this:

STOCK17_1

Stockfish 1.7 JA - Rybka 3 mp (2907) 41.0 - 36.0 53.25% Perf=2929

Which one is stronger?

[sockmonkey]

"Close" doesnt explain much. The point is how do we interpret this:

STOCK17_1

Stockfish 1.7 JA - Rybka 3 mp (2907) 41.0 - 36.0 53.25% Perf=2929

Which one is stronger?

I'm not sure I understand the question. Stockfish 1.7 has won 53.25% of its games against Rybka 3, after 768 games.

Jeremy

[Lusakan]

My question really was pushing for someone to answer Swami's question bravely. Now looks like what you are saying is that even though stockfish 1.7 came out on top after 1000 games against Rybka 3 we cant say if its stronger or not because at 53.25% the results are too close to call, right?

[sockmonkey]

My question really was pushing for someone to answer Swami's question bravely. Now looks like what you are saying is that even though stockfish 1.7 came out on top after 1000 games against Rybka 3 we cant say if its stronger or not because at 53.25% the results are too close to call, right?

A sample size of 1000 = ~3.16% inaccuracy, I believe. So we don't really know. It's close.

Jeremy

[Lion]

I'd agree with you. It detected 2 threads on my quad, fortunately I saw that before I started testing it on playchess . It does use all 4 when set too, though.

Guys, maybe a 1.7.1 with proper thread detection and null move bug fix is needed.

Peter

Could you (and also all the other people that experience problems with CPU detection) please post type of CPU you have ? Thanks.

The null move bug will require regression test anyway so although the fix could be quick the testing will take some time and we would wait at least a week or two to collect all the bug reports before to release a mainteinance version.

Main is an i7 980x with 6 cores.
It detected only 3 cores.

Also when I then set to 6, and opened my internet explorer, it went down to 5 CPU.

Dont know if this is normal but other engines like Rybka dont do that.

[Lion]

I'd agree with you. It detected 2 threads on my quad, fortunately I saw that before I started testing it on playchess . It does use all 4 when set too, though.

Guys, maybe a 1.7.1 with proper thread detection and null move bug fix is needed.

Peter

Could you (and also all the other people that experience problems with CPU detection) please post type of CPU you have ? Thanks.

The null move bug will require regression test anyway so although the fix could be quick the testing will take some time and we would wait at least a week or two to collect all the bug reports before to release a mainteinance version.

What is the philosophical idea behind auto detecting then number of cores? Based on the things I read before, in the spirit of SF it looked like it should leave this task to the interface. Right?

Miguel

The GUI does not set this value.

Until 1.6.3 we used what is now called builtin_cpu_count() in misc.cpp to detect the number of cores, this works but has the problem that _if_ hyper threading is enabled it _usually_ reports the double of the real physical cores.

So this time we add function HT_enabled() to detect hyperthreading and, in this case, divide by two the number of reported CPUs, see cpu_count().

The idea is that if you have, say 4 physical cores, but with HT enabled the builtin_cpu_count() returns 8 then you really want to consider only 4 cores for optimal performance because SF, as also all the other SMP engines works best if the number of CPU is set to the real physical cores, not the "logical" ones when HT is enabled.

The problem that is arising is that function HT_enabled() is broken at least for some i7 CPU models.

So we have two options:

1) Fix HT_enabled() to works always: but this is very difficult because, reading Intel documentation, the proper way to detect HT is brain damaged complex (this time Intel made a real idiotic decision with HT detection design). Current routine works always on older types of CPU, but newer ones require a much more complex approach.

2) Remove HT_enabled() entirely and revert to 1.6.3 behaviour. In this case people should remember of disable HT or, when HT is enabled to manually set the correct number of cores through UCI options "threads" to the number of real cores (that normally is the half of what is reported with HT enabled).

Is HT usefull for Stockfish 1.7 ?

[Dr.Wael Deeb]

My question really was pushing for someone to answer Swami's question bravely. Now looks like what you are saying is that even though stockfish 1.7 came out on top after 1000 games against Rybka 3 we cant say if its stronger or not because at 53.25% the results are too close to call, right?

In the upcoming 3-4 months you'll be astonished how many engines will squash Rybka 3 and please don't mention the release of Rybka 4 because this is a myth until I see it with my own eyes....and even then it will not dominant the rating lists....
Dr.D

[Uri Blass]

"Close" doesnt explain much. The point is how do we interpret this:

STOCK17_1

Stockfish 1.7 JA - Rybka 3 mp (2907) 41.0 - 36.0 53.25% Perf=2929

Which one is stronger?

I'm not sure I understand the question. Stockfish 1.7 has won 53.25% of its games against Rybka 3, after 768 games.

Jeremy

No
Most of the games are not against rybka3

so far Stockfish scored 45 points out of 86 games against rybka(there are also games against other engines)

STOCK17_1

Stockfish 1.7 JA - Rybka 3 mp (2907) 45.0 - 41.0 52.33% Perf=2923
Stockfish 1.7 JA - Naum 4.2 (2823) 51.5 - 34.5 59.88% Perf=2892
Stockfish 1.7 JA - Deep Shredder 12 (2801) 58.0 - 29.0 66.67% Perf=2921
Stockfish 1.7 JA - Komodo64 1.0 JA (2781) 57.0 - 29.0 66.28% Perf=2898
Stockfish 1.7 JA - Zappa Mexico II (2709) 65.0 - 21.0 75.58% Perf=2905
Stockfish 1.7 JA - Protector 1.3.2 JA (2698) 68.5 - 17.5 79.65% Perf=2935
Stockfish 1.7 JA - Onno-1-1-1 (2682) 62.0 - 24.0 72.09% Perf=2846
Stockfish 1.7 JA - Spark-0.3 VC(a) (2672) 69.0 - 17.0 80.23% Perf=2915
Stockfish 1.7 JA - Deep Sjeng WC2008 (2671) 67.5 - 18.5 78.49% Perf=2895
Stockfish 1.7 JA - Toga II 1.4 beta5c BB (2661) 68.5 - 17.5 79.65% Perf=2898
612.0 - 249.0 71.08% Perf=2896




861 out of 1000 games played