How many rating points is SF 8 over SF 7?

[Dann Corbit]

Hi Dann,

it will be 30 ELO not more.

If engines started with optimal settings (in case of many opponents with Contempt).

Can be see in my list if ASM Fish 8 BMI2 x64 will be start in three days. With shorter time controls I think also much more as 30 Elo. I am absolutely sure that ASM isn't stronger as Komodo 10.2 with more as 30 Elo. And we are speaking from the strongest available SF version.

The 40 days old SF with the fastest compile by ii u. ll is at the moment 8.5 Elo stronger as Komodo 10.2! Fact with 4.0 GHz hardware and many opponents and 45 minutes per game with a very very equal opening book.

Important is again and again ...
A big group of opponents or never a rating can be exactly enough. After my experiments with my own list ... 26 opponents are min. a "to have" for a rating list. In all other cases, not important how many games ... ratings are not exactly enough.

Easy to find out it with my database in self work!

Best
Frank

And SF 7 to SF 8 ...

Code: Select all

   SF 18Sep2016 BMI2 x64 C10        :  3216.77   2650    87.4   24.0  14.48  2837.65   8.28   53.0
   Stockfish 7 KP BMI2 x64          :  3160.17   3200    85.8   26.6  12.93  2808.96   7.91   64.0

= 56 + 5-10 Elo more to the Version 8 + max. 5-10 Elo more for ASM +15-20 Elo more for Contempt = around 85-100 Elo with longer time controls.

All the early returns like Pohl's list and 40/4 will be at fast time control. At TCEC speed and hardware, maybe only 15 Elo. This indicates a problem in the Elo system, or, rather, shows that Elo is not linear.

[corres]

[quote="Frank Quisinsky"]
... ratings are not exactly enough.
.[/quote]

Hi Frank,
I have seen exact rating never.
The main reason is that there is no definition of exact rating.
Moreover testers have freedom of choice as to what they wish to test and how. This sounds very liberal but totally opposed to the much basic technical requirements.
Greetings
Robert

[corres]

[quote="Frank Quisinsky"]
... ratings are not exactly enough.
[/quote]
Topic does not work well!

[corres]

[quote="Frank Quisinsky"]
... ratings are not exactly enough.
.[/quote]

Hi Frank,
Sorry, but I have seen exact rating never.
The main reason is that there is no definition of exact rating.
Moreover testers have freedom of choice as to what they wish to test and how. This sounds very liberal but this attitude totally opposed to the much basic technical demands to make something up exact mode.
I think there are too much subjectivity in making tests and the interpretation of the results.

[ThatsIt]

2000 games for the CEGT 40/4 + ... are played so far.

http://cegt.forumieren.com/t709-testing-stockfish-8-0

Code: Select all

Stockfish 8.0      x64 1CPU = ca. ELO 3333 out of 2000 games (+26 / +80)
Stockfish 20160716 x64 1CPU =     ELO 3307 out of 2000 games
Stockfish 7.0      x64 1CPU =     ELO 3253 out of 3300 games