SPCC: Testrun of Stockfish 170503 finished

[pohl4711]

Testrun of Stockfish 170503 finished.

Long thinking-time tournament updated.

http://spcc.beepworld.de

(Perhaps you have to clear your browsercache or reload the website)

[Lyudmil Tsvetkov]

-12 elo looks absurd here, there were some worthwhile patches in this period.
maybe some settings went wrong, or it is the book, with some specific patch responsible for SF particularly disliking this book.

[pohl4711]

The problem is, that this Stockfish-version has a measureable higher draw-rate in my testrun and because of this, the number of wins is lower. In the framework, the opening-testsets are producing extremly high draw-rates. So, perhaps, a more drawish playing Stockfish cant be recognized there.
But lets see, how asmFish with the same patches will score. Then we will know, if we have a regression in the Stockfish-code or a statistical “accident“...

Stefan

[Guenther]

-12 elo looks absurd here, there were some worthwhile patches in this period.
maybe some settings went wrong, or it is the book, with some specific patch responsible for SF particularly disliking this book.

Well even at 7000 games you see the error bars are +-7 for both entries.

[JJJ]

The regression test is showing a solid +20 elo on 40K game and I think it is more accurate than others test. Also +17 elo at 180 sec + 1,8 sec.

[Eelco de Groot]

The regression test is showing a solid +20 elo on 40K game and I think it is more accurate than others test. Also +17 elo at 180 sec + 1,8 sec.

If you add 20 points to 3390 which is where Stockfish 8 is in Stefan's list, you get to 3410. That is about the same point you land at interpolating for example Stefan's last 5 data points, the first three are on this line, then one above and the last below the interpolated line. So that would mean Stockfish development version in Stefan's list now (virtually) at 3410 and no real regression but also no big jump with the last patch.

[JJJ]

The regression test is showing a solid +20 elo on 40K game and I think it is more accurate than others test. Also +17 elo at 180 sec + 1,8 sec.

If you add 20 points to 3390 which is where Stockfish 8 is in Stefan's list, you get to 3410. That is about the same point you land at interpolating for example Stefan's last 5 data points, the first three are on this line, then one above and the last below the interpolated line. So that would mean Stockfish development version in Stefan's list now (virtually) at 3410 and no real regression but also no big jump with the last patch.

Yes. And I trust more a regression test with 40K game and a lower error bar.