Stockfish 11

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Alayan
Posts: 431
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: Stockfish 11

Post by Alayan » Sat Jan 18, 2020 8:37 pm

Modern Times wrote:
Sat Jan 18, 2020 6:38 pm
I think it might be better than that on the ratings lists, but let's see. The error margins are big with a small-ish number of games, so not conclusive, but Stockfish 2019-12-10 was already about +70 Elo over Stockfish 10 at CCRL 40/15.
Small sample size is plaguing CCRL. The few hundreds of games one engine usually gets, against a variety of opponents, is already producing huge error bars.

A sample of a few dozen games can be outright disregarded for the purpose of knowing if the gain is rather +30, +50 or +70 ; the data is way too unreliable.

Modern Times
Posts: 2604
Joined: Thu Jun 07, 2012 9:02 pm

Re: Stockfish 11

Post by Modern Times » Sat Jan 18, 2020 9:39 pm

Alayan wrote:
Sat Jan 18, 2020 8:37 pm

A sample of a few dozen games can be outright disregarded for the purpose of knowing if the gain is rather +30, +50 or +70 ; the data is way too unreliable.
I wouldn't call nearly 700 games a few dozen...
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Alayan
Posts: 431
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: Stockfish 11

Post by Alayan » Sat Jan 18, 2020 9:59 pm

Modern Times wrote:
Sat Jan 18, 2020 9:39 pm
I wouldn't call nearly 700 games a few dozen...
Point taken, because that version didn't show up in the main list when I looked it up (and I missed it in the SF list between Sugar and other SF versions), I thought it was below the threshold that CCRL uses to display the results from a new version. My mistake.

684 games is more in keeping with what is usually seen at CCRL.

Nonetheless, when you deal with SPRT tests, you quickly learn that a result that appears to be really good or really bad after 1K games is more often than not just a fluke.

In this case, this is compounded by other elements :
  • Games against much weaker opponents hold less informative value (that is, 100 games against an equal opponent reduce the error bars much more than 100 games against a 200 elo weaker opponent), and most games in the testing were done against opponents that scored less than 30%.
  • This isn't a head to head vs SF10 obviously, so the inaccuracy of SF10's rating (over or underrated) will make a difference measurement less reliable.
  • The mix of opponents is different. Just look at SF9 and SF10 being almost tied at CCRL blitz, despite SF10 having a much better score against their shared opponents.
So while the +71 difference between 1 core SF10 and 1 core SF-19-12-10 looks good at first sight, I still don't think it's a reliable estimate. It proves the new version is clearly stronger, though.

SPCC, which uses ordo rather than bayeselo, shows around 50 elo above SF10 for its recent SF version tests. Usually, bayeselo compresses ratings, so +70 at CCRL would be very unexpected.

supersharp77
Posts: 917
Joined: Sat Jul 05, 2014 5:54 am
Location: Southwest USA

Re: Stockfish 11

Post by supersharp77 » Sun Jan 19, 2020 8:02 pm

snicolet wrote:
Sat Jan 18, 2020 12:59 am
It is our pleasure to release Stockfish 11 to our fans and supporters.

Downloads are freely available at http://stockfishchess.org/download/

This version 11 of Stockfish is 50 Elo stronger than the last version, and
150 Elo stronger than the version which famously lost a match to AlphaZero
two years ago. This makes Stockfish the strongest chess engine running on
your smartphone or normal desktop PC, and we estimate that on a modern four
cores CPU, Stockfish 11 could give 1:1000 time odds to the human chess champion
having classical time control, and be on par with him. More specific data,
including nice cumulative curves for the progression of Stockfish strength
over the last seven years, can be found on [our progression page][1], at
[Stefan Pohl site][2] or at [NextChessMove][3].

In October 2019 Stockfish has regained its crown in the TCEC competition,
beating in the superfinal of season 16 an evolution of the neural-network
engine Leela that had won the previous season. This clash of style between an
alpha-beta and an neural-network engine produced spectacular chess as always,
with Stockfish [emerging victorious this time][0].

Compared to Stockfish 10, we have made hundreds of improvements to the
[codebase][4], from the evaluation function (improvements in king attacks,
middlegame/endgame transitions, and many more) to the search algorithm (some
innovative coordination methods for the searching threads, better pruning of
unsound tactical lines, etc), and fixed a couple of bugs en passant.

Our testing framework [Fishtest][5] has also seen its share of improvements
to continue propelling Stockfish forward. Along with a lot of small enhancements,
Fishtest has switched to new SPRT bounds to increase the chance of catching Elo
gainers, along with a new testing book and the use of pentanomial statistics to
be more resource-efficient.

Overall the Stockfish project is an example of open-source at its best, as
its buzzing community of programmers sharing ideas and daily reviewing their
colleagues' patches proves to be an ideal form to develop innovative ideas for
chess programming, while the mathematical accuracy of the testing framework
allows us an unparalleled level of quality control for each patch we put in
the engine. If you wish, you too can help our ongoing efforts to keep improving
it, just [get involved][6] :-)

Stockfish is also special in that every chess fan, even if not a programmer,
[can easily help][7] the team to improve the engine by connecting their PC to
Fishtest and let it play some games in the background to test new patches.
Individual contributions vary from 1 to 32 cores, but this year Bojun Guo
made it a little bit special by plugging a whole data center during the whole
year: it was a vertiginous experience to see Fishtest spikes with 17466 cores
connected playing [25600 games/minute][8]. Thanks Guo!

The Stockfish team

[0]: <http://mytcecexperience.blogspot.com/20 ... 1-100.html>
[1]: <https://github.com/glinscott/fishtest/w ... sion-Tests>
[2]: <https://www.sp-cc.de/index.htm>
[3]: <https://nextchessmove.com/dev-builds>
[4]: <https://github.com/official-stockfish/Stockfish>
[5]: <https://tests.stockfishchess.org/tests>
[6]: <https://stockfishchess.org/get-involved/>
[7]: <https://github.com/glinscott/fishtest/wiki>
[8]: <https://groups.google.com/forum/?fromgr ... %5B1-25%5D>
Congratulations Stockfish Team And The Legendary Snicolet...Loco...Voyager...Visvez...Etc.....Great Work Guys!!
.........THE LEGEND ENDURES........'STOCKFISH 11'....... :D :wink:

stavros
Posts: 160
Joined: Tue Dec 02, 2014 12:29 am

Re: Stockfish 11

Post by stavros » Mon Jan 20, 2020 8:40 pm

snicolet wrote:
Sat Jan 18, 2020 12:59 am
It is our pleasure to release Stockfish 11 to our fans and supporters.

Downloads are freely available at http://stockfishchess.org/download/

This version 11 of Stockfish is 50 Elo stronger than the last version,....
let me disagree. that elo was from sf vs sf at bullet tc
i expect the "disappointing" 35-40+ elo vs others engines and diff. TC

dedolletoren
Posts: 1
Joined: Wed Feb 19, 2020 6:42 pm
Full name: Symon Riedstra

Re: Stockfish 11

Post by dedolletoren » Thu Feb 20, 2020 8:28 am

Not burdened with any knowledge on how chess engines work, I wondered how it is possible that for a simple puzzle (white Kh1, Qg6, Rf1, g7, h2 and black Kg8, Qd7, Re8 and h3) both stockfish 10 and 11 quickly find the main idea (Rf8+, Rxf8, Qf7+ Kxf7, xf8:N) with winning endgame but that stockfish 10 finds (at depth=46) finds that it is mate in #51 whereas stockfish 11 (at depths 46, 47, 48, 49) evaluates this as +68,18. (by the way it is a tablebase win with mate in 26).

Werewolf
Posts: 1314
Joined: Thu Sep 18, 2008 8:24 pm

Re: Stockfish 11

Post by Werewolf » Thu Feb 20, 2020 2:58 pm

Which is the best version of Stockfish 11 to use with the new Ryzen CPU?

Is it the BMI?

Alayan
Posts: 431
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: Stockfish 11

Post by Alayan » Thu Feb 20, 2020 3:18 pm

The BMI version should absolutely be avoided on ryzens, the popcount version should be used instead.
dedolletoren wrote:
Thu Feb 20, 2020 8:28 am
Not burdened with any knowledge on how chess engines work, I wondered how it is possible that for a simple puzzle (white Kh1, Qg6, Rf1, g7, h2 and black Kg8, Qd7, Re8 and h3) both stockfish 10 and 11 quickly find the main idea (Rf8+, Rxf8, Qf7+ Kxf7, xf8:N) with winning endgame but that stockfish 10 finds (at depth=46) finds that it is mate in #51 whereas stockfish 11 (at depths 46, 47, 48, 49) evaluates this as +68,18. (by the way it is a tablebase win with mate in 26).
Stockfish 11 contains a mate-finding regression that has been fixed in the dev version.

corres
Posts: 3500
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Stockfish 11

Post by corres » Thu Feb 20, 2020 4:37 pm

snicolet wrote:
Sat Jan 18, 2020 12:59 am
...
Compared to Stockfish 10, we have made hundreds of improvements to the
[codebase][4], from the evaluation function (improvements in king attacks,
middlegame/endgame transitions, and many more) to the search algorithm (some
innovative coordination methods for the searching threads, better pruning of
unsound tactical lines, etc), and fixed a couple of bugs en passant.

Our testing framework [Fishtest][5] has also seen its share of improvements
to continue propelling Stockfish forward. Along with a lot of small enhancements,
Fishtest has switched to new SPRT bounds to increase the chance of catching Elo
gainers, along with a new testing book and the use of pentanomial statistics to
be more resource-efficient.

Overall the Stockfish project is an example of open-source at its best, as
its buzzing community of programmers sharing ideas and daily reviewing their
colleagues' patches proves to be an ideal form to develop innovative ideas for
chess programming, while the mathematical accuracy of the testing framework
allows us an unparalleled level of quality control for each patch we put in
the engine. If you wish, you too can help our ongoing efforts to keep improving
it, just [get involved][6] :-)
...
The Stockfish team

[0]: <http://mytcecexperience.blogspot.com/20 ... 1-100.html>
[1]: <https://github.com/glinscott/fishtest/w ... sion-Tests>
[2]: <https://www.sp-cc.de/index.htm>
[3]: <https://nextchessmove.com/dev-builds>
[4]: <https://github.com/official-stockfish/Stockfish>
[5]: <https://tests.stockfishchess.org/tests>
[6]: <https://stockfishchess.org/get-involved/>
[7]: <https://github.com/glinscott/fishtest/wiki>
[8]: <https://groups.google.com/forum/?fromgr ... %5B1-25%5D>
Many thanks for every each helper and you go further on this way.

User avatar
CMCanavessi
Posts: 907
Joined: Thu Dec 28, 2017 3:06 pm
Location: Argentina

Re: Stockfish 11

Post by CMCanavessi » Thu Feb 20, 2020 5:08 pm

Werewolf wrote:
Thu Feb 20, 2020 2:58 pm
Which is the best version of Stockfish 11 to use with the new Ryzen CPU?

Is it the BMI?
No, use modern (aka popcount).
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls

Post Reply