Stockfish and latest +6 ELO patch!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Jouni
Posts: 2121
Joined: Wed Mar 08, 2006 7:15 pm

Stockfish and latest +6 ELO patch!

Post by Jouni » Thu Mar 05, 2020 8:07 am

LTC (This exact version. . . more simplified)
LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
Total: 5392 W: 729 L: 631 D: 4032 Elo +6.32

This is endgame patch and it's difficult to believe it's much/any gain there. And NCM test from same patch gives -5 ELO! Don't expect SF 12 before 2022 :D .
Jouni

User avatar
Ozymandias
Posts: 1171
Joined: Sun Oct 25, 2009 12:30 am

Re: Stockfish and latest +6 ELO patch!

Post by Ozymandias » Thu Mar 05, 2020 10:38 am

If we're talking the usual 50 Elo increase between releases, I'd say that well into 2022 by the looks of it:

Image

The graph covers data from ncm spanning more than 6 months and shows an increase of less than 10 Elo points for that time period.

zullil
Posts: 6071
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Stockfish and latest +6 ELO patch!

Post by zullil » Thu Mar 05, 2020 11:09 am

Jouni wrote:
Thu Mar 05, 2020 8:07 am
LTC (This exact version. . . more simplified)
LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
Total: 5392 W: 729 L: 631 D: 4032 Elo +6.32

This is endgame patch and it's difficult to believe it's much/any gain there. And NCM test from same patch gives -5 ELO! Don't expect SF 12 before 2022 :D .
The patch apparently replaces two arrays of values with linear formulas. Hard to believe that this alters Elo significantly in either direction.

https://github.com/official-stockfish/S ... c26L57-R59

Dann Corbit
Posts: 10742
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Stockfish and latest +6 ELO patch!

Post by Dann Corbit » Thu Mar 05, 2020 7:39 pm

I'm not surprised.

A memory lookup is expensive. If you can calculate the same thing from a formula it will be much faster. If it is something that happens a lot, you could see a big benefit.

Bench numbers tell the tale.
5123316/4923286 = 4% faster for the entire program. That is amazing.

Since it is an endgame change, I guess that endgame speedup will be even more dramatic than the average.

It is clear to me that the NCM measurement is the one that is buggy. There really isn't a functional change and the new version is much faster.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

User avatar
Eelco de Groot
Posts: 4236
Joined: Sun Mar 12, 2006 1:40 am
Location: Groningen

Re: Stockfish and latest +6 ELO patch!

Post by Eelco de Groot » Thu Mar 05, 2020 9:00 pm

Maybe in some endgames there are big tuning changes still waiting, and this functionally changes just something right? As Protospring is saying, the bench right now is not really affected by a lot of endgame changes, they are not triggered in the searched positions. But it is a functional change from the change in benchmark. But that is not a speedtest so you can't tell nodes per second from it? It will be interesting what Stefan Pohl's testing may say about performance against other programs. Can't test much myself right now, maybe someone is interested in testing some endgames?
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan

Jouni
Posts: 2121
Joined: Wed Mar 08, 2006 7:15 pm

Re: Stockfish and latest +6 ELO patch!

Post by Jouni » Thu Mar 05, 2020 9:15 pm

"Bench numbers tell the tale. 5123316/4923286 = 4% faster for the entire program. That is amazing." Really Dann I don't believe this 4%. Bench is not nps!
Jouni

Dann Corbit
Posts: 10742
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Stockfish and latest +6 ELO patch!

Post by Dann Corbit » Thu Mar 05, 2020 9:37 pm

Fewer nodes is equally good.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Alayan
Posts: 252
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: Stockfish and latest +6 ELO patch!

Post by Alayan » Thu Mar 05, 2020 10:39 pm

Did you actually interpret the node count of bench as meaningful for strength ?

Extremely minor changes that have about 0 elo impact can impact bench's nodes count by more than 10%.

While massive differences in bench node count are indicative of different amounts of pruning and re-searching, minor differences between close versions are pseudo-random.

Post Reply