No +100 Elo for SF on Windows for me

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Dann Corbit
Posts: 12615
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: No +100 Elo for SF on Windows for me

Post by Dann Corbit »

It may be that the full 100 Elo is available.
It suddenly occurs to me that CuteChess is launching the programs single threaded.
So NUMA optimizations will have no effect.
I would need to run an SMP contest to have some idea what the NUMA Elo gain is.

So...
Pilot error
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Dann Corbit
Posts: 12615
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: No +100 Elo for SF on Windows for me

Post by Dann Corbit »

I am running a contest with 30 threads right now.
After 16 games we have 16 draws.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
AndrewGrant
Posts: 1819
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: No +100 Elo for SF on Windows for me

Post by AndrewGrant »

"Fixing -100 slowdown" != "+100 speedup". Also top kek that Albert Silver is here.
Friendly reminder that stealing is a crime, is wrong, and makes you a thief.
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
AndrewGrant
Posts: 1819
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: No +100 Elo for SF on Windows for me

Post by AndrewGrant »

A proper explanation, which maybe someone will read:

As time passes, Stockfish has used more and more memory, the primary point of interest being the ever increasing size of the Network files. It was noticed that Stockfish suffers an extreme slowdown from expectation at TCEC. The thought was that this was being caused by memory contention issues, on machines with multiple NUMA nodes.

You might imagine your computer as having some threads, some memory, and some caches. And that all threads can very cleanly access all caches and all memory locations -- a very nice sharing. This is not entirely true, for example you might have a microarchitecture where the L3 caches are shared by certain blocks of threads. Or perhaps the two threads that comprise a physical core might share an L2. But for the most part, we can pretend, and things work out pretty well. Unfortunately this picture of our CPU crashes and burns quite fast when you introduce multiple NUMA nodes, most often in the form of having multiple CPUs. Server motherboards/configurations often have 2, 4, or sometimes 8 CPUs in a single machine. Namely, there is a cost of sharing memory across NUMA nodes.

Okay so now lets make things very simple:

Who is impacted?
People who have machines with multiple NUMA nodes -- sometimes. CCC machine for example seems not to have much of an issue, but TCEC does. If you have a typical consumer CPU, this does not impact you.

What is the fix?
Making sure that machines with NUMA nodes have multiple copies of the NNUE weights ( and ensure threads are bound to said nodes )

How much of an improvement is it?
Its a bit of an improvement in the general sense, but its more importantly a sort of "bug fix" for machines that experience massive slowdowns. Stockfish very gradually over time lost {{insert your elo value}} on certain NUMA systems over time. Now, that is clawed back. Possibly in full, and possibly a bit more.

How can I tell if I was impacted?
Playing games won't be a good way to do this. Certainly not if you don't understand proper testing methodologies. The difference here is extreme enough that you can show the gains by simply benching Stockfish. You'll want to take the NUMA version and compare it to a version before then.
Friendly reminder that stealing is a crime, is wrong, and makes you a thief.
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
Albert Silver
Posts: 3026
Joined: Wed Mar 08, 2006 9:57 pm
Location: Rio de Janeiro, Brazil

Re: No +100 Elo for SF on Windows for me

Post by Albert Silver »

Uri Blass wrote: Thu May 30, 2024 9:48 am
Albert Silver wrote: Thu May 30, 2024 5:19 am
Dann Corbit wrote: Thu May 30, 2024 12:58 am Since there are no windows builds, I made one using the current code.
The NUMA stuff was compiled in, since I had the NUMA option for NUMA policy, which was set to auto in the Arena GUI.
I ran 2000 games between the last released Windows version which has no NUMA and the current source code base.
The result was +10 Elo +/- 5 Elo.
They did mention that Windows 10 has "partial" implementation of NUMA.
It is worth mentioning that the 104 Elo calculation is based on 278 games, not tens of thousands.

Total: 278 W: 110 L: 29 D: 139 Elo +104.25

So it may not be exact...
I do not understand what you are talking about.

I never read about the 104 elo calculation based on 278 games and I do not know what is the source for it.

I only read a thread with the title no +100 elo for SF when I never expected +100 elo for stockfish.
I think it is better to explain first what is the reason to expect this type of result.

If the claim is that people got
Total: 278 W: 110 L: 29 D: 139 Elo +104.25 then it is better to give a link to the result and not only to give numbers here when I need to search for the source.

Edit:I now read the second thread that gives this link

https://tests.stockfishchess.org/tests/ ... d5e27db0d8
Actually, I simply quoted the entry from Abrok.eu. It never occurred to me this was not obvious.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
Albert Silver
Posts: 3026
Joined: Wed Mar 08, 2006 9:57 pm
Location: Rio de Janeiro, Brazil

Re: No +100 Elo for SF on Windows for me

Post by Albert Silver »

AndrewGrant wrote: Fri May 31, 2024 1:59 am "Fixing -100 slowdown" != "+100 speedup". Also top kek that Albert Silver is here.
Moderation.

Removed insult.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
Viz
Posts: 109
Joined: Tue Apr 09, 2024 6:24 am
Full name: Michael Chaly

Re: No +100 Elo for SF on Windows for me

Post by Viz »

Albert Silver wrote: Fri May 31, 2024 7:13 pm
AndrewGrant wrote: Fri May 31, 2024 1:59 am "Fixing -100 slowdown" != "+100 speedup". Also top kek that Albert Silver is here.
Moderation.

Removed insult.

Moderation.

Removed insult.
User avatar
Eelco de Groot
Posts: 4596
Joined: Sun Mar 12, 2006 2:40 am
Full name:   

Re: No +100 Elo for SF on Windows for me

Post by Eelco de Groot »

I was not here, at least not often, when all this happened with Chessbase but Dann Corbit resigned over the treatment of it in this forum so I get it was not pleasant and probably not fair either. For most of us, Albert goes a long way back to somewhere in the past century, 1999 or even further before CCC was founded. Someone here wrote once that Albert was the person she would like to meet the most in person 8-) For me that is enough of a validation of Albert, he can't be all that bad. He is a good person at heart.

Thanks Andrew by the way for the explanation of NUMA in this thread that's appreciated.

Last two entries in this thread not so much.
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
Dann Corbit
Posts: 12615
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: No +100 Elo for SF on Windows for me

Post by Dann Corbit »

I appreciate your civil reply Eelco.
Name calling and mud-slinging accomplish no useful purpose.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
User avatar
Graham Banks
Posts: 41990
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: No +100 Elo for SF on Windows for me

Post by Graham Banks »

Eelco de Groot wrote: Fri May 31, 2024 9:38 pm I was not here, at least not often, when all this happened with Chessbase but Dann Corbit resigned over the treatment of it in this forum so I get it was not pleasant and probably not fair either. For most of us, Albert goes a long way back to somewhere in the past century, 1999 or even further before CCC was founded. Someone here wrote once that Albert was the person she would like to meet the most in person 8-) For me that is enough of a validation of Albert, he can't be all that bad. He is a good person at heart.

Thanks Andrew by the way for the explanation of NUMA in this thread that's appreciated.

Last two entries in this thread not so much.
I agree 100%.
gbanksnz at gmail.com