Leela vs Stockfish nnue style

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

h1a8
Posts: 518
Joined: Fri Jun 04, 2010 7:23 am

Re: Leela vs Stockfish nnue style

Post by h1a8 »

wickedpotus wrote: Mon Nov 27, 2023 6:42 pm
h1a8 wrote: Mon Nov 27, 2023 1:17 pm
The lowest I seen was about 45knps for the ryzen 9 7950x
And about 53knps for ryzen 9 7945x

The m3 max chip scores extremely well on geekbench 6.
Therefore it should correlate stockfish bench scores of comparable to ryzen chips.
So it's definitely not the hardware.

Doing further research I found that decreasing hash to 512mb (down from 1gb) drastically improves knps. For example, this increases the m1 chip (not the max) to about 16knps.
Just scale the m3 max from there
All ARM processors score better on Geekbench than they do on loads like Cinebench, Torch, Stockfish, LC0, etc. Geekbench is an unsuccessful attempt to compare different CPU architectures, which it fails miserably at.

All benchmarks (including Apple’s own) show that the multithreaded performance per core is only 10-15% faster than M2/M1. So there is no magic there, regardless of how many fanboys would like it to be. If you want to run chess programs, the Apple ARMs will neither give you top performance nor decent value for money
But that makes absolutely no sense. Integer calculation is integer calculation. I'm referring to the integer score, not the overall geekbench score.

Also your math is not correct. If m1 chip is 16knps
Then how fast should m3 max be?
Ryzen 7950x is 45knps.
Compare
smatovic
Posts: 3330
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: Leela vs Stockfish nnue style

Post by smatovic »

"Just buy an AMD Ryzen" ;)

--
Srdja
Hai
Posts: 693
Joined: Sun Aug 04, 2013 1:19 pm

Re: Leela vs Stockfish nnue style

Post by Hai »

h1a8 wrote: Tue Nov 28, 2023 3:14 am
wickedpotus wrote: Mon Nov 27, 2023 6:42 pm
h1a8 wrote: Mon Nov 27, 2023 1:17 pm
The lowest I seen was about 45knps for the ryzen 9 7950x
And about 53knps for ryzen 9 7945x

The m3 max chip scores extremely well on geekbench 6.
Therefore it should correlate stockfish bench scores of comparable to ryzen chips.
So it's definitely not the hardware.

Doing further research I found that decreasing hash to 512mb (down from 1gb) drastically improves knps. For example, this increases the m1 chip (not the max) to about 16knps.
Just scale the m3 max from there
All ARM processors score better on Geekbench than they do on loads like Cinebench, Torch, Stockfish, LC0, etc. Geekbench is an unsuccessful attempt to compare different CPU architectures, which it fails miserably at.

All benchmarks (including Apple’s own) show that the multithreaded performance per core is only 10-15% faster than M2/M1. So there is no magic there, regardless of how many fanboys would like it to be. If you want to run chess programs, the Apple ARMs will neither give you top performance nor decent value for money
But that makes absolutely no sense. Integer calculation is integer calculation. I'm referring to the integer score, not the overall geekbench score.

Also your math is not correct. If m1 chip is 16knps
Then how fast should m3 max be?
Ryzen 7950x is 45knps.
Compare
That's because he doesn't know what he is talking about.
He doesn't even can read the m1 to m2 to m3 stuff from the talkchess link which was posted before.
Just buy a MacBook Pro 16-inch M3 MAX. It delivers far more advantages and it is much better.
Sopel
Posts: 391
Joined: Tue Oct 08, 2019 11:39 pm
Full name: Tomasz Sobczyk

Re: Leela vs Stockfish nnue style

Post by Sopel »

Everyone always seems to recommend M# macs for Stockfish but no one ever has Stockfish benchmarks to back it up. Weird.
That's because he doesn't know what he is talking about.
I'm sure you would realize that he does in fact know what he's talking about, should you actually attempt verifying your claims.

Image
dangi12012 wrote:No one wants to touch anything you have posted. That proves you now have negative reputations since everyone knows already you are a forum troll.

Maybe you copied your stockfish commits from someone else too?
I will look into that.
Magnum
Posts: 195
Joined: Thu Feb 04, 2021 10:24 pm
Full name: Arnold Magnum

Re: Leela vs Stockfish nnue style

Post by Magnum »

h1a8 wrote: Mon Nov 27, 2023 1:17 pm
wickedpotus wrote: Sun Nov 26, 2023 10:38 pm
Magnum wrote: Sun Nov 26, 2023 8:12 pm
http://www.talkchess.com/forum3/viewtop ... 31#p951231
64544 - ((30% no ARMv9) 19363) = 45181 kn/s
45.192.261 AMD Ryzen 9 7945HX @4.0Ghz DDR5 5200 CL42 32threads avx512 Ipmanchess W11

M4 MAX will have ARMv9.5 or at least 9.4. Worst case will be 9.2.
A lot of speed improvements are possible on ARM.
I am not sure why you are trying to (dishonestly) talk-up Apple HW here?

It’s crucial to examine the facts:

Modern gaming laptops priced over $1500, equipped with Nvidia GPUs, consistently outperform even the highest-end Macs (M3/M2/M1), which can cost up to $8000, on GPU-bound Chess-engines.

Consider the ipman benchmark results for AMD Ryzen and Apple M CPUs:

AMD Ryzen 9 7945HX @4.0Ghz DDR5 5200 CL42 achieves a score of 79,168,663.
Apple M1 Max @3.23Ghz DDR5 6400 achieves a score of 12,556,848.
Apple M2 @3.23Ghz achieves a score of 7,420,726.

The AMD Ryzen 32-thread score is nearly double that of the Apple M1 Max, and up to 10 times that of the Apple M2.

Furthermore, the price-performance ratio of Apple laptops is worth noting. The M3 Max Pro, priced between $4000 and $7500, is approximately twice as fast as a 10-core M1 Max. Even without official benchmarks, it’s reasonable to estimate a score of around 25knps for the latest MacBook. This suggests that a modern Ryzen 9 gaming laptop could be about three times faster than Apple's top-end, often at a lower price point. These laptops also typically include a GPU better suited for running LC0.
The lowest I seen was about 45knps for the ryzen 9 7950x
And about 53knps for ryzen 9 7945x

The m3 max chip scores extremely well on geekbench 6.
Therefore it should correlate stockfish bench scores of comparable to ryzen chips.
So it's definitely not the hardware.

Doing further research I found that decreasing hash to 512mb (down from 1gb) drastically improves knps. For example, this increases the m1 chip (not the max) to about 16knps.
Just scale the m3 max from there
https://www.topcpu.net/en/cpu-c/apple-m ... i9-13980hx

https://www.cpu-monkey.com/en/compare_c ... i9_13980hx

https://www.topcpu.net/en/cpu-c/apple-m ... 9-7945hx3d

https://www.cpu-monkey.com/en/compare_c ... 9_7945hx3d

https://www.topcpu.net/en/cpu-c/apple-m ... pro-7995wx

https://www.cpu-monkey.com/en/compare_c ... pro_7995wx

https://www.topcpu.net/en/cpu-c/apple-m ... on-x-elite

https://www.cpu-monkey.com/en/compare_c ... on_x_elite
Sopel
Posts: 391
Joined: Tue Oct 08, 2019 11:39 pm
Full name: Tomasz Sobczyk

Re: Leela vs Stockfish nnue style

Post by Sopel »

dangi12012 wrote:No one wants to touch anything you have posted. That proves you now have negative reputations since everyone knows already you are a forum troll.

Maybe you copied your stockfish commits from someone else too?
I will look into that.
Werewolf
Posts: 2034
Joined: Thu Sep 18, 2008 10:24 pm

Re: Leela vs Stockfish nnue style

Post by Werewolf »

Children, children.

I've benched the M2 Ultra (should be at least as fast as the M3 Max) and it got 17Mn/s

Not great performance.

Excellent performance to power ratio though, but I do think this argument is overplayed.
h1a8
Posts: 518
Joined: Fri Jun 04, 2010 7:23 am

Re: Leela vs Stockfish nnue style

Post by h1a8 »

Werewolf wrote: Thu Nov 30, 2023 4:08 pm Children, children.

I've benched the M2 Ultra (should be at least as fast as the M3 Max) and it got 17Mn/s

Not great performance.

Excellent performance to power ratio though, but I do think this argument is overplayed.
Did you use 512mb hash instead of 1024mb?
Did you use stockfish 14.1 nnue?
Did you do the command
"bench 512 24 26 default depth nnue"?

Someone reported that nps increased tremendously (more than double) after decreasing hash from 1gb to 512mb. I'll try to find the thread.
Sopel
Posts: 391
Joined: Tue Oct 08, 2019 11:39 pm
Full name: Tomasz Sobczyk

Re: Leela vs Stockfish nnue style

Post by Sopel »

h1a8 wrote: Fri Dec 01, 2023 1:14 pm
Werewolf wrote: Thu Nov 30, 2023 4:08 pm Children, children.

I've benched the M2 Ultra (should be at least as fast as the M3 Max) and it got 17Mn/s

Not great performance.

Excellent performance to power ratio though, but I do think this argument is overplayed.
Did you use 512mb hash instead of 1024mb?
Did you use stockfish 14.1 nnue?
Did you do the command
"bench 512 24 26 default depth nnue"?

Someone reported that nps increased tremendously (more than double) after decreasing hash from 1gb to 512mb. I'll try to find the thread.
I would be kinda funny if a machine with 8GB of RAM couldn't do 1GB of hash. If so then it's pretty lucky that ipman's test accounts for this.
dangi12012 wrote:No one wants to touch anything you have posted. That proves you now have negative reputations since everyone knows already you are a forum troll.

Maybe you copied your stockfish commits from someone else too?
I will look into that.
h1a8
Posts: 518
Joined: Fri Jun 04, 2010 7:23 am

Re: Leela vs Stockfish nnue style

Post by h1a8 »

Sopel wrote: Fri Dec 01, 2023 10:32 pm
h1a8 wrote: Fri Dec 01, 2023 1:14 pm
Werewolf wrote: Thu Nov 30, 2023 4:08 pm Children, children.

I've benched the M2 Ultra (should be at least as fast as the M3 Max) and it got 17Mn/s

Not great performance.

Excellent performance to power ratio though, but I do think this argument is overplayed.
Did you use 512mb hash instead of 1024mb?
Did you use stockfish 14.1 nnue?
Did you do the command
"bench 512 24 26 default depth nnue"?

Someone reported that nps increased tremendously (more than double) after decreasing hash from 1gb to 512mb. I'll try to find the thread.
I would be kinda funny if a machine with 8GB of RAM couldn't do 1GB of hash. If so then it's pretty lucky that ipman's test accounts for this.
Someone posted their results from another site. It shows the nps increasing significantly when lowering the hash to 512. I have no skin in the game and asking someone to verify this claim.