You asked a controversial question: when are cpu and gpu “equal?” At best it’s difficult if not impossible to answer. How do you define “equal?”
A much more useful question would be “how do I know if matches run on two different machines between gpu and cpu engines are comparable?” A primitive approach has been to determine the nps ratio from the start position for sf8 and 11248 running on lc0 v18. The hypothesis is that if those ratios are the same, then the results would be comparable.
Not everyone has those engines available to them, so it might be both primitive and impractical.
How do you know if your cpu is equal to your gpu?
Moderators: hgm, Rebel, chrisw
-
- Posts: 1631
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: How do you know if your cpu is equal to your gpu?
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: How do you know if your cpu is equal to your gpu?
That of course depends on the curves (performance vs. time or nodes per move) being the same shape, so again there's need for those curves.
And comparable results can still be dangerous, because invariably people will think of them as "comparably fair", not "equally unfair".
-
- Posts: 1631
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: How do you know if your cpu is equal to your gpu?
There was an attempt to create a better “leela ratio” maybe a month ago. It turned out not to be successful. Instead they decided to peg a ratio at that of the current TCEC hardware (which will famously change this season). I’ve tested the ratio hypothesis — that the same ratio on different hardware leads to similar results — on a 1060, a 1070 and a 2070. I’d need many more thousands of games, but I think it’s good enough to upgrade it from a hypothesis to a “thesis.”
Meh. Now you’re getting into the region of religion. The point may be moot when CPU’s begin to pick up the characteristics of GPU’s.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: How do you know if your cpu is equal to your gpu?
In what way was it unsuccessful?
Why hasn't there been more interest in Cscuile's ratio? Obviously it takes some effort to find (if it changes, and if it doesn't you could say that means no real progress), but where an engine goes flatline is what actually matters, not hardware.
-
- Posts: 1631
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: How do you know if your cpu is equal to your gpu?
Too complex and unworkable vs a simple benchmark.
Probably for the same reasons -- too complex and unworkable. Also, the concept it not fully fleshed out and there's no evidence to far that its useful for comparing results across hardware.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: How do you know if your cpu is equal to your gpu?
If one ratio "works" (i.e. allows comparison) across different time controls and hardware, as you suggested before, then another more meaningful ratio should also work.
Leela maxing out its performance at 200000 npm in Cscuile's testing back then is not good.
Testing that just avoids this issue is also not good.
-
- Posts: 1631
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: How do you know if your cpu is equal to your gpu?
More meaningful? Ratio = C * leela nps / sf nps. The only thing that changed was the C. Meh.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: How do you know if your cpu is equal to your gpu?
Yes, more meaningful. Cscuile's ratio is not arbitrary and has something to do with chess and computer chess. The other ratio is arbitrary and has almost nothing to do with chess or computer chess.
-
- Posts: 1631
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: How do you know if your cpu is equal to your gpu?
Well, I’m glad you have a favorite ratio. Cscuile has had so many ratios with so many formulas and acronyms that I couldn’t keep track of them all. But his heart was in the right place.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: How do you know if your cpu is equal to your gpu?
I don't know what all Cscuile's ideas were, so I can't judge them all, just this one.
Belittling him won't change the merits of this ratio, nor will it increase the merits of a largely meaningless one.
What matters most is the (shape of the) curves of performance vs. computation time.
The question is whether Leela (or other engines) still hits the wall at 200000npm.
Belittling him won't change the merits of this ratio, nor will it increase the merits of a largely meaningless one.
What matters most is the (shape of the) curves of performance vs. computation time.
The question is whether Leela (or other engines) still hits the wall at 200000npm.