How do you know if your cpu is equal to your gpu?

[dkappe]

You guys are amusing. Cheap as chips gtx1060 Desktop which was $900 a few years ago does 5k nps on a recent t40 net. A more recent desktop with a rtx2060 goes for $1000 and does about 20k nps. When CPU’s start incorporating GPU features what sort of rationalizations will you then employ?

[Ovyron]

I don't know. I haven't checked DirectX Leela yet, waiting for a proper release. Perhaps then my "Leela ratio" will make a leap. A graph that shows when Leela nodes and Stockfish nodes cross up would help (maybe the Leela ratio should just be calculated where the lines cross, as Leela plays stronger than Stockfish without needing so many nodes.)

[jp]

No, what matters most is just the shape of the graphs of performance versus time showing long-time behavior.

Getting the curves needs some choices about how to do it, and obviously enough computer time, so it's not easy, but it is necessary.

As for the other ideas for ratios (most of them old and often proposed), we now have rough numbers for them. Those ideas aren't my first preference, but it's not our problem if someone doesn't like the numbers that came out.

[Ovyron]

It should be easy enough: Make a graph showing time control vs. hardware. At the point you'd perform the same when doubling the time control, that's a 0.5 ratio, if you can perform the same as halving the time control, that's a ratio of 2.

Just look at an equivalence that would give a 50% performance between Stockfish at some fixed nodes per second vs. Leela at 2 minutes/game (could have +1 sec increment, which sounds reasonable), call that a 1.0 ratio, and go from there.

Hardware and energy costs don't get in the calculations, we assume infinite wealth.

[jp]

It sounds like you want to find the hardware that'll make the two engines equal in strength.

What I want is to find the asymptotic behavior of each engine.

[mmt]

On second thoughts, it seems I should have added to the GPU energy cost the energy cost of some CPU because it still needs a CPU. It looks like most CPUs are at least 65W (even those you would not want to use with the RTX 2080Ti), so

You can put in multiple GPUs on quite a few motherboards, though. So adding the full 65W is not fair. But I think you're right and this evidence is enough to show that the Leela ratio of 875 is now outdated and it should be something like 1800. All thanks to new AMDs. If these rumors https://www.techradar.com/news/nvidias- ... ver-turing turn out to be true though, then the picture will change a lot.

[Ovyron]

It sounds like you want to find the hardware that'll make the two engines equal in strength.

Wasn't this what this thread was about? Do that. Then see what CPU is equal to some GPU. You're done.

[jp]

Wasn't this what this thread was about? Do that. Then see what CPU is equal to some GPU. You're done.

No. The OP wanted to know what is "fair", i.e. hardware that doesn't favor one side, not "hardware handicapping" like horse handicapping to achieve equal chess outcomes.
e.g.

Back then they would say if your two PC's were not running identical hardware then the results were not valid.

He wants to be able to say a horse is better because it won and it's not just the handicap or a bad barrier draw.


Of course, people have argued about what "fair" means from the start. That's why there are multiple ideas. And we've now seen the rough ratios that come from those ideas currently (except the most important thing: the scaling curves).

[Ovyron]

The OP wanted to know what is "fair", i.e. hardware that doesn't favor one side

Finding the hardware that'll make the two engines equal in strength will achieve just that. After you do what I say, you'll be able to look at the graph, figure out what's your CPU, and then look what GPU would give you a ratio of 1.0, or figure out what's your GPU, and then look what CPU would give you a ratio of 1.0.

Or see what you already have and see a ratio <1.0 (that's unfair to the CPU), or a ratio >1.0 (that's unfair to the GPU.)

This is simple enough (...with the caveat that this would only be relevant for 2 minute/game time controls, but once you do this it ought to be easier to extrapolate what would happen with slower time controls, perhaps get a graph that tells you how much of a Stockfish node is equivalent to a Leela node. Is there something graph can't do?)

[jp]

Getting an equal outcome (e.g. 50% score in a match) when the engines are not equal in strength (which is what he's assuming for any two different engines) is exactly what the OP does not want. He does not want equal outcomes. Isn't your "1.0" meaning equal outcomes, or is your "1.0" supposed to mean something else?