How do you know if your cpu is equal to your gpu?

[JohnW]

I am hoping the experts here can educate me. When comparing a cpu based engine like Stockfish 11 to a gpu based engine like Lc0 how do you know that your cpu is equal to your gpu and the games will be on a level playing field? If one piece of hardware is better than the other then the results are meaningless right? My PC runs an i7-6700K cpu\16GB RAM and has a Nvidia GTX1080Ti, how do I know those two pieces of hardware make for a fair game?

[mwyoung]

I am hoping the experts here can educate me. When comparing a cpu based engine like Stockfish 11 to a gpu based engine like Lc0 how do you know that your cpu is equal to your gpu and the games will be on a level playing field? If one piece of hardware is better than the other then the results are meaningless right? My PC i7 runs an i7-6700K cpu\16GB RAM and has a Nvidia GTX1080Ti, how do I know those two pieces of hardware make for a fair game?

No, the results are not meaningless.

And this has always been a difficult question. One way to compare that is used is called the Leela ratio. Google Leela ratio for details. On my testing setup. My Leela ratio is 1.1 for example. And the Alpha Zero vs Stockfish match is the geniuses of the Leela ratio. And would be 1.0

[Ozymandias]

Not an easy question to answer. The easiest way would be to indicate the CPU and GPU used, as if it were a surname: Lc0 0.22.0 T40B.4-160 GTX1050 (like they already do at CCRL), but also Stockfish 2019-10-09 64-bit 4CPU i7 2600K HT off (for example). This wouldn't be feasible for rating lists with more than one computer model testing engines, but for an individual using just one PC, it would be informative enough.

People might complain tough, depending which kind of engines they favor, which is why I support testing that brings cost (TCO) to the spotlight. This of course, present problems like price fluctuation (both temporal and geographical), not just of the components themselves, but of the electricity they need. If one'd be able to establish an average price for HW and Watts, a table could be compiled. Still, we'd first need to find a fair moment to even begin to compile such a list, for example, right now with Ampere around the corner, any GPU will be at a disadvantage, but if you wait until it comes out, then how about waiting for Zen 3? You see the problem.

Some components will be needed to operate either an AB engine or a NN. In both cases you need: monitor, keyboard, mouse, case, PSU, HDD and RAM. These components are of no consequence when comparing. The motherboard on the other hand, should be given extra attention.

For AB engines, you want a strong processor, which means a good MOBO with quality VRM. If you plan on buying with a NN in mind, you can save money on this front, as a low power processor will do the job just fine and the graphics card's VRM, will take care of cleaning up the current delivered to the GPU.

Of course, you finally have to look at prices for CPU and GPU, but they aren't exclusive. As mentioned, the NN will also use CPU cores (although not as many as you need for AB), but inversely, the AB engine will also need a GPU, even if just for 2D output. The comparison could be summed up like this:

(Cheap CPU + cheap MOBO + expensive GPU) vs (expensive CPU + expensive MOBO + cheap GPU)

[JohnW]

I am hoping the experts here can educate me. When comparing a cpu based engine like Stockfish 11 to a gpu based engine like Lc0 how do you know that your cpu is equal to your gpu and the games will be on a level playing field? If one piece of hardware is better than the other then the results are meaningless right? My PC i7 runs an i7-6700K cpu\16GB RAM and has a Nvidia GTX1080Ti, how do I know those two pieces of hardware make for a fair game?

No, the results are not meaningless.

And this has always been a difficult question. One way to compare that is used is called the Leela ratio. Google Leela ratio for details. On my testing setup. My Leela ratio is 1.1 for example. And the Alpha Zero vs Stockfish match is the geniuses of the Leela ratio. And would be 1.0

Ok, I guess I was thinking of decades ago when you might match two engines that have their own gui's using two separate PC's.
I can remember matches between Rebel 8 vs MChess 5 but on running on separate PC's and I would enter the moves in manually for each game.
Back then they would say if you're two PC's were not running identical hardware then the results were not valid. I was thinking the same would be true today if the cpu was not equal to the gpu.

[mwyoung]

I am hoping the experts here can educate me. When comparing a cpu based engine like Stockfish 11 to a gpu based engine like Lc0 how do you know that your cpu is equal to your gpu and the games will be on a level playing field? If one piece of hardware is better than the other then the results are meaningless right? My PC i7 runs an i7-6700K cpu\16GB RAM and has a Nvidia GTX1080Ti, how do I know those two pieces of hardware make for a fair game?

No, the results are not meaningless.

And this has always been a difficult question. One way to compare that is used is called the Leela ratio. Google Leela ratio for details. On my testing setup. My Leela ratio is 1.1 for example. And the Alpha Zero vs Stockfish match is the geniuses of the Leela ratio. And would be 1.0

Ok, I guess I was thinking of decades ago when you might match two engines that have their own gui's using two separate PC's.
I can remember matches between Rebel 8 vs MChess 5 but on running on separate PC's and I would enter the moves in manually for each game.
Back then they would say if you're two PC's were not running identical hardware then the results were not valid. I was thinking the same would be true today if the cpu was not equal to the gpu.

No, that was not true ever. The games are always valid, if you have a common rating pool. And describe what hardware the programs were using to play the game. It is no different then how we rated dedicated chess machine in the past playing them by hand. All the dedicated chess machines had vastly different cpu and memory configurations.

[CMCanavessi]

You don't, because they are not and will never be the same.

[jp]

Back then they would say if your two PC's were not running identical hardware then the results were not valid. I was thinking the same would be true today if the cpu was not equal to the gpu.

You could of course run SF on a CPU against Lc0 on the same CPU, but then two things would be certain: Lc0 would be crushed, and Lc0 fans would be outraged that you ran the CPU version of Lc0.

[Ovyron]

Lc0 fans would be outraged that you ran the CPU version of Lc0.

Why? If it's the only version I can run? If they want me to run GPU Leela they should get me one that doesn't need CUDA

[CMCanavessi]

Lc0 fans would be outraged that you ran the CPU version of Lc0.

Why? If it's the only version I can run? If they want me to run GPU Leela they should get me one that doesn't need CUDA

Same reason SF fans would be outraged if I ran stockfisn on my 486 DX4-100 vs. Lc0 on my RTX 2080TI.

[jp]

Same reason SF fans would be outraged if I ran stockfisn on my 486 DX4-100 vs. Lc0 on my RTX 2080TI.

But can you actually run your RTX 2080Ti on the machine with the 486 DX4-100?
Otherwise, you're deliberately withholding the CPU of the machine where you do run your RTX 2080Ti from SF.