NN Engine GPU Usage

[Stephen Ham]

Hello All,

I just purchased a new PC with dual NVIDIA GeForce RTX 2080 TI GPUs in order to maximize NN engine performance. However, when running Leela (Fritz 17 version) in Fritz GUI, Task Manager shows only 15%-16% GPU usage.

Switching to Fat Fritz, the GPU usage drops to 12%.

Is this normal or do I need better configurations? Yes, I have Leela's Backend set as: cudnn-fp16 with 2 CPU threads. Switching the Backend to multiplex and 4 CPU threads, the GPU usage drops to 8%, but the kN/s increases substantially.

My perception is that performance is superior with the former Backend rather than the latter, but I have not tested them in head-to-head competition.

ChessBase promised configuration information back on 11/13, but to date have provided only silence. Here's the quote:
macauley 11/13/2019 07:58
@EatMyShorts - We're going to have a follow-up article discussing this in more detail in the next day or so.

Surely they must have several frustrated clients...like me.

Thoughts, gentlemen?

All the best,
-Steve-

[George Tsavdaris]

Hello All,

I just purchased a new PC with dual NVIDIA GeForce RTX 2080 TI GPUs in order to maximize NN engine performance. However, when running Leela (Fritz 17 version) in Fritz GUI, Task Manager shows only 15%-16% GPU usage.

Switching to Fat Fritz, the GPU usage drops to 12%.

Is this normal or do I need better configurations? Yes, I have Leela's Backend set as: cudnn-fp16 with 2 CPU threads. Switching the Backend to multiplex and 4 CPU threads, the GPU usage drops to 8%, but the kN/s increases substantially.

With 2080 Ti you should ALWAYS use fp16.
Multiplex is irrelevant though if you have only 1 GPU. Multiplex is when you want to use more than 1 GPU (e.g 2,3,4 etc).

Second you say "task manager". But in there you need to check:
Performance -> GPU0(where 2080 Ti is) -> and now choose a ↓ down arrow in any of the setting, e.g Video Decode and choose Compute_0. That shows the Leela GPU usage and not the 3D that you probably see and confuse you.
So i'm sure all are fine.

[corres]

Hello All,
I just purchased a new PC with dual NVIDIA GeForce RTX 2080 TI GPUs in order to maximize NN engine performance. However, when running Leela (Fritz 17 version) in Fritz GUI, Task Manager shows only 15%-16% GPU usage.
Switching to Fat Fritz, the GPU usage drops to 12%.
Is this normal or do I need better configurations? Yes, I have Leela's Backend set as: cudnn-fp16 with 2 CPU threads. Switching the Backend to multiplex and 4 CPU threads, the GPU usage drops to 8%, but the kN/s increases substantially.
My perception is that performance is superior with the former Backend rather than the latter, but I have not tested them in head-to-head competition.
ChessBase promised configuration information back on 11/13, but to date have provided only silence. Here's the quote:
macauley 11/13/2019 07:58
@EatMyShorts - We're going to have a follow-up article discussing this in more detail in the next day or so.
Surely they must have several frustrated clients...like me.
Thoughts, gentlemen?
All the best,
-Steve-

If you want to know the real GPU usage you should use GPU-Z and not Task Manager.
You can download GPU-Z from techpowerup.com the last version is 2.27. This utility gives you every important info from your GPUs.
For Backend the Multiplexing with 4 Threads gives the best result (in general) and for BackendOptions
backend=cudnn-fp16,(gpu=0),(gpu=1) is O.K.

[Nordlandia]

Some say 3 is better than 4.

TCEC use 3 threads.

[corres]

Some say 3 is better than 4.
TCEC use 3 threads.

For Backend = Demux is enough 3 threads (TCEC params) but it is slower than Multiplexing with 4 threads.

[Ozymandias]

Performance -> GPU0(where 2080 Ti is) -> and now choose a ↓ down arrow in any of the setting, e.g Video Decode and choose Compute_0. That shows the Leela GPU usage and not the 3D that you probably see and confuse you.

Are you sure he shouldn't be choosing "Cuda" from the dropdown menu?

[Stephen Ham]

If you want to know the real GPU usage you should use GPU-Z and not Task Manager.
You can download GPU-Z from techpowerup.com the last version is 2.27. This utility gives you every important info from your GPUs.
For Backend the Multiplexing with 4 Threads gives the best result (in general) and for BackendOptions
backend=cudnn-fp16,(gpu=0),(gpu=1) is O.K.

Thank you, Corres.

I saw that those Backend Options were recommended for Fat Fritz, but have not yet seen them associated with Leela. When I switch to Multiplexing in Leela and 4 CPUs and then the above BackendOptions, the engine crashes. The error message is: "Lc0 v0.22.0 cuda caused an exception. This is a problem in the engine".

For the record, the same problem happens with Fat Fritz. But these crashes only happen with this BackendOption.

What do you suggest?

Thanks in advance.

-Steve-

[Stephen Ham]

With 2080 Ti you should ALWAYS use fp16.
Multiplex is irrelevant though if you have only 1 GPU. Multiplex is when you want to use more than 1 GPU (e.g 2,3,4 etc).

Second you say "task manager". But in there you need to check:
Performance -> GPU0(where 2080 Ti is) -> and now choose a ↓ down arrow in any of the setting, e.g Video Decode and choose Compute_0. That shows the Leela GPU usage and not the 3D that you probably see and confuse you.
So i'm sure all are fine.

Thank you, George. I'm truly impressed and grateful for the kind assistance that you and Corres have provided.

OK, switching Leela to Multiplexing, since I have two GPUs, and 4 CPUs, but leaving BackendOptions blank, since it crashes when the recommended data is entered (see my other post) I now have Task Manager open exactly as you advised.

Using these Leela specifications, Task Manager reports GPU utilization dropped from 16% to 7-8% (it oscillates between the two). Using Compute_0 for both GPUs, the output is 0%! In fact, all GPU data is 0: 3D, Copy, Dedicated GPU memory usage, and Shared GPU memory usage.

Unlike you gents, I'm a computer dummy. Nonetheless, that output seems totally wrong. However, Leela still seems to perform well. But I question if I have the optimal configuration. Please advise.

Thanks in advance!

Sincerely,
-Steve-

[brianr]

I have one 2080ti with Win10Pro 64bit.

In Task Manager one of the GPU options for me is Cuda (as was mentioned before), which does show reasonable activity.

However (as was also mentioned), I rely on MSI Afterburner and GPU-Z for the best picture of things.
The MSI tool can also display (graphically) other broad system resources such as CPU use and temp, RAM use, etc. Suggest looking for the Detach option and then right-click the graphs window to select always on top or not as you prefer.

[shrapnel]

Hello All,

I just purchased a new PC with dual NVIDIA GeForce RTX 2080 TI GPUs in order to maximize NN engine performance. However, when running Leela (Fritz 17 version) in Fritz GUI, Task Manager shows only 15%-16% GPU usage.

Switching to Fat Fritz, the GPU usage drops to 12%.

Is this normal or do I need better configurations? Yes, I have Leela's Backend set as: cudnn-fp16 with 2 CPU threads. Switching the Backend to multiplex and 4 CPU threads, the GPU usage drops to 8%, but the kN/s increases substantially.

My perception is that performance is superior with the former Backend rather than the latter, but I have not tested them in head-to-head competition.

ChessBase promised configuration information back on 11/13, but to date have provided only silence. Here's the quote:
macauley 11/13/2019 07:58
@EatMyShorts - We're going to have a follow-up article discussing this in more detail in the next day or so.

Surely they must have several frustrated clients...like me.

Thoughts, gentlemen?

All the best,
-Steve-

Welcome to the 2x2080Ti Club !
All you need to do is use GPU-Z software Utility. I use it always to tweak and maximize NN and GPU Utilization.
What are your GPUs anyway ?
I use twin ASUS OC RTX 11 GB 2080 Tis.
You have to use Multiplexing option on Backend.
In Backend Options, type backend=cudnn-fp16,(gpu=0),(gpu=1).
That's it !
Don't bother with Fat Fritz and its NN, even if you have them.
Fat Fritz is full of bugs and will only frustrate you.
The free lc0 is good enough.
Download the Sergio NNs, which are the largest and most powerful NNs on the 'net.
Don't listen to people who will tell you Sergio NN is weak.
Testers run it on their cheap, low-end GPUs and then complain it is weak.
It REQUIRES TWO powerful GPUs to fully utilize its capabilities.
Then, sit back and watch your PC whomp everyone on the Playchess Server.
Even if you don't win all the time, you will very rarely be beaten, given a decent Book.
Just yesterday, I drew three times with someone who was using Stockfish with 100 Threads !
Play me sometimes.
My userid is shrapnel on Playchess.
Regards