Page 1 of 2

Lc0 GPU load

Posted: Tue Mar 05, 2019 1:17 am
by lovetb
Hi, I have been using Lc0 for a week now. The GPU load keeps going up & down.
You can see in the graph below,

Image

It doesn't stay at 100% like the CPUs when playing Stockfish.

Is this normal ?

Re: Lc0 GPU load

Posted: Tue Mar 05, 2019 2:03 am
by mwyoung
lovetb wrote: Tue Mar 05, 2019 1:17 am Hi, I have been using Lc0 for a week now. The GPU load keeps going up & down.
You can see in the graph below,

Image

It doesn't stay at 100% like the CPUs when playing Stockfish.

Is this normal ?
Yes, looks exactly like my graph. Put a 5xxxx NN on and see what happends.

Re: Lc0 GPU load

Posted: Tue Mar 05, 2019 8:42 am
by M ANSARI
Maybe thermal issues and GPU is throttling?

Re: Lc0 GPU load

Posted: Tue Mar 05, 2019 8:52 am
by lovetb
M ANSARI wrote: Tue Mar 05, 2019 8:42 am Maybe thermal issues and GPU is throttling?
You are right. I left the case open for better air circulation & I unsinstalled MSI Afterburner.
I had better results.

Image

Re: Lc0 GPU load

Posted: Tue Mar 05, 2019 4:00 pm
by mwyoung
lovetb wrote: Tue Mar 05, 2019 8:52 am
M ANSARI wrote: Tue Mar 05, 2019 8:42 am Maybe thermal issues and GPU is throttling?
You are right. I left the case open for better air circulation & I unsinstalled MSI Afterburner.
I had better results.

Image
If you reinstall msi afterburner. And go to settings and setup a custom fan curve. You will not have to leave your case open.

Unless your case has inadequate air flow.

The Factory fan curve is not good when running Lc0.

I have my fan curve hitting 100% fan speed when temps hit 60c. And this fan curve has worked very well.

Re: Lc0 GPU load

Posted: Tue Mar 05, 2019 7:51 pm
by M ANSARI
I think throttling of the GPU was the cause of a few lost points for Lc0 in TCEC 14. There were a few moves that simply made no sense and I was using the same network with similar hardware. It might actually make sense to have some sort of water cooling for the GPU's.

Re: Lc0 GPU load

Posted: Tue Mar 05, 2019 8:21 pm
by smatovic
M ANSARI wrote: Tue Mar 05, 2019 7:51 pm I think throttling of the GPU was the cause of a few lost points for Lc0 in TCEC 14. There were a few moves that simply made no sense and I was using the same network with similar hardware. It might actually make sense to have some sort of water cooling for the GPU's.
Yea :)

I built a custom water cooling for my 275 Watt Fury X, and for hobbyists i can recommend a Zalman Reserator all in one passive cooler plus 120mm radiator and a 120mm water-temperature controlled fan. I really like the Zalman, i estimate it alone cools about 150 Watt...

https://www.newegg.com/Product/Product. ... 6835118111

As a rule of thumb - a 120mm radiator cools about 100 Watt, depended on the rpm of the fans.

--
Srdja

Re: Lc0 GPU load

Posted: Tue Mar 05, 2019 8:25 pm
by mwyoung
M ANSARI wrote: Tue Mar 05, 2019 7:51 pm I think throttling of the GPU was the cause of a few lost points for Lc0 in TCEC 14. There were a few moves that simply made no sense and I was using the same network with similar hardware. It might actually make sense to have some sort of water cooling for the GPU's.
Lc0 is very hard on a GPU. If you do not have a beefy air cooler. It will beat up a GPU. I bought a card with a beefy air cooler with triple fans. And I still had use a custom fan curve to keep temp under control. At full load now the card runs in the low 60's c.

After testing I found that it was not my temps causing weak play and moves. It was using minibatchsize = 1024. I lost some NPS by going back to 256, but test play is now much better. And the latest 4xxxx nets are on the verge of overtaking the latest DEV. of Stockfish playing on a 2950x at 5+5 TC. Around 10 elo difference.

Re: Lc0 GPU load

Posted: Wed Mar 06, 2019 11:46 am
by lovetb
Thanks for all your replies.

If I try to host the GPU out of the box using a raiser card adopter something like shown below, will it affect the performance?
The idea is to give the gpu more cooler air to breath & to cool down.

Image

Re: Lc0 GPU load

Posted: Wed Mar 06, 2019 12:34 pm
by smatovic
lovetb wrote: Wed Mar 06, 2019 11:46 am Thanks for all your replies.

If I try to host the GPU out of the box using a raiser card adopter something like shown below, will it affect the performance?
The idea is to give the gpu more cooler air to breath & to cool down.
Maybe this 6 way LC0 rig via PCIe risers is of interest for you

https://groups.google.com/forum/#!msg/l ... ihGJYCAwAJ

--
Srdja