I stumbled upon this article on the new Nvidia RTX GPUs

[smatovic]

Someone who understands better can see and explain something here?

https://www.reddit.com/r/MachineLearnin ... 0_for_dnn/

Page 22 seems interesting:

https://www.nvidia.com/content/dam/en-z ... per-V1.pdf

But I am not able to forecast you how this 2xFP32 and 2xTensorOps of RTX 30xx will translate to current Lc0.

--
Srdja

What I gathered says "it's complicated", the cycles come from many parts of the computation, so is the cycle number reduction by different components like tensor cores. CUDA etc. The range 1.40 to 2.20 is still possible for the speed-up from RTX 2080 to 3080 with Lc0 as it is built now. Some links:

https://timdettmers.com/2020/09/07/whic ... nsor_Cores
https://www.pugetsystems.com/labs/hpc/R ... nary-1885/

Hmm, according to all those articles and papers, a slow down of RTX 3080 compared to RTX 2080 TI in some AI tasks is also possible.

You should compare 3080 with 2080 TI, they have similar SM core count.

--
Srdja

I seem to manage to extrapolate from this chart:
https://www.pugetsystems.com/labs/hpc/R ... nary-1885/



Here we cannot compare directly old RTX 2080 Ti numbers with the RTX 3080 test. RTX 3080 test (1023 images/sec) uses significantly newer drivers, TensorFlow version, testing setup. But we have RTX Titan tested now which got 1082 images/sec on FP16 with the old result which was 653, compared with RTX 2080 Ti of 532 (old result). By extrapolation, with the "old setup" 3080 would get 617 images/sec, and now we can compare to 2080 Ti (532) and 2080 (382). The speed-up RTX 2080 --> 3080 is about 60%, which is good but not great.

In this scope RTX 3080 would be only 1.16x compared to RTX 2080 TI, what can be
explained by frequency increase alone, without any architecture changes or
improvements in SM design.

--
Srdja

[Laskos]

Someone who understands better can see and explain something here?

https://www.reddit.com/r/MachineLearnin ... 0_for_dnn/

Page 22 seems interesting:

https://www.nvidia.com/content/dam/en-z ... per-V1.pdf

But I am not able to forecast you how this 2xFP32 and 2xTensorOps of RTX 30xx will translate to current Lc0.

--
Srdja

What I gathered says "it's complicated", the cycles come from many parts of the computation, so is the cycle number reduction by different components like tensor cores. CUDA etc. The range 1.40 to 2.20 is still possible for the speed-up from RTX 2080 to 3080 with Lc0 as it is built now. Some links:

https://timdettmers.com/2020/09/07/whic ... nsor_Cores
https://www.pugetsystems.com/labs/hpc/R ... nary-1885/

Hmm, according to all those articles and papers, a slow down of RTX 3080 compared to RTX 2080 TI in some AI tasks is also possible.

You should compare 3080 with 2080 TI, they have similar SM core count.

--
Srdja

I seem to manage to extrapolate from this chart:
https://www.pugetsystems.com/labs/hpc/R ... nary-1885/



Here we cannot compare directly old RTX 2080 Ti numbers with the RTX 3080 test. RTX 3080 test (1023 images/sec) uses significantly newer drivers, TensorFlow version, testing setup. But we have RTX Titan tested now which got 1082 images/sec on FP16 with the old result which was 653, compared with RTX 2080 Ti of 532 (old result). By extrapolation, with the "old setup" 3080 would get 617 images/sec, and now we can compare to 2080 Ti (532) and 2080 (382). The speed-up RTX 2080 --> 3080 is about 60%, which is good but not great.

In this scope RTX 3080 would be only 1.16x compared to RTX 2080 TI, what can be
explained by frequency increase alone, without any architecture changes or
improvements in SM design.

--
Srdja

The second hand Turings would seem the most efficient per dollar GPUs for deep learning, especially the cheapo second hand RTX 2060 Super and 2070 Super. I might end up buying one of those atop of my old RTX 2070 for the same speed as a single RTX 3080. That is, if I am not using GPU for other reasons too.

[mwyoung]

A lot of reviews of 3080 are out but they focus almost exclusively on games. The closest I've seen is this https://babeltechreviews.com/rtx-3080-a ... hmarked/4/ with a 27% increase of half-float performance over 2080Ti.

CUDA cores seem to perform well. Maybe for Leela case a 60-70% or so over RTX 2080 (non-super), but let's see the benches.

However maybe a bit less than 50% improvement with 3070 over my 2070 next month. This time 3080 seems the best deal. Last time it was 2070 or 2060. But aside money, 3080 seems way too large, and it consumes like an air conditioner, in real terms in excess of 350W.

After all, if only for Chess and Go, maybe the best is to spend just $250 and get a second hand 2060 Super on top of my 2070, close to 100% boost. Will anyway have to upgrade the case and the overall cooling.

Or buy yourself 20 cores and abandon giant networks that insist on GPU

That is your most insane advice I have seen yet.

Sure buy at 20 core system, but don't spend an extra 500 bucks for a RTX GPU.
That you would need, and want anyway with a 20 core system.

Why, because the nets are 131 MB. And take us 15 seconds to download.

[chrisw]

A lot of reviews of 3080 are out but they focus almost exclusively on games. The closest I've seen is this https://babeltechreviews.com/rtx-3080-a ... hmarked/4/ with a 27% increase of half-float performance over 2080Ti.

CUDA cores seem to perform well. Maybe for Leela case a 60-70% or so over RTX 2080 (non-super), but let's see the benches.

However maybe a bit less than 50% improvement with 3070 over my 2070 next month. This time 3080 seems the best deal. Last time it was 2070 or 2060. But aside money, 3080 seems way too large, and it consumes like an air conditioner, in real terms in excess of 350W.

After all, if only for Chess and Go, maybe the best is to spend just $250 and get a second hand 2060 Super on top of my 2070, close to 100% boost. Will anyway have to upgrade the case and the overall cooling.

Or buy yourself 20 cores and abandon giant networks that insist on GPU

That is your most insane advice I have seen yet.

Sure buy at 20 core system, but don't spend an extra 500 bucks for a RTX GPU.
That you would need, and want anyway with a 20 core system.

Que? I already have two boxes with 2080ti in each. I know you’re an end user, so your requirements are different. Dev kit for a CPU based engine doesn’t need anything more than the simplest GPU. If there is any GPU dev requirement, and sometimes there is, then there’s capacity on other equipment.

Why, because the nets are 131 MB. And take us 15 seconds to download.

One word of advice. I am not you.

[Laskos]

Finally, some reliable data with Lc0, although using OpenCL backend, but the relative speeds of different RTX GPUs should be reflected:
https://www.phoronix.com/scan.php?page= ... pute&num=6

3080 ---> 33% above 2080 Ti and a whopping 88% over the same price at the release 2080. Good numbers, now I will wait and see the 3070 speeds.

[Laskos]

Hmmm, the things changed a bit with the v26.3 engine and CUDA backend. The previous graph in the thread was for v26 engine and OpenCL backend. The new CUDA backend (it's the fastest one with RTX GPUs) shows the largest improvement with RTX 2xxx GPUs and smaller with RTX 3xxx. So that I saw a benchmark showing now RTX 3090 just 40% above RTX 2080Ti, meaning 3080 would stand at mere 10% or so above 2080Ti with v26.3 CUDA backend. Not that good, and I don't know whether there are foreseen backend improvements targeted towards 3xxx series.

[smatovic]

Hmmm, the things changed a bit with the v26.3 engine and CUDA backend. The previous graph in the thread was for v26 engine and OpenCL backend. The new CUDA backend (it's the fastest one with RTX GPUs) shows the largest improvement with RTX 2xxx GPUs and smaller with RTX 3xxx. So that I saw a benchmark showing now RTX 3090 just 40% above RTX 2080Ti, meaning 3080 would stand at mere 10% or so above 2080Ti with v26.3 CUDA backend. Not that good, and I don't know whether there are foreseen backend improvements targeted towards 3xxx series.

10% would confirm the speedup via clock increase only with no benefits from architecture changes.

--
Srdja

[Laskos]

Hmmm, the things changed a bit with the v26.3 engine and CUDA backend. The previous graph in the thread was for v26 engine and OpenCL backend. The new CUDA backend (it's the fastest one with RTX GPUs) shows the largest improvement with RTX 2xxx GPUs and smaller with RTX 3xxx. So that I saw a benchmark showing now RTX 3090 just 40% above RTX 2080Ti, meaning 3080 would stand at mere 10% or so above 2080Ti with v26.3 CUDA backend. Not that good, and I don't know whether there are foreseen backend improvements targeted towards 3xxx series.

10% would confirm the speedup via clock increase only with no benefits from architecture changes.

--
Srdja

Yeah, really disappointing. And not a better deal with the coming more affordable RTX 3070 which by extrapolation would be only 30% faster with Lc0 than my current 2070. Maybe a backend improvement targeting 3xxx series will be available sometime soon, I don't know, someone should ask Ankan.

[George Tsavdaris]

Hmmm, the things changed a bit with the v26.3 engine and CUDA backend. The previous graph in the thread was for v26 engine and OpenCL backend. The new CUDA backend (it's the fastest one with RTX GPUs) shows the largest improvement with RTX 2xxx GPUs and smaller with RTX 3xxx. So that I saw a benchmark showing now RTX 3090 just 40% above RTX 2080Ti, meaning 3080 would stand at mere 10% or so above 2080Ti with v26.3 CUDA backend. Not that good, and I don't know whether there are foreseen backend improvements targeted towards 3xxx series.

Where's that CUDA benchmark page link?

[Laskos]

Hmmm, the things changed a bit with the v26.3 engine and CUDA backend. The previous graph in the thread was for v26 engine and OpenCL backend. The new CUDA backend (it's the fastest one with RTX GPUs) shows the largest improvement with RTX 2xxx GPUs and smaller with RTX 3xxx. So that I saw a benchmark showing now RTX 3090 just 40% above RTX 2080Ti, meaning 3080 would stand at mere 10% or so above 2080Ti with v26.3 CUDA backend. Not that good, and I don't know whether there are foreseen backend improvements targeted towards 3xxx series.

Where's that CUDA benchmark page link?

I saw on Lc0 discord someone benching with v26.3 CUDA the RTX 3090 as being 40% faster than RTX 2080Ti (30.5 vs 22.0 knps with a large JH net).