Upcoming graphics cards for Leela

[whereagles]

I just bought a 1080 Ti because you can get them under $700 now.

will it overheat?

PLS I HAVE TO KNOW

[Werewolf]

Here's some better info, based on what's now released and in terms of TFLOPS, rather than FPS or something silly like that.

Titan RTX (??)
Quadro RTX 6000 - 15.9
Titan V - 14.9 (29.8 half P)
2080 Ti - 13.4
Quadro P6000 - 11.7
1080 Ti - 11.3
Quadro RTX 5000 - 10.6
2080 - 10.1
Quadro P5000 - 9.5
1080 - 8.9
2070 - 7.5
1060 - 4.4

I think this is incorrect. While we don't have actual test benchmarks yet, the RTX lineup shares wiith the Titan V one important advantage over the GTX 10x0 cards: It can utilize fp16 inference fully, which gives a factor of 2 to speed in addition to the TFLOPS you posted. Currently, a Titan V can do around triple the Leela NPS that a 1080Ti can do, and the 2080Ti should be very close to that.

It would be good if you are right, but it would also be a bit odd that Nvidia would eat into the profits of their more expensive cards.

[Werewolf]

Here's some better info, based on what's now released and in terms of TFLOPS, rather than FPS or something silly like that.

Titan RTX (??)
Quadro RTX 6000 - 15.9
Titan V - 14.9 (29.8 half P)
2080 Ti - 13.4
Quadro P6000 - 11.7
1080 Ti - 11.3
Quadro RTX 5000 - 10.6
2080 - 10.1
Quadro P5000 - 9.5
1080 - 8.9
2070 - 7.5
1060 - 4.4

I think this is incorrect. While we don't have actual test benchmarks yet, the RTX lineup shares wiith the Titan V one important advantage over the GTX 10x0 cards: It can utilize fp16 inference fully, which gives a factor of 2 to speed in addition to the TFLOPS you posted. Currently, a Titan V can do around triple the Leela NPS that a 1080Ti can do, and the 2080Ti should be very close to that.

Why this BS-ing when it is also known that neither 2070, nor 2080, nor 2080Ti will have real FP16, i.e. all 3 will have it as 10xx cards at 1:64 rate?
Only Quadro's and Titan RTX will have actual FP16.

For some reason Wikipedia doesn't list the FP16 performance of the Quadro cards. They definitely do half precision?

https://en.wikipedia.org/wiki/List_of_N ... xxx_series

[Milos]

Here's some better info, based on what's now released and in terms of TFLOPS, rather than FPS or something silly like that.

Titan RTX (??)
Quadro RTX 6000 - 15.9
Titan V - 14.9 (29.8 half P)
2080 Ti - 13.4
Quadro P6000 - 11.7
1080 Ti - 11.3
Quadro RTX 5000 - 10.6
2080 - 10.1
Quadro P5000 - 9.5
1080 - 8.9
2070 - 7.5
1060 - 4.4

I think this is incorrect. While we don't have actual test benchmarks yet, the RTX lineup shares wiith the Titan V one important advantage over the GTX 10x0 cards: It can utilize fp16 inference fully, which gives a factor of 2 to speed in addition to the TFLOPS you posted. Currently, a Titan V can do around triple the Leela NPS that a 1080Ti can do, and the 2080Ti should be very close to that.

Why this BS-ing when it is also known that neither 2070, nor 2080, nor 2080Ti will have real FP16, i.e. all 3 will have it as 10xx cards at 1:64 rate?
Only Quadro's and Titan RTX will have actual FP16.

For some reason Wikipedia doesn't list the FP16 performance of the Quadro cards. They definitely do half precision?

https://en.wikipedia.org/wiki/List_of_N ... xxx_series

Nothing is sure before someone actually tests card for specific performance, but there are quite a few indicators.
If you look for example fp64 performance it is real one (1:2) while in the past it was 1:32.

[mirek]

For some reason Wikipedia doesn't list the FP16 performance of the Quadro cards. They definitely do half precision?

https://en.wikipedia.org/wiki/List_of_N ... xxx_series

Nothing is sure before someone actually tests card for specific performance, but there are quite a few indicators.
If you look for example fp64 performance it is real one (1:2) while in the past it was 1:32.

Tensor core performance on RTX 8000 was explicitly mentioned during the new Quadro cards reveal.
Going by memory it's 125 TFLOPs for fp16, 250 TOPs for int8 and 500 TOPs for int4
and these numbers seem to be confirmed also by https://wccftech.com/nvidia-quadro-rtx- ... cing-demo/

[Werewolf]

Tensor core performance on RTX 8000 was explicitly mentioned during the new Quadro cards reveal.
Going by memory it's 125 TFLOPs for fp16,

Lc0 can't use Tensor yet though ?

[mirek]

Tensor core performance on RTX 8000 was explicitly mentioned during the new Quadro cards reveal.
Going by memory it's 125 TFLOPs for fp16,

Lc0 can't use Tensor yet though ?

There are some benchmarks showing that lc0 is ~ 3x faster on Titan V than on 1080 Ti
Huge part of it I am sure comes from efficient fp16 operations that are enabled on tensor cores.

Also worth noting is that while Titan V has very similar specs to RTX 8000 (110 TFLOPs fp16)
and thus should be theoretically about 10x faster then 1080 TI (11 TFLOPs fp32) this is clearly not the case.
I am not sure if current implementation could be further optimized to take better advantage of the tensor cores or if there
is some fundamental limit i.e. like saturated memory bandwidth for current NN architectire.

[Dann Corbit]

I just bought a 1080 Ti because you can get them under $700 now.

will it overheat?

PLS I HAVE TO KNOW

I doubt it.

I have a 1200 watt power supply, and I have it external to the machine. So all of that heat that is normally given off by the power supply inside the box is dumped into the room instead. The machine and the cards have plenty of cooling. The 1080 ti does not have a history of overheating. So I am not worried about it.

But until you actually do it for a while, there is no way to know for sure.

[Werewolf]

Tensor core performance on RTX 8000 was explicitly mentioned during the new Quadro cards reveal.
Going by memory it's 125 TFLOPs for fp16,

Lc0 can't use Tensor yet though ?

There are some benchmarks showing that lc0 is ~ 3x faster on Titan V than on 1080 Ti
Huge part of it I am sure comes from efficient fp16 operations that are enabled on tensor cores.

Also worth noting is that while Titan V has very similar specs to RTX 8000 (110 TFLOPs fp16)
and thus should be theoretically about 10x faster then 1080 TI (11 TFLOPs fp32) this is clearly not the case.
I am not sure if current implementation could be further optimized to take better advantage of the tensor cores or if there
is some fundamental limit i.e. like saturated memory bandwidth for current NN architectire.

Saw this on the blog today:

The learning rate for the test10 training run has been lowered to 0.0002. Network id 11013 will be the first network trained with the new LR.

This is the last time we lower it for test10 to squeeze some more Elo out of it. It's expected that the result will be visible within a day or two.

The test10 will probably stay for some weeks, and after that the plan is to do a reset and to start a main2 run from scratch again.

What will change after restart:
int8 quantization during training
That's how DeepMind did it. This will produce networks compatible with TensorRT framework which should considerably improve nps on supported hardware.

[Lion]

Do we know more about the RTX 2080 Ti?

Also are there benchmark available for Titan V and 1080 Ti.

I am considering upgrading my GPU but not sure the the 3k USD is worth for the Titan V....

rgds