TalkChess.com

Milos wrote: ↑Mon Oct 15, 2018 10:32 pm

Werewolf wrote: ↑Mon Oct 15, 2018 12:22 pm

Milos wrote: ↑Sun Oct 14, 2018 10:55 pm

Werewolf wrote: ↑Sun Oct 14, 2018 10:35 pm GPUs have been improving much faster than CPUs. If this continues, it'll get better and better for the GPU engines...

Ofc they were improving much faster when 9xx series was on 28nm, while Intel is already 4 generations on 14nm (10nm Cannon Lake is a total failure since Intel can't make a reliable and commercially viable 10nm CPU process, and TSMC can't either).
Once now 20xx series reached 12nm you are gonna witness the same thing that happened to Intel since 4 generations ago. Stagnation...

The performance improvement from Pascal (16nm) to Turing (12nm) was about 15% IIRC. Probably the next shrink will yield less.

But with cards they seem to be freer than CPUs to try stuff if it's A.I related. So for Leela the change from Pascal to Turning is HUGE. If, for example, Nvidia produced a card which did away with CUDA cores and focused exclusively on things related to A.I - we could see at least one more decent jump up, IMO.

There is even less than those 15% between Pascal and Turing architecture thanks to scaling. Tensor cores are not very useful and gain from them is minimal, at least for inference. The actual speed up above those 15% comes only from enabling FP16 in Turing that was intentionally crippled in Pascal and earlier architectures.
But this is one-trick pony. There is almost nothing more that Nvidia can bring architecturally in hardware. And as things look like now, we will wait at least 3 more years for working 10nm node.
I know ppl are dreamers but in this case this is totally unsound.

megamau wrote: ↑Fri Sep 21, 2018 2:25 pm

Milos wrote: ↑Sun Sep 16, 2018 7:22 am

ankan wrote: ↑Sun Sep 16, 2018 6:09 am

Milos wrote: ↑Sun Sep 16, 2018 1:24 am Since FP16 is not enabled in 20xx cards the same it wasn't enabled in 10xx cards the only gain is those 15% in extra CUDA cores and higher frequency. Therefore 2080Ti will be faster than 1080Ti for exactly those 15%. Anyone who believes in some other magical speed-up is frankly speaking just daydreaming.

This is definitely not true. The fp16 path of lc0 uses tensor cores on Volta and they do help 3x3 convolutions. The reason you see only about 3x speedup at best (compared to 8x if you compare the peak fp16 tensor math vs regular fp32 throughput) is because fp32 path uses winograd algorithm which is 2-3x faster than regular implicit gmem algorithm used by fp16 path. As you said tensor cores just gives you 4x4 matrix multiplications and making them work with winograd algorithm is hard.

2080Ti should be almost as fast as a TitanV for lc0 (or ~3X faster than 1080Ti when using fp16 mode).

Well you might be one of thousands of other Indian guys writing drivers for Nvidia, but what you are writing is definitively false.
Titan V has FP16 working in CUDA, 2080Ti doesn't. ....
Since 2080Ti doesn't have FP16 working in CUDA cores, 2080Ti additional speed up can be only 5% thanks to Tensor cores (plus around 15% thanks to more CUDA cores). Your stories about 3x speed up for 2080Ti compared to 1080Ti are nothing but marketing of your company. You are simply biased since you have vested interest.

So Milos, as the cards and the benchmarks are now available, is it time to admit you were wrong ?

Astatos wrote: ↑Tue Oct 16, 2018 10:33 pm What's the problem anyway? You can have something like Tesla.

megamau wrote: ↑Sat Oct 20, 2018 4:31 pm While clearly LC0 is not the strongest engine (that title is currently with Stockfish), I don't understand why we shouldn't "trust" and engine which in 7 months became a member of the "big 4".

marsell wrote: ↑Sun Oct 21, 2018 6:35 pm Leela Fanbase claims LC0 as number one every day. But evidence is missing

Werewolf wrote: ↑Mon Oct 15, 2018 12:22 pm

Milos wrote: ↑Sun Oct 14, 2018 10:55 pm

Werewolf wrote: ↑Sun Oct 14, 2018 10:35 pm GPUs have been improving much faster than CPUs. If this continues, it'll get better and better for the GPU engines...

Ofc they were improving much faster when 9xx series was on 28nm, while Intel is already 4 generations on 14nm (10nm Cannon Lake is a total failure since Intel can't make a reliable and commercially viable 10nm CPU process, and TSMC can't either).
Once now 20xx series reached 12nm you are gonna witness the same thing that happened to Intel since 4 generations ago. Stagnation...

The performance improvement from Pascal (16nm) to Turing (12nm) was about 15% IIRC. Probably the next shrink will yield less.

But with cards they seem to be freer than CPUs to try stuff if it's A.I related. So for Leela the change from Pascal to Turning is HUGE. If, for example, Nvidia produced a card which did away with CUDA cores and focused exclusively on things related to A.I - we could see at least one more decent jump up, IMO.