Official: Lc0 is the strongest engine :)

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Posts: 3971
Joined: Wed Nov 25, 2009 12:47 am

Re: Official: Lc0 is the strongest engine :)

Post by Milos » Sun Oct 21, 2018 11:22 pm

cucumber wrote:
Sun Oct 21, 2018 7:57 pm
Tensor cores as is are just matmul ASICs. Matrix multiplication makes up a large part of convolutions, but it is totally possible to get even more application specific, should you want another large jump up. Currently, matmul ASICS are limited by data movement, which puts an upper bound on latency, which is a big Leela killer. That's probably what will be benefiting Leela the most. Whether or not that will be enough to fix Leela in calculation heavy endgames is open for debate.
This is not really correct.
Tensor cores are 4x4 direct matrix multiplication.
In most of the cases inference of CNNs convolutions are 3x3 realized using Vinograd and this implementation is faster when using regular (CUDA) cores then using 4x4 direct matrix multiplication in Tensor cores. In addition you waste a full 4x4 Tensor core for single 3x3 convolution meaning you have roughly 30% efficiency. So, much lower efficiency, plus slower operation yields very little additional benefit of Tensor cores compared to only CUDA cores.

Post Reply