Can somebody explain what makes Leela as strong as Komodo?

[lkaufman]

Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA

My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.

[corres]

My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.

Team of AlphaZero used tensor units to work their engine and not used GPUs.
Common GPUs have no tensor cores because of this the common GPUs are not too effective for NN type chess engines.
NVIDIA developed a "real-time ray-tracing technology" for more realistic lightning and this technology based on NN. NN technology can use tensor cores also. Because tensor cores need lot of cores specialized for tensor operations, for getting a more powerfull GPU NVIDIA modified the cores of their GPUs to be able to working as tensor units.
This is one of the explanation for why much stronger Leela is when it is running on an NVIDIA Ti card.
I am very curious to a Komodo without NN what is based on GPU using.

[chrisw]

Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA

My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.

There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)

[Javier Ros]

Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA

My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.

There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)

See

http://chessgpgpu.blogspot.com/

and

http://nikolachess.com/

It is a pity that the links of the videos don't work, only the donation button works!

[grahamj]

Like lkaufman, I think it's mainly a successful exploitation of hardware.

I look at the AlphaZero paper in two quite different ways. As a scientific contribution, the important thing is the generality of the algorithm. The importance of the `zero' approach is not to avoid human bias, it is to demonstrate the method can work when little human knowledge is available. Many important real-world inference problems are poorly understood by humans. Whether the AlphaZero algorithm can be made to work for these problems remains to be seen, but that's the promise.

On the other hand, from the very narrow perspective of chess engines, I do not think AlphaZero represents the future. GPUs (and perhaps other massively parallel architectures) and machine learning (but not necessarily neural nets) will very much be part of the future. The hard part is effectively parallelising tree search. Tree search algorithms are up for grabs. My atttempt is roughly described here: http://indriid.com/2019/2019-01-06-tinsmith.pdf

On `other massively parallel architectures':

The 104 machine will be a single card frame incorporating 12 48-node cards. It will have 10,368 ARM processor cores, typically deployed as 9,216 application processors, 216 Monitor Processors and 216 spares. The 104 machine will be the largest portable SpiNNaker machine and is intended to operate in a desk-top environment from a standard mains supply. The card frame will incorporate a ~1kW power supply for the SpiNNaker cards... http://apt.cs.manchester.ac.uk/projects ... /hardware/

[corres]

Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA

My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.

There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)

The main issue is the cores of GPUs used in parallel to get higher effectiveness.
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.

[chrisw]

Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA

My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.

There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)

The main issue is the cores of GPUs used in parallel to get higher effectiveness.
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.

AB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.

[corres]

Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA

My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.

There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)

The main issue is the cores of GPUs used in parallel to get higher effectiveness.
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.

AB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.

Except what is impossible.
Or it is possible but it has bad efficiency.
To make a weaker Stockfish for GPUs only for virtue it is not a wise thing, I think.

[smatovic]

Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA

My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.

There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)

The main issue is the cores of GPUs used in parallel to get higher effectiveness.
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.

AB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.

It is possible to port an simple AB engine to gpu...

https://zeta-chess.app26.de/post/zeta-v099/

and i am not the only one who worked on it...

https://zeta-chess.app26.de/post/papers ... ee-search/
https://zeta-chess.app26.de/post/other- ... -projects/

But NN just seem to be an "natural" fit for gpus.

--
Srdja

[chrisw]

Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA

My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.

There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)

The main issue is the cores of GPUs used in parallel to get higher effectiveness.
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.

AB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.

It is possible to port an simple AB engine to gpu...

https://zeta-chess.app26.de/post/zeta-v099/

and i am not the only one who worked on it...

https://zeta-chess.app26.de/post/papers ... ee-search/
https://zeta-chess.app26.de/post/other- ... -projects/

But NN just seem to be an "natural" fit for gpus.

--
Srdja

Can GPUs do integer shift and logic operations? If so you can generate all pseudo legal at once in a few instructions (using logic and unrolling the usual loops), all attacks in a few instructions. Which implies evaluate N positions all at once, N subject to GPU RAM size. Knowing all children evals at once at each node would massively increase AB performance, or MCTS performance for that matter. Same operations across parallel threads and you’ld be evaluating and exploiting N nodes across the tree all at once. The limit on N is GPU RAM.
Bingo, stronger, much faster variant of AB. Just needs a new mind set. ducks behind sandbags as waves of negativity ensues .....