Can somebody explain what makes Leela as strong as Komodo?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
lkaufman
Posts: 3724
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Can somebody explain what makes Leela as strong as Komodo?

Post by lkaufman » Thu Jan 17, 2019 5:28 am

Chessqueen wrote:
Wed Jan 16, 2019 4:41 pm
Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA
My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.
Komodo rules!

corres
Posts: 1644
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Can somebody explain what makes Leela as strong as Komodo?

Post by corres » Thu Jan 17, 2019 8:45 am

lkaufman wrote:
Thu Jan 17, 2019 5:28 am
My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.
Team of AlphaZero used tensor units to work their engine and not used GPUs.
Common GPUs have no tensor cores because of this the common GPUs are not too effective for NN type chess engines.
NVIDIA developed a "real-time ray-tracing technology" for more realistic lightning and this technology based on NN. NN technology can use tensor cores also. Because tensor cores need lot of cores specialized for tensor operations, for getting a more powerfull GPU NVIDIA modified the cores of their GPUs to be able to working as tensor units.
This is one of the explanation for why much stronger Leela is when it is running on an NVIDIA Ti card.
I am very curious to a Komodo without NN what is based on GPU using.

chrisw
Posts: 2091
Joined: Tue Apr 03, 2012 2:28 pm

Re: Can somebody explain what makes Leela as strong as Komodo?

Post by chrisw » Thu Jan 17, 2019 8:51 am

lkaufman wrote:
Thu Jan 17, 2019 5:28 am
Chessqueen wrote:
Wed Jan 16, 2019 4:41 pm
Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA
My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.
There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)

Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: Can somebody explain what makes Leela as strong as Komodo?

Post by Javier Ros » Thu Jan 17, 2019 9:09 am

chrisw wrote:
Thu Jan 17, 2019 8:51 am
lkaufman wrote:
Thu Jan 17, 2019 5:28 am
Chessqueen wrote:
Wed Jan 16, 2019 4:41 pm
Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA
My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.
There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)

See

http://chessgpgpu.blogspot.com/

and

http://nikolachess.com/

It is a pity that the links of the videos don't work, only the donation button works! :D :D
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

grahamj
Posts: 37
Joined: Thu Oct 11, 2018 12:26 pm
Full name: Graham Jones

Re: Can somebody explain what makes Leela as strong as Komodo?

Post by grahamj » Thu Jan 17, 2019 9:18 am

Like lkaufman, I think it's mainly a successful exploitation of hardware.

I look at the AlphaZero paper in two quite different ways. As a scientific contribution, the important thing is the generality of the algorithm. The importance of the `zero' approach is not to avoid human bias, it is to demonstrate the method can work when little human knowledge is available. Many important real-world inference problems are poorly understood by humans. Whether the AlphaZero algorithm can be made to work for these problems remains to be seen, but that's the promise.

On the other hand, from the very narrow perspective of chess engines, I do not think AlphaZero represents the future. GPUs (and perhaps other massively parallel architectures) and machine learning (but not necessarily neural nets) will very much be part of the future. The hard part is effectively parallelising tree search. Tree search algorithms are up for grabs. My atttempt is roughly described here: http://indriid.com/2019/2019-01-06-tinsmith.pdf

On `other massively parallel architectures':
The 104 machine will be a single card frame incorporating 12 48-node cards. It will have 10,368 ARM processor cores, typically deployed as 9,216 application processors, 216 Monitor Processors and 216 spares. The 104 machine will be the largest portable SpiNNaker machine and is intended to operate in a desk-top environment from a standard mains supply. The card frame will incorporate a ~1kW power supply for the SpiNNaker cards... http://apt.cs.manchester.ac.uk/projects ... /hardware/
Graham Jones, www.indriid.com

corres
Posts: 1644
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Can somebody explain what makes Leela as strong as Komodo?

Post by corres » Thu Jan 17, 2019 9:37 am

chrisw wrote:
Thu Jan 17, 2019 8:51 am
lkaufman wrote:
Thu Jan 17, 2019 5:28 am
Chessqueen wrote:
Wed Jan 16, 2019 4:41 pm
Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA
My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.
There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)
The main issue is the cores of GPUs used in parallel to get higher effectiveness.
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.

chrisw
Posts: 2091
Joined: Tue Apr 03, 2012 2:28 pm

Re: Can somebody explain what makes Leela as strong as Komodo?

Post by chrisw » Thu Jan 17, 2019 10:10 am

corres wrote:
Thu Jan 17, 2019 9:37 am
chrisw wrote:
Thu Jan 17, 2019 8:51 am
lkaufman wrote:
Thu Jan 17, 2019 5:28 am
Chessqueen wrote:
Wed Jan 16, 2019 4:41 pm
Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA
My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.
There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)
The main issue is the cores of GPUs used in parallel to get higher effectiveness.
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.
AB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.

corres
Posts: 1644
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Can somebody explain what makes Leela as strong as Komodo?

Post by corres » Thu Jan 17, 2019 11:07 am

chrisw wrote:
Thu Jan 17, 2019 10:10 am
corres wrote:
Thu Jan 17, 2019 9:37 am
chrisw wrote:
Thu Jan 17, 2019 8:51 am
lkaufman wrote:
Thu Jan 17, 2019 5:28 am
Chessqueen wrote:
Wed Jan 16, 2019 4:41 pm
Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA
My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.
There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)
The main issue is the cores of GPUs used in parallel to get higher effectiveness.
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.
AB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.
Except what is impossible.
Or it is possible but it has bad efficiency.
To make a weaker Stockfish for GPUs only for virtue it is not a wise thing, I think.

smatovic
Posts: 905
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: Can somebody explain what makes Leela as strong as Komodo?

Post by smatovic » Thu Jan 17, 2019 11:17 am

chrisw wrote:
Thu Jan 17, 2019 10:10 am
corres wrote:
Thu Jan 17, 2019 9:37 am
chrisw wrote:
Thu Jan 17, 2019 8:51 am
lkaufman wrote:
Thu Jan 17, 2019 5:28 am
Chessqueen wrote:
Wed Jan 16, 2019 4:41 pm
Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA
My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.
There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)
The main issue is the cores of GPUs used in parallel to get higher effectiveness.
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.
AB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.
It is possible to port an simple AB engine to gpu...

https://zeta-chess.app26.de/post/zeta-v099/

and i am not the only one who worked on it...

https://zeta-chess.app26.de/post/papers ... ee-search/
https://zeta-chess.app26.de/post/other- ... -projects/

But NN just seem to be an "natural" fit for gpus.

--
Srdja

chrisw
Posts: 2091
Joined: Tue Apr 03, 2012 2:28 pm

Re: Can somebody explain what makes Leela as strong as Komodo?

Post by chrisw » Thu Jan 17, 2019 11:54 am

smatovic wrote:
Thu Jan 17, 2019 11:17 am
chrisw wrote:
Thu Jan 17, 2019 10:10 am
corres wrote:
Thu Jan 17, 2019 9:37 am
chrisw wrote:
Thu Jan 17, 2019 8:51 am
lkaufman wrote:
Thu Jan 17, 2019 5:28 am
Chessqueen wrote:
Wed Jan 16, 2019 4:41 pm
Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA
My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.
There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)
The main issue is the cores of GPUs used in parallel to get higher effectiveness.
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.
AB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.
It is possible to port an simple AB engine to gpu...

https://zeta-chess.app26.de/post/zeta-v099/

and i am not the only one who worked on it...

https://zeta-chess.app26.de/post/papers ... ee-search/
https://zeta-chess.app26.de/post/other- ... -projects/

But NN just seem to be an "natural" fit for gpus.

--
Srdja
Can GPUs do integer shift and logic operations? If so you can generate all pseudo legal at once in a few instructions (using logic and unrolling the usual loops), all attacks in a few instructions. Which implies evaluate N positions all at once, N subject to GPU RAM size. Knowing all children evals at once at each node would massively increase AB performance, or MCTS performance for that matter. Same operations across parallel threads and you’ld be evaluating and exploiting N nodes across the tree all at once. The limit on N is GPU RAM.
Bingo, stronger, much faster variant of AB. Just needs a new mind set. ducks behind sandbags as waves of negativity ensues .....

Post Reply