My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.Chessqueen wrote: ↑Wed Jan 16, 2019 5:41 pm Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
Can somebody explain what makes Leela as strong as Komodo?
Moderators: hgm, Rebel, chrisw
-
- Posts: 5961
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Can somebody explain what makes Leela as strong as Komodo?
Komodo rules!
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: Can somebody explain what makes Leela as strong as Komodo?
Team of AlphaZero used tensor units to work their engine and not used GPUs.lkaufman wrote: ↑Thu Jan 17, 2019 6:28 am My answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.
Common GPUs have no tensor cores because of this the common GPUs are not too effective for NN type chess engines.
NVIDIA developed a "real-time ray-tracing technology" for more realistic lightning and this technology based on NN. NN technology can use tensor cores also. Because tensor cores need lot of cores specialized for tensor operations, for getting a more powerfull GPU NVIDIA modified the cores of their GPUs to be able to working as tensor units.
This is one of the explanation for why much stronger Leela is when it is running on an NVIDIA Ti card.
I am very curious to a Komodo without NN what is based on GPU using.
-
- Posts: 4345
- Joined: Tue Apr 03, 2012 4:28 pm
Re: Can somebody explain what makes Leela as strong as Komodo?
There’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)lkaufman wrote: ↑Thu Jan 17, 2019 6:28 amMy answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.Chessqueen wrote: ↑Wed Jan 16, 2019 5:41 pm Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
-
- Posts: 200
- Joined: Fri Oct 12, 2012 12:48 pm
- Location: Seville (SPAIN)
- Full name: Javier Ros
Re: Can somebody explain what makes Leela as strong as Komodo?
chrisw wrote: ↑Thu Jan 17, 2019 9:51 amThere’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)lkaufman wrote: ↑Thu Jan 17, 2019 6:28 amMy answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.Chessqueen wrote: ↑Wed Jan 16, 2019 5:41 pm Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
See
http://chessgpgpu.blogspot.com/
and
http://nikolachess.com/
It is a pity that the links of the videos don't work, only the donation button works!
-
- Posts: 43
- Joined: Thu Oct 11, 2018 2:26 pm
- Full name: Graham Jones
Re: Can somebody explain what makes Leela as strong as Komodo?
Like lkaufman, I think it's mainly a successful exploitation of hardware.
I look at the AlphaZero paper in two quite different ways. As a scientific contribution, the important thing is the generality of the algorithm. The importance of the `zero' approach is not to avoid human bias, it is to demonstrate the method can work when little human knowledge is available. Many important real-world inference problems are poorly understood by humans. Whether the AlphaZero algorithm can be made to work for these problems remains to be seen, but that's the promise.
On the other hand, from the very narrow perspective of chess engines, I do not think AlphaZero represents the future. GPUs (and perhaps other massively parallel architectures) and machine learning (but not necessarily neural nets) will very much be part of the future. The hard part is effectively parallelising tree search. Tree search algorithms are up for grabs. My atttempt is roughly described here: http://indriid.com/2019/2019-01-06-tinsmith.pdf
On `other massively parallel architectures':
I look at the AlphaZero paper in two quite different ways. As a scientific contribution, the important thing is the generality of the algorithm. The importance of the `zero' approach is not to avoid human bias, it is to demonstrate the method can work when little human knowledge is available. Many important real-world inference problems are poorly understood by humans. Whether the AlphaZero algorithm can be made to work for these problems remains to be seen, but that's the promise.
On the other hand, from the very narrow perspective of chess engines, I do not think AlphaZero represents the future. GPUs (and perhaps other massively parallel architectures) and machine learning (but not necessarily neural nets) will very much be part of the future. The hard part is effectively parallelising tree search. Tree search algorithms are up for grabs. My atttempt is roughly described here: http://indriid.com/2019/2019-01-06-tinsmith.pdf
On `other massively parallel architectures':
The 104 machine will be a single card frame incorporating 12 48-node cards. It will have 10,368 ARM processor cores, typically deployed as 9,216 application processors, 216 Monitor Processors and 216 spares. The 104 machine will be the largest portable SpiNNaker machine and is intended to operate in a desk-top environment from a standard mains supply. The card frame will incorporate a ~1kW power supply for the SpiNNaker cards... http://apt.cs.manchester.ac.uk/projects ... /hardware/
Graham Jones, www.indriid.com
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: Can somebody explain what makes Leela as strong as Komodo?
The main issue is the cores of GPUs used in parallel to get higher effectiveness.chrisw wrote: ↑Thu Jan 17, 2019 9:51 amThere’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)lkaufman wrote: ↑Thu Jan 17, 2019 6:28 amMy answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.Chessqueen wrote: ↑Wed Jan 16, 2019 5:41 pm Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.
-
- Posts: 4345
- Joined: Tue Apr 03, 2012 4:28 pm
Re: Can somebody explain what makes Leela as strong as Komodo?
AB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.corres wrote: ↑Thu Jan 17, 2019 10:37 amThe main issue is the cores of GPUs used in parallel to get higher effectiveness.chrisw wrote: ↑Thu Jan 17, 2019 9:51 amThere’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)lkaufman wrote: ↑Thu Jan 17, 2019 6:28 amMy answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.Chessqueen wrote: ↑Wed Jan 16, 2019 5:41 pm Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: Can somebody explain what makes Leela as strong as Komodo?
Except what is impossible.chrisw wrote: ↑Thu Jan 17, 2019 11:10 amAB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.corres wrote: ↑Thu Jan 17, 2019 10:37 amThe main issue is the cores of GPUs used in parallel to get higher effectiveness.chrisw wrote: ↑Thu Jan 17, 2019 9:51 amThere’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)lkaufman wrote: ↑Thu Jan 17, 2019 6:28 amMy answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.Chessqueen wrote: ↑Wed Jan 16, 2019 5:41 pm Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.
Or it is possible but it has bad efficiency.
To make a weaker Stockfish for GPUs only for virtue it is not a wise thing, I think.
-
- Posts: 2696
- Joined: Wed Mar 10, 2010 10:18 pm
- Location: Hamburg, Germany
- Full name: Srdja Matovic
Re: Can somebody explain what makes Leela as strong as Komodo?
It is possible to port an simple AB engine to gpu...chrisw wrote: ↑Thu Jan 17, 2019 11:10 amAB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.corres wrote: ↑Thu Jan 17, 2019 10:37 amThe main issue is the cores of GPUs used in parallel to get higher effectiveness.chrisw wrote: ↑Thu Jan 17, 2019 9:51 amThere’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)lkaufman wrote: ↑Thu Jan 17, 2019 6:28 amMy answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.Chessqueen wrote: ↑Wed Jan 16, 2019 5:41 pm Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.
https://zeta-chess.app26.de/post/zeta-v099/
and i am not the only one who worked on it...
https://zeta-chess.app26.de/post/papers ... ee-search/
https://zeta-chess.app26.de/post/other- ... -projects/
But NN just seem to be an "natural" fit for gpus.
--
Srdja
-
- Posts: 4345
- Joined: Tue Apr 03, 2012 4:28 pm
Re: Can somebody explain what makes Leela as strong as Komodo?
Can GPUs do integer shift and logic operations? If so you can generate all pseudo legal at once in a few instructions (using logic and unrolling the usual loops), all attacks in a few instructions. Which implies evaluate N positions all at once, N subject to GPU RAM size. Knowing all children evals at once at each node would massively increase AB performance, or MCTS performance for that matter. Same operations across parallel threads and you’ld be evaluating and exploiting N nodes across the tree all at once. The limit on N is GPU RAM.smatovic wrote: ↑Thu Jan 17, 2019 12:17 pmIt is possible to port an simple AB engine to gpu...chrisw wrote: ↑Thu Jan 17, 2019 11:10 amAB programmer just has to change his mindset and he can write perfectly fine AB program on GPU.corres wrote: ↑Thu Jan 17, 2019 10:37 amThe main issue is the cores of GPUs used in parallel to get higher effectiveness.chrisw wrote: ↑Thu Jan 17, 2019 9:51 amThere’s a chess engine reported a few years ago that runs on gpu. Gpu has enough instructions to also operate as a “computer” and, as you presumably know, one can use logic operations as a alternative to branching (branching instructions exist I think, but are not so easy as with cpu.)lkaufman wrote: ↑Thu Jan 17, 2019 6:28 amMy answer is a bit different than others. Leela (following AlphaZero) has found a way to use a powerful GPU for chess, which is itself composed of hundreds or even thousands of cores. So far the only method that has made use of the GPU for chess is Neural Network. If we (Komodo or any other non-NN engine) could find a way to make use of these thousands of cores for chess other than NN we would probably have clearly the top engine, but it's not easy because these GPU cores are not able to do all the things a CPU can do. If Leela (or AlphaZero) runs on a CPU, it's rather pathetic.Chessqueen wrote: ↑Wed Jan 16, 2019 5:41 pm Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
But a classical AB chess engine works such a way that it can not exploit well the parallel computation.
This is true for a PC with many cores and a GPU what has much more cores.
Moreover cores of GPUs are specialized for image representation and not for common logical and numerical computations.
I think one of the most effective usage of many cores for developing chess engines is the using of NN technique.
GPU as it’s name suggests was developed to enable throwing around lots of screen memory, but it’s actually a general purpose computer with memory and instruction set. If programmer adopts appropriate mind set everything and anything is possible.
https://zeta-chess.app26.de/post/zeta-v099/
and i am not the only one who worked on it...
https://zeta-chess.app26.de/post/papers ... ee-search/
https://zeta-chess.app26.de/post/other- ... -projects/
But NN just seem to be an "natural" fit for gpus.
--
Srdja
Bingo, stronger, much faster variant of AB. Just needs a new mind set. ducks behind sandbags as waves of negativity ensues .....