chrisw wrote: ↑Wed Aug 15, 2018 12:58 pm
Poor score indicates something is broken. Which is always likely with entered late changes.
High draw rate indicates higher than "normal" width to depth ratio. In general depth finds interesting lines and possible wins. Width defends against overlooking stuff.
More width and less depth = safe, defensive play, but dull. High draw rate. Simples.
Could be broken somewhere, could be late changes conspired to alter the search profile, could be that the scaling to more nodes tends to reflect in width rather than depth. Could be random, but the operating assumption has to be that there is a problem.
I do not think LC0 developers made serious changes what drastically lessens the chess power of LC0.
The main evidence for it the difference between LC0 and DeusX does not change a lot in the games of TCEC 13 Division 3.
As I mentioned yet the weakness is in connection with NN namely the size of NN and the filling of NN.
The knowledge what is absent in NN the MCTS can not substitute fully.
The current state of Leela and DeusX is only good for STC games - that is all.
There are some strange things going on with leela at TCEC.
The event has exposed 'bugs' and optimisations to be addressed, but that may not be the whole explanation.
Wonder if there are also hardware problems.
For example, compare the performance in TCEC with leela's performance in the test chess.com event - where it scored 4/5 against stronger opponents, albeit with different hardware, and played like leela. (or perhaps it is all a time control/nodes per second/depth-width effect)
frankp wrote: ↑Thu Aug 16, 2018 9:06 am
There are some strange things going on with leela at TCEC.
The event has exposed 'bugs' and optimisations to be addressed, but that may not be the whole explanation.
Wonder if there are also hardware problems.
For example, compare the performance in TCEC with leela's performance in the test chess.com event - where it scored 4/5 against stronger opponents, albeit with different hardware, and played like leela. (or perhaps it is all a time control/nodes per second/depth-width effect)
What time control and hardware at chess.com event? The question to me is is whether Lc0 scales fine with time * hardware at lots of them. As for blitz games, the progress lately is remarkable. The nets in the region 10770-10780 are very strong. I got at 2' + 2'' on GTX 1060 against SF8 on 1 core the following:
Score of lc0_v16 10774 vs SF 8: +20 -10 =50 [0.563]
Elo difference: 43.66 +/- 46.78
80 of 80 games finished.
Probably in these conditions, Lc0 is already close to SF8 on 2 cores. When it will achieve the level of SF8 on 4 cores, I could casually say "Lc0 is the level of SF8", like people are used to say about engines, meaning on home reasonable balanced CPU/GPU hardware configuration and blitz time controls.
chrisw wrote: ↑Wed Aug 15, 2018 12:58 pm
Poor score indicates something is broken. Which is always likely with entered late changes.
High draw rate indicates higher than "normal" width to depth ratio. In general depth finds interesting lines and possible wins. Width defends against overlooking stuff.
More width and less depth = safe, defensive play, but dull. High draw rate. Simples.
Could be broken somewhere, could be late changes conspired to alter the search profile, could be that the scaling to more nodes tends to reflect in width rather than depth. Could be random, but the operating assumption has to be that there is a problem.
I do not think LC0 developers made serious changes what drastically lessens the chess power of LC0.
The main evidence for it the difference between LC0 and DeusX does not change a lot in the games of TCEC 13 Division 3.
As I mentioned yet the weakness is in connection with NN namely the size of NN and the filling of NN.
The knowledge what is absent in NN the MCTS can not substitute fully.
The current state of Leela and DeusX is only good for STC games - that is all.
I do not see how the fact that the difference between LC0 and DeusX did not change by a lot is an evidence that LC0 developers did not do serious changes that reduce the power of LC0
I expected LC0 to perform clearly better than division 3 based on what I read about the new net so
I guess that they increased the power of LC0 by at least 100 elo by a new net but also reduced the power of it by at least 100 elo by other changes.
frankp wrote: ↑Thu Aug 16, 2018 9:06 am
There are some strange things going on with leela at TCEC.
The event has exposed 'bugs' and optimisations to be addressed, but that may not be the whole explanation.
Wonder if there are also hardware problems.
For example, compare the performance in TCEC with leela's performance in the test chess.com event - where it scored 4/5 against stronger opponents, albeit with different hardware, and played like leela. (or perhaps it is all a time control/nodes per second/depth-width effect)
What time control and hardware at chess.com event? The question to me is is whether Lc0 scales fine with time * hardware at lots of them. As for blitz games, the progress lately is remarkable. The nets in the region 10770-10780 are very strong. I got at 2' + 2'' on GTX 1060 against SF8 on 1 core the following:
Score of lc0_v16 10774 vs SF 8: +20 -10 =50 [0.563]
Elo difference: 43.66 +/- 46.78
80 of 80 games finished.
Probably in these conditions, Lc0 is already close to SF8 on 2 cores. When it will achieve the level of SF8 on 4 cores, I could casually say "Lc0 is the level of SF8", like people are used to say about engines, meaning on home reasonable balanced CPU/GPU hardware configuration and blitz time controls.
https://www.chess.com/news/view/announc ... ampionship
GPU to be announced. I thought I heard 4 x v Tesla v100, but this may be wishful thinking. On the test leela was getting in the region of 70k n/s at the time I was watching.
corres wrote: ↑Wed Aug 15, 2018 6:31 pm
...
The current state of Leela and DeusX is only good for STC games - that is all.
I do not see how the fact that the difference between LC0 and DeusX did not change by a lot is an evidence that LC0 developers did not do serious changes that reduce the power of LC0
I expected LC0 to perform clearly better than division 3 based on what I read about the new net so
I guess that they increased the power of LC0 by at least 100 elo by a new net but also reduced the power of it by at least 100 elo by other changes.
The results what you see about new net is derived from more weaker (~very Short TC games on TCEC) systems than TCEC uses.
Laskos wrote: ↑Thu Aug 16, 2018 10:28 am
Probably in these conditions, Lc0 is already close to SF8 on 2 cores. When it will achieve the level of SF8 on 4 cores, I could casually say "Lc0 is the level of SF8", like people are used to say about engines, meaning on home reasonable balanced CPU/GPU hardware configuration and blitz time controls.
You keep repeating that but it doesn't make it true. A price-wise balanced CPU/GPU configuration is 16 threads on 8 cores Ryzen vs. 1060 (plus cheap CPU).
Power (TDP) balanced one is 32 threads (16 cores) Ryzen (Threadripper) vs 1060 (plus lower power CPU).
I think at first we would know how scale LC0 on Nvidia GPUs.
We would make a lot of tests with the same net and 1xGTX 1060, 2xGTX 1 60, 1xGTX 1080 T1, 2xGTX 1080 Ti, etc. to take the picture of Leela scaling.
Basing this knowledge and knowledge of AB engine scaling we can decide what a GPU is needed by Leela to gain such a boost what AB engines get on TCEC hardware.
Is there anybody who really actually knows with sufficient testing and good data how the LC0 MCTS algo scales up and beyond TCEC playing conditions? I don't think so.
Milos wrote: ↑Thu Aug 16, 2018 2:03 pm A price-wise balanced CPU/GPU configuration is 16 threads on 8 cores Ryzen vs. 1060 (plus cheap CPU).
Power (TDP) balanced one is 32 threads (16 cores) Ryzen (Threadripper) vs 1060 (plus lower power CPU).
What sort of nps ratio Lc0:SF do these two balanced configs give?
(I know it depends on many things, but just something typical e.g. for larger NN, no TB hits, etc.)
