Page 46 of 61

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Posted: Wed May 16, 2018 4:29 pm
by Albert Silver
yanquis1972 wrote: Wed May 16, 2018 11:24 am do they say "this graph is bogus btw" in their own paper, or would i have to read the dozens of articles to find that?

i guess the main problem is, while chess & chess engines fascinate me & are an on/off hobby, i have no knowledge in the sciences. did i miss a forum rule? is there one particular degree i should pursue or will i need a couple? is post-grad at least recommended? would appreciate any further advice on what i need to do before posting on the internet.
Why would you need a degree in sciences to look up an article and read it? It's like 5 pages long. You can skim over the mathematics and just read the results and then you would know what everyone else here knows instead of guessing or stumbling. If something fascinates you, doesn't it make sense to try to know things as opposed to guessing? I expect the stumbling block here is that for me being interested in something means reading and trying to learn about these things. I find it incomprehensible that anyone who declares themselves interested at first, much less fascinated, would make excuses to not try to inform themselves at all. Oh, and for the record, I studied French Lit, not math or computer science.

https://en.chessbase.com/post/the-futur ... arns-chess
https://en.chessbase.com/post/leela-che ... for-the-pc

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Posted: Wed May 16, 2018 6:19 pm
by Laskos
Albert Silver wrote: Tue May 15, 2018 7:14 pm
Laskos wrote: Tue May 15, 2018 6:02 pm
Albert Silver wrote: Tue May 15, 2018 5:50 pm

You mean as opposed to the normal builds, or are you referring to the normal builds? The LC0-cudnn builds are indeed the strongest, though they only run in machines equipped with Nvidia GPUs.

The self-attributed ratings for the NN are unreliable IMHO. I ran a 300-game match with v10 between NN223 and NN253, and they were about equal (facing each other). NN223 actually pulled fractionally ahead (+8 Elo) but well within the error margins obviously.
I just posted in this thread the result for ID292, it is the strongest ever (the standard v0.10 CPU and GPU build on master).
If confirmed, that will be very promising as it will clearly indicates that not only was the rut the neural network was in caused by the bug but that it is also past it and finally making genuine progress.
Yes, it was outside error margins. Since ID292, again something fishy happens, ID300 in their self-games is about 100 Elo points stronger than ID292, but against AB engine in my test, I see no improvement.

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Posted: Wed May 16, 2018 7:07 pm
by yanquis1972
Albert Silver wrote: Wed May 16, 2018 4:29 pm
yanquis1972 wrote: Wed May 16, 2018 11:24 am do they say "this graph is bogus btw" in their own paper, or would i have to read the dozens of articles to find that?

i guess the main problem is, while chess & chess engines fascinate me & are an on/off hobby, i have no knowledge in the sciences. did i miss a forum rule? is there one particular degree i should pursue or will i need a couple? is post-grad at least recommended? would appreciate any further advice on what i need to do before posting on the internet.
Why would you need a degree in sciences to look up an article and read it? It's like 5 pages long. You can skim over the mathematics and just read the results and then you would know what everyone else here knows instead of guessing or stumbling. If something fascinates you, doesn't it make sense to try to know things as opposed to guessing? I expect the stumbling block here is that for me being interested in something means reading and trying to learn about these things. I find it incomprehensible that anyone who declares themselves interested at first, much less fascinated, would make excuses to not try to inform themselves at all. Oh, and for the record, I studied French Lit, not math or computer science.

https://en.chessbase.com/post/the-futur ... arns-chess
https://en.chessbase.com/post/leela-che ... for-the-pc
i can be interested & not care to be an expert, or enjoy being informed but not wish to be consumed. i have several other things that interest be more, & for my own mental health (not a shot at anyone here, i'm talking about me) i can't devote hours to abstractions right now. i don't see what it matters, & i assumed there would be no point in a layman reading the AZ paper. that's my mistake & i'll take your advice & check it out.

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Posted: Wed May 16, 2018 7:56 pm
by jp
yanquis1972 wrote: Wed May 16, 2018 7:07 pm
Albert Silver wrote: Wed May 16, 2018 4:29 pm
yanquis1972 wrote: Wed May 16, 2018 11:24 am do they say "this graph is bogus btw" in their own paper, or would i have to read the dozens of articles to find that?
It's like 5 pages long. You can skim over the mathematics and just read the results and then you would know what everyone else here knows instead of guessing or stumbling.
i can be interested & not care to be an expert, or enjoy being informed but not wish to be consumed. i have several other things that interest be more, & for my own mental health (not a shot at anyone here, i'm talking about me) i can't devote hours to abstractions right now. i don't see what it matters, & i assumed there would be no point in a layman reading the AZ paper. that's my mistake & i'll take your advice & check it out.
Yeah, read it to get the idea but don't swallow everything it claims. Its defects are in those dozens of articles...
e.g.
https://medium.com/@josecamachocollados ... 66ae1c84f2

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Posted: Wed May 16, 2018 8:34 pm
by Jesse Gersenson
Laskos wrote: Wed May 16, 2018 6:19 pm ...against AB engine in my test, I see no improvement....
What are the conditions of the test?

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Posted: Wed May 16, 2018 9:09 pm
by Albert Silver
jp wrote: Wed May 16, 2018 7:56 pm
yanquis1972 wrote: Wed May 16, 2018 7:07 pm
Albert Silver wrote: Wed May 16, 2018 4:29 pm
It's like 5 pages long. You can skim over the mathematics and just read the results and then you would know what everyone else here knows instead of guessing or stumbling.
i can be interested & not care to be an expert, or enjoy being informed but not wish to be consumed. i have several other things that interest be more, & for my own mental health (not a shot at anyone here, i'm talking about me) i can't devote hours to abstractions right now. i don't see what it matters, & i assumed there would be no point in a layman reading the AZ paper. that's my mistake & i'll take your advice & check it out.
Yeah, read it to get the idea but don't swallow everything it claims. Its defects are in those dozens of articles...
e.g.
https://medium.com/@josecamachocollados ... 66ae1c84f2
I didn't really see anything particular pertinent. Complaints about hardware differences don't make much sense since the design requires different hardware to work. His argument about their having access to special hardware to do it in so little time is correct, but then: so what? He also mistakenly says that the 7-piece tablebases were done with much less computing power, which is hard to measure, and most likely dead wrong. They were done over a period of months, and a lot of computer time on the Lomonosov Supercomputer. That is neither weak, nor accessible to the Average Joe, no matter how deep their pockets. The rest is just a lot of talk on the match conditions, which is to completely miss the point: the novel approach and the self-learning setup. The match was just designed to test it out, and show how it performed, however it was also completely incidental.

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Posted: Wed May 16, 2018 9:37 pm
by Laskos
Jesse Gersenson wrote: Wed May 16, 2018 8:34 pm
Laskos wrote: Wed May 16, 2018 6:19 pm ...against AB engine in my test, I see no improvement....
What are the conditions of the test?
1s/move on 1 CPU thread against 1 weak AB engine. Master v0.10 engine. Not very sound conditions, but I usually saw my results later confirmed by more soud gauntlets. I will soon have my GTX 1060, so the conditions will change dramatically.

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Posted: Wed May 16, 2018 10:27 pm
by Werewolf
Laskos wrote: Wed May 16, 2018 6:19 pm
Since ID292, again something fishy happens, ID300 in their self-games is about 100 Elo points stronger than ID292, but against AB engine in my test, I see no improvement.
Does this still hold true for ID 302? The self play elo has really gone up now.

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Posted: Thu May 17, 2018 12:22 am
by Laskos
Werewolf wrote: Wed May 16, 2018 10:27 pm
Laskos wrote: Wed May 16, 2018 6:19 pm
Since ID292, again something fishy happens, ID300 in their self-games is about 100 Elo points stronger than ID292, but against AB engine in my test, I see no improvement.
Does this still hold true for ID 302? The self play elo has really gone up now.
Yes, the same for ID302 compared to ID292, no improvement (well, within error margins, so there is maybe at most 20 Elo points improvement). They see 130 Elo points improvement in self-games. Either something is wrong with my testing, or again something is fishy in their framework.

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Posted: Thu May 17, 2018 3:45 am
by David Xu
jp wrote: Wed May 16, 2018 7:56 pm
yanquis1972 wrote: Wed May 16, 2018 7:07 pm
Albert Silver wrote: Wed May 16, 2018 4:29 pm
It's like 5 pages long. You can skim over the mathematics and just read the results and then you would know what everyone else here knows instead of guessing or stumbling.
i can be interested & not care to be an expert, or enjoy being informed but not wish to be consumed. i have several other things that interest be more, & for my own mental health (not a shot at anyone here, i'm talking about me) i can't devote hours to abstractions right now. i don't see what it matters, & i assumed there would be no point in a layman reading the AZ paper. that's my mistake & i'll take your advice & check it out.
Yeah, read it to get the idea but don't swallow everything it claims. Its defects are in those dozens of articles...
e.g.
https://medium.com/@josecamachocollados ... 66ae1c84f2
I have a question for you: do you have any idea what you're talking about when you comment in these threads?