I do not see a reason to assume that LC0 with adjusting the time control can mimic a 2800 fide level human opponent better than A-B engines.Laskos wrote: ↑Tue Apr 09, 2019 9:22 amI got fairly confused this morning, bad sleep probably .lkaufman wrote: ↑Tue Apr 09, 2019 5:41 amI believe those were roughly the engines that tied matches with Kasparov at standard time back then, probably running on four threads, though I'm not sure about that. I'm not sure how many times faster the hardware is today; I suppose we have to clarify whether we are talking about playing on the old machines CCRL uses as a standard or on our current machines. Since you are running your matches on your new machine, not on an old AMD like the ones used as reference by CCRL, I suppose we should pick an engine that would be Carlsen level on that hardware, and if so I think you are picking too strong an engine; I imagine that those two engines on your current machine on 1 thread are as good as whatever they had around 2003 on four threads, so it should be 2800+ level even at 40/2 hours, and hence a big favorite at 45' + 15". But you are more knowledgeable about hardware and a much better mathematician than I am, so please correct me if I am wrong.Laskos wrote: ↑Mon Apr 08, 2019 9:56 pmI am on the phone now, but that's not that hard, there were games against top humans in 2003-2004 and we can extrapolate tc and hardware. Take some Fritz 8 or Junior 9 on one core CCRL 40/40 level. My estimate it is some Kasparov or Magnus level at 45' + 15''.lkaufman wrote: ↑Mon Apr 08, 2019 9:04 pmSince we already know that this same LCO network defeated GM Naroditsky by a wide margin in blitz games averaging around knight odds, and he is probably within a class of Magnus in blitz strength, we already have good reason to think knight odds vs. Carlsen would be close, so this test is mostly just to confirm that we can make meaningful predictions of human results by these simulartions. As for 45' + 15", what CCRL 40/40 rating do you think would come closest to matching Magnus in strength at 45' + 15" ?Laskos wrote: ↑Mon Apr 08, 2019 8:06 amI am not sure at 3' + 2'' blitz, Lc0 11248 on 2080 is probably close at Knight odds to Carlsen, but my claim here:lkaufman wrote: ↑Mon Apr 08, 2019 6:43 am I ran a twenty game blitz (3' + 2") knight odds match (two ply book for variety, both b1 and g1 knights) between Lc0 11248 (on my 2080) vs. Giraffe (best version, on my 5 GHz i7 laptop), with Giraffe as a proxy for Magnus Carlsen, a good one since it "knows" to simplify when up a piece while some similarly rated A/B engines may not know or appreciate this. "Magnus" won by 12 to 8 (no draws!). So perhaps it's not yet time to bet against the champ if such a match took place, but we're getting close it seems.
http://www.talkchess.com/forum3/viewtop ... =2&t=69956
was that at longer 45' + 15'' a top GM might not win all 10 games out of 10 being a Knight up, more probably 7-9, 1-3 being drawn or even lost by the human. It would be fun to watch such a match, as the top human will be happy having an upper hand most of the times, while enjoying some 1-3 setbacks at Knight odds in 10 games! The prize could be proportional to (Wins - Draws - Losses) or even (Wins - Draws - 2*Losses), to give incentive to the human to not lose any points. Lc0 should be left with some temperature (say 0.5) for the first 4-5 moves, for diversity and for not playing into prepared openings. Or by providing Lc0 with a small, prepared book.
First baseline, which I remembered correctly:
From Wiki:
X3D Fritz was a version of the Fritz chess program, which in November 2003 played a four-game Human–computer chess match against world number one Grandmaster Garry Kasparov. The match was tied 2–2. Fritz ran on four Intel Pentium 4 Xeon CPUs at 2.8 GHz.
X3D Fritz is something in-between Fritz 8 and Fritz 8 Bilbao, and close in strength to them. Fritz 8 Bilbao itself played some 12 games against top humans (weaker than Kasparov) and won. So, all the data corroborated (similar data on some Junior 8 and 9 matches against top humans), it's reasonable to say that Fritz 8 Bilbao and Junior 9 on "four Intel Pentium 4 Xeon CPUs at 2.8 GHz" are some Kasparov/Kramnik level of 2003-4 at 40/2 hours.
One my i7 core is close to being 2.5 faster than one of those cores of Xeon. Their effective speed-up on 4 cores in those times was not that good, maybe around 2.8-3.0, so basically these Fritz and Junior engines on one my i7 core are level with Kasparov/Kramnik at 40/2 hours.
No need to extrapolate up to now.
There are two scaling issues needed for extrapolation:
1/ Scaling of human versus machine with TC
2/ Scaling of Knight odds with TC
And here I got stuck with this Leela thing. I am playing Lc0 11248 as the ODDS TAKER (handicapped engine by one Knight) and the conventional Arasan 14.3 as taking the KNIGHT UP chances. It's quite the opposite of handicapped very strong classical AB engine versus human, as we used to see all the handicaps and scaling behaviors. Here I am mimicking basically a super-human Lc0 being a Knight down against a pure classical engine Fritz 8.
Going to 1/ and 2/ --- what is "machine" in that scaling issue? The scaling of Knight odds is clear as a slope, it increases with TC, but I am not sure of magnitude, again, depending on what is "machine".
I understand your reasoning that Fritz 8, Junior 9 (both some Arasan 14.3 level) seem too strong at 45' + 15'', but my doubts about the validity of my and your usual reasoning are validated by the crazy result I got:
At 15' + 5'' in 10 games Lc0 11248 on 2070 (first 4 moves using temperature of 0.5) scored 4 Wins and 6 Losses against Arasan 14.3 on one strong i7 core at Knight Odds. I do not know what to make of this result and what it means. In our usual reasoning, that would mean that Lc0 11248 can score several wins against Magnus at Knight Odds in 10 games at 45' + 15'', but let's not say stupid things out of "usual reasoning".
I clearly need to change the opponent of Lc0 11248 from Arasan 14.3 to another Lc0 to mimic a human opponent. By the way, Arasan seems quite dumb in converting the advantage, and I need a good Leela net ID which knows how to convert large advantages. Do you know a net ID to play well being a Knight up? I will adjust its TC (or nodes) to mimic a 2800 FIDE level human, and I will explain how I did it.
And to add: CCRL ratings don't help me much here, they actually confused me more . Humans are not obeying them, Leela is not obeying them, all this mess.
I guess that LC0 knows too much things in the evaluation that 2800 GM's do not know.
I guess that if A-B engines fail to mimic 2800 humans because of a relatively stupid evaluation then lc0 nets fail to mimic 2800 humans because of a relatively stupid search.