Page 2 of 10
Re: 87% of the poll voted that SF will win Tcec Sufi 19.
Posted: Mon Oct 05, 2020 9:30 am
by peter
Coincidence.
After 30 games 80% remis, of the 6 games with full poinst, 4 were 1:1 pairs.
King chance with such performance- differences after 100 games still ruling absolutistically, after 30 games it's even just a cruel rule regards
Re: 87% of the poll voted that SF will win Tcec Sufi 19.
Posted: Mon Oct 05, 2020 9:46 am
by Laskos
Jouni wrote: ↑Mon Oct 05, 2020 7:14 am
I am really surprised about current tie! SF won last superfinal and according to framework it's improved by 100 points. Lc0 is improved more?
No, nothing yet to infer. 15 pairs of games side-reversed. The pairwise score is +1 -1 =13. The pentanomial here degenerates to trinomial. It seems it's hard to find appropriate openings at LTC with very strong engines on very big hardware, so that the true draw ratio pairwise doesn't get high.
ps What hardware they used in superfinal season 18? Here it seems the old Leela ratio is about 1.6-1.7, which is pretty favorable to Leela.
Re: 87% of the poll voted that SF will win Tcec Sufi 19.
Posted: Mon Oct 05, 2020 10:29 am
by yurikvelo
Laskos wrote: ↑Mon Oct 05, 2020 9:46 am
ps What hardware they used in superfinal season 18?
https://wiki.chessdom.org/TCEC_Season_1 ... C_Hardware
Re: 87% of the poll voted that SF will win Tcec Sufi 19.
Posted: Mon Oct 05, 2020 11:12 am
by Laskos
Thanks, seems the same a bit Leela favoring hardware. At that time I had some data that the SV net used with Lc0 was scaling badly. Maybe this among other things has to do with Lc0 clear loss to SF classical.
Re: 87% of the poll voted that SF will win Tcec Sufi 19.
Posted: Mon Oct 05, 2020 11:14 am
by yurikvelo
LC0 was performing 30...50 knps in the opening, SF 130...145 MNps
network was v0.25.1-svjio-t60-3972-mlh
https://wiki.chessdom.org/TCEC_Season_1 ... C_Hardware
now it is up to 70-80 knps on the same hardware [v0.26.3-rc1_T60.SV.JH.92-190]
SF-NNUE is 90-100 MNps
Re: 87% of the poll voted that SF will win Tcec Sufi 19.
Posted: Mon Oct 05, 2020 11:22 am
by Laskos
yurikvelo wrote: ↑Mon Oct 05, 2020 11:14 am
LC0 was performing 30...50 knps in the opening, SF 130...145 MNps
network was v0.25.1-svjio-t60-3972-mlh
Thanks, and Lc0 some 60 knps in midgames, right? The SV 384x30 nets are 3.3-3.4 times computationally larger than his earlier 256x20 nets based on T40. To get the old Leela ratio for current TCEC, you have to multiply by this factor the current speed of Lc0 in TCEC.
Re: 87% of the poll voted that SF will win Tcec Sufi 19.
Posted: Mon Oct 05, 2020 12:20 pm
by MMarco
Laskos wrote: ↑Mon Oct 05, 2020 11:22 am
yurikvelo wrote: ↑Mon Oct 05, 2020 11:14 am
LC0 was performing 30...50 knps in the opening, SF 130...145 MNps
network was v0.25.1-svjio-t60-3972-mlh
Thanks, and Lc0 some 60 knps in midgames, right? The SV 384x30 nets are 3.3-3.4 times computationally larger than his earlier 256x20 nets based on T40. To get the old Leela ratio for current TCEC, you have to multiply by this factor the current speed of Lc0 in TCEC.
And we should multiply Stockfish nodes by the ratio of Classical nps / NNUE nps. A Stockfish dev also mentionned that SF (classical) was going slower over time and that should taken into account too if we really want to precisely reconstruct the leela-ratio.
In anycase, it is the same hardware as last season where Leela got soundly beaten. Lets see how it unfolds this time. In my opinion the scaling problem you uncovered might have been the main one. New nets by JHortos are not plagued by this. Two weeks ago I did a quick test with J92-XX and the T60-3010 that you had used for your example. J92-XX was now winning by a similar margin (or even larger, I can't remember) at the longer time control. It shown no signs of bad scaling for sure. Also the newer binary has some improvements (apart from the speed-up) that should help Leela reaching higher depths (one being "twofoldsdraws"). MLH settings been changed too. I would be curious the see a match at 3m+2s or similar between Lc0 tcec-18 and Lc0 tcec-19. That would help to evaluate the apparent Stockfish underperformance.
Re: 87% of the poll voted that SF will win Tcec Sufi 19.
Posted: Mon Oct 05, 2020 1:45 pm
by Nay Lin Tun
We have got the answers.
Vote yours if you wish!
https://strawpoll.com/awvxb45k5/r
Re: 87% of the poll voted that SF will win Tcec Sufi 19.
Posted: Mon Oct 05, 2020 3:19 pm
by Laskos
MMarco wrote: ↑Mon Oct 05, 2020 12:20 pm
Laskos wrote: ↑Mon Oct 05, 2020 11:22 am
yurikvelo wrote: ↑Mon Oct 05, 2020 11:14 am
LC0 was performing 30...50 knps in the opening, SF 130...145 MNps
network was v0.25.1-svjio-t60-3972-mlh
Thanks, and Lc0 some 60 knps in midgames, right? The SV 384x30 nets are 3.3-3.4 times computationally larger than his earlier 256x20 nets based on T40. To get the old Leela ratio for current TCEC, you have to multiply by this factor the current speed of Lc0 in TCEC.
And we should multiply Stockfish nodes by the ratio of Classical nps / NNUE nps. A Stockfish dev also mentionned that SF (classical) was going slower over time and that should taken into account too if we really want to precisely reconstruct the leela-ratio.
In anycase, it is the same hardware as last season where Leela got soundly beaten. Lets see how it unfolds this time. In my opinion the scaling problem you uncovered might have been the main one. New nets by JHortos are not plagued by this. Two weeks ago I did a quick test with J92-XX and the T60-3010 that you had used for your example. J92-XX was now winning by a similar margin (or even larger, I can't remember) at the longer time control. It shown no signs of bad scaling for sure. Also the newer binary has some improvements (apart from the speed-up) that should help Leela reaching higher depths (one being "twofoldsdraws"). MLH settings been changed too. I would be curious the see a match at 3m+2s or similar between Lc0 tcec-18 and Lc0 tcec-19. That would help to evaluate the apparent Stockfish underperformance.
Yes, for current TCEC, one has to consider the new Stockfish NPS too. From Youri numbers in the current Sufi I estimate an old Leela Ratio of about 1.5 or so. Certainly Lc0 is not disfavored, especially with the newer faster compiles.
So, you tackled too the issue with possible bad scaling of SV 3010 and later nets, right? Interesting, if J-92 nets scale well indeed as you found out, this Sufi might look different from expected. I might check too the scaling of J92-190 versus SV 3972mlh used in Season 18.
Re: 87% of the poll voted that SF will win Tcec Sufi 19.
Posted: Mon Oct 05, 2020 5:58 pm
by CMCanavessi
Leela leading now after an amazing game 33. Check it out if you can, it was really cool. Reverse is playing right now.
