87% of the poll voted that SF will win Tcec Sufi 19.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
peter
Posts: 2426
Joined: Sat Feb 16, 2008 6:38 am
Full name: Peter Martan

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by peter » Mon Oct 05, 2020 7:30 am

Nay Lin Tun wrote:
Sun Oct 04, 2020 4:38 pm
Reasons?
Coincidence.

After 30 games 80% remis, of the 6 games with full poinst, 4 were 1:1 pairs.

King chance with such performance- differences after 100 games still ruling absolutistically, after 30 games it's even just a cruel rule regards
Peter.

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Laskos » Mon Oct 05, 2020 7:46 am

Jouni wrote:
Mon Oct 05, 2020 5:14 am
I am really surprised about current tie! SF won last superfinal and according to framework it's improved by 100 points. Lc0 is improved more?
No, nothing yet to infer. 15 pairs of games side-reversed. The pairwise score is +1 -1 =13. The pentanomial here degenerates to trinomial. It seems it's hard to find appropriate openings at LTC with very strong engines on very big hardware, so that the true draw ratio pairwise doesn't get high.

ps What hardware they used in superfinal season 18? Here it seems the old Leela ratio is about 1.6-1.7, which is pretty favorable to Leela.

User avatar
yurikvelo
Posts: 634
Joined: Sat Dec 06, 2014 12:53 pm

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by yurikvelo » Mon Oct 05, 2020 8:29 am

Laskos wrote:
Mon Oct 05, 2020 7:46 am

ps What hardware they used in superfinal season 18?
https://wiki.chessdom.org/TCEC_Season_1 ... C_Hardware

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Laskos » Mon Oct 05, 2020 9:12 am

yurikvelo wrote:
Mon Oct 05, 2020 8:29 am
Laskos wrote:
Mon Oct 05, 2020 7:46 am

ps What hardware they used in superfinal season 18?
https://wiki.chessdom.org/TCEC_Season_1 ... C_Hardware
Thanks, seems the same a bit Leela favoring hardware. At that time I had some data that the SV net used with Lc0 was scaling badly. Maybe this among other things has to do with Lc0 clear loss to SF classical.

User avatar
yurikvelo
Posts: 634
Joined: Sat Dec 06, 2014 12:53 pm

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by yurikvelo » Mon Oct 05, 2020 9:14 am

LC0 was performing 30...50 knps in the opening, SF 130...145 MNps
network was v0.25.1-svjio-t60-3972-mlh

https://wiki.chessdom.org/TCEC_Season_1 ... C_Hardware
now it is up to 70-80 knps on the same hardware [v0.26.3-rc1_T60.SV.JH.92-190]
SF-NNUE is 90-100 MNps
Last edited by yurikvelo on Mon Oct 05, 2020 9:25 am, edited 2 times in total.

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Laskos » Mon Oct 05, 2020 9:22 am

yurikvelo wrote:
Mon Oct 05, 2020 9:14 am
LC0 was performing 30...50 knps in the opening, SF 130...145 MNps
network was v0.25.1-svjio-t60-3972-mlh
Thanks, and Lc0 some 60 knps in midgames, right? The SV 384x30 nets are 3.3-3.4 times computationally larger than his earlier 256x20 nets based on T40. To get the old Leela ratio for current TCEC, you have to multiply by this factor the current speed of Lc0 in TCEC.

MMarco
Posts: 190
Joined: Sat Apr 11, 2020 11:09 pm
Full name: Marc-O Moisan-Plante

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by MMarco » Mon Oct 05, 2020 10:20 am

Laskos wrote:
Mon Oct 05, 2020 9:22 am
yurikvelo wrote:
Mon Oct 05, 2020 9:14 am
LC0 was performing 30...50 knps in the opening, SF 130...145 MNps
network was v0.25.1-svjio-t60-3972-mlh
Thanks, and Lc0 some 60 knps in midgames, right? The SV 384x30 nets are 3.3-3.4 times computationally larger than his earlier 256x20 nets based on T40. To get the old Leela ratio for current TCEC, you have to multiply by this factor the current speed of Lc0 in TCEC.
And we should multiply Stockfish nodes by the ratio of Classical nps / NNUE nps. A Stockfish dev also mentionned that SF (classical) was going slower over time and that should taken into account too if we really want to precisely reconstruct the leela-ratio.

In anycase, it is the same hardware as last season where Leela got soundly beaten. Lets see how it unfolds this time. In my opinion the scaling problem you uncovered might have been the main one. New nets by JHortos are not plagued by this. Two weeks ago I did a quick test with J92-XX and the T60-3010 that you had used for your example. J92-XX was now winning by a similar margin (or even larger, I can't remember) at the longer time control. It shown no signs of bad scaling for sure. Also the newer binary has some improvements (apart from the speed-up) that should help Leela reaching higher depths (one being "twofoldsdraws"). MLH settings been changed too. I would be curious the see a match at 3m+2s or similar between Lc0 tcec-18 and Lc0 tcec-19. That would help to evaluate the apparent Stockfish underperformance.

Nay Lin Tun
Posts: 705
Joined: Mon Jan 16, 2012 5:34 am

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Nay Lin Tun » Mon Oct 05, 2020 11:45 am

Nay Lin Tun wrote:
Sun Oct 04, 2020 4:38 pm
Some people even say Leela winning Sufi is impossible.


And now it is still tied for 26 games so far.
https://cdn.discordapp.com/attachments/ ... sufi19.png


Reasons? :D :mrgreen: :mrgreen:
We have got the answers. :twisted:
Vote yours if you wish!

https://strawpoll.com/awvxb45k5/r

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Laskos » Mon Oct 05, 2020 1:19 pm

MMarco wrote:
Mon Oct 05, 2020 10:20 am
Laskos wrote:
Mon Oct 05, 2020 9:22 am
yurikvelo wrote:
Mon Oct 05, 2020 9:14 am
LC0 was performing 30...50 knps in the opening, SF 130...145 MNps
network was v0.25.1-svjio-t60-3972-mlh
Thanks, and Lc0 some 60 knps in midgames, right? The SV 384x30 nets are 3.3-3.4 times computationally larger than his earlier 256x20 nets based on T40. To get the old Leela ratio for current TCEC, you have to multiply by this factor the current speed of Lc0 in TCEC.
And we should multiply Stockfish nodes by the ratio of Classical nps / NNUE nps. A Stockfish dev also mentionned that SF (classical) was going slower over time and that should taken into account too if we really want to precisely reconstruct the leela-ratio.

In anycase, it is the same hardware as last season where Leela got soundly beaten. Lets see how it unfolds this time. In my opinion the scaling problem you uncovered might have been the main one. New nets by JHortos are not plagued by this. Two weeks ago I did a quick test with J92-XX and the T60-3010 that you had used for your example. J92-XX was now winning by a similar margin (or even larger, I can't remember) at the longer time control. It shown no signs of bad scaling for sure. Also the newer binary has some improvements (apart from the speed-up) that should help Leela reaching higher depths (one being "twofoldsdraws"). MLH settings been changed too. I would be curious the see a match at 3m+2s or similar between Lc0 tcec-18 and Lc0 tcec-19. That would help to evaluate the apparent Stockfish underperformance.
Yes, for current TCEC, one has to consider the new Stockfish NPS too. From Youri numbers in the current Sufi I estimate an old Leela Ratio of about 1.5 or so. Certainly Lc0 is not disfavored, especially with the newer faster compiles.
So, you tackled too the issue with possible bad scaling of SV 3010 and later nets, right? Interesting, if J-92 nets scale well indeed as you found out, this Sufi might look different from expected. I might check too the scaling of J92-190 versus SV 3972mlh used in Season 18.

User avatar
CMCanavessi
Posts: 1036
Joined: Thu Dec 28, 2017 3:06 pm
Location: Argentina

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by CMCanavessi » Mon Oct 05, 2020 3:58 pm

Leela leading now after an amazing game 33. Check it out if you can, it was really cool. Reverse is playing right now.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls

Post Reply