TCEC S15, END of an ERA event is much more Brutal than I thought!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
supersharp77
Posts: 753
Joined: Sat Jul 05, 2014 5:54 am
Location: Southwest USA

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by supersharp77 » Fri May 17, 2019 7:41 pm

glennsamuel32 wrote:
Fri May 17, 2019 6:05 am
Here's another comparison...

Leela used 32.2 billion transistors while SF used 14.4 billion :D :D
Well my friend if that "Hardware Comparison" is true "This Match Was Indeed RIGGED"
What opening book are they using? Whats the move cutoff? Thx AR :D :wink:



ps "We believe in Fair Matches..Not Handicap Matches"

Nay Lin Tun
Posts: 520
Joined: Mon Jan 16, 2012 5:34 am

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Nay Lin Tun » Sat May 18, 2019 6:45 am

Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
Capture (2).PNG (14.54 KiB) Viewed 1674 times

Joost Buijs
Posts: 986
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Joost Buijs » Sat May 18, 2019 7:30 am

Nay Lin Tun wrote:
Sat May 18, 2019 6:45 am
Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
This doesn't mean anything, with just 41 games the error margin is much higher than 50 Elo.

Nay Lin Tun
Posts: 520
Joined: Mon Jan 16, 2012 5:34 am

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Nay Lin Tun » Sat May 18, 2019 8:20 am

Joost Buijs wrote:
Sat May 18, 2019 7:30 am
Nay Lin Tun wrote:
Sat May 18, 2019 6:45 am
Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
This doesn't mean anything, with just 41 games the error margin is much higher than 50 Elo.
Of course this public sample is small and it will always small forever.
If you dont believe those sample size, then TCEC champion for Stockfish 4 times, Houdini 3 times, Komodo 3 times, etc were just flukes.

And also, there were more than 30,000 games Leela vs SF with exactly same speed ratio in TCEC conditions in leela discords,
Those tests results shows T40 is +35 elo above previous T30.
I dont want to reference them cos most people dont know those stuff and they dont care those stuff.
It looks like unfair comeptition, Leela project has tons of tests against SF but Stockfish project has zero testing against Leela.

User avatar
Laskos
Posts: 9449
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Laskos » Sat May 18, 2019 8:36 am

Nay Lin Tun wrote:
Sat May 18, 2019 8:20 am
Joost Buijs wrote:
Sat May 18, 2019 7:30 am
Nay Lin Tun wrote:
Sat May 18, 2019 6:45 am
Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
This doesn't mean anything, with just 41 games the error margin is much higher than 50 Elo.
Of course this public sample is small and it will always small forever.
If you dont believe those sample size, then TCEC champion for Stockfish 4 times, Houdini 3 times, Komodo 3 times, etc were just flukes.

And also, there were more than 30,000 games Leela vs SF with exactly same speed ratio in TCEC conditions in leela discords,
Those tests results shows T40 is +35 elo above previous T30.
I dont want to reference them cos most people dont know those stuff and they dont care those stuff.
It looks like unfair comeptition, Leela project has tons of tests against SF but Stockfish project has zero testing against Leela.
Come on, often no one needs more than 100 games. Say, if the match continues in this fashion, and the final result will be +16 -6 = 78 for Lc0 T40, then LOS will be a decisive 98.3% for Lc0, and the result will be outside 2 standard deviations error margins. Also, using pentanomial variance for paired games, error margins can be shrunk and LOS increased. We had 2 pairs of games with 1-0 result for White both sides. Roughly to get the LOS now using pentanomial variance, set each pair of won games to draws (4 more draws in total), and the "true wins result" as of now is +6 -1 for Leela, which gives a LOS of 96.4% for Leela even right now. The pentanomial computation is more complicated, but the result should be similar. And at this pace, in 100 paired games, the pentanomial error margins for paired games might well be much smaller than the difference between Lc0 and SF. I guess 100 games will be enough to have a decisive result (LOS above 95% or 98%).

chrisw
Posts: 2104
Joined: Tue Apr 03, 2012 2:28 pm

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by chrisw » Sat May 18, 2019 9:04 am

Laskos wrote:
Sat May 18, 2019 8:36 am
Nay Lin Tun wrote:
Sat May 18, 2019 8:20 am
Joost Buijs wrote:
Sat May 18, 2019 7:30 am
Nay Lin Tun wrote:
Sat May 18, 2019 6:45 am
Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
This doesn't mean anything, with just 41 games the error margin is much higher than 50 Elo.
Of course this public sample is small and it will always small forever.
If you dont believe those sample size, then TCEC champion for Stockfish 4 times, Houdini 3 times, Komodo 3 times, etc were just flukes.

And also, there were more than 30,000 games Leela vs SF with exactly same speed ratio in TCEC conditions in leela discords,
Those tests results shows T40 is +35 elo above previous T30.
I dont want to reference them cos most people dont know those stuff and they dont care those stuff.
It looks like unfair comeptition, Leela project has tons of tests against SF but Stockfish project has zero testing against Leela.
Come on, often no one needs more than 100 games. Say, if the match continues in this fashion, and the final result will be +16 -6 = 78 for Lc0 T40, then LOS will be a decisive 98.3% for Lc0, and the result will be outside 2 standard deviations error margins. Also, using pentanomial variance for paired games, error margins can be shrunk and LOS increased. We had 2 pairs of games with 1-0 result for White both sides. Roughly to get the LOS now using pentanomial variance, set each pair of won games to draws (4 more draws in total), and the "true wins result" as of now is +6 -1 for Leela, which gives a LOS of 96.4% for Leela even right now. The pentanomial computation is more complicated, but the result should be similar. And at this pace, in 100 paired games, the pentanomial error margins for paired games might well be much smaller than the difference between Lc0 and SF. I guess 100 games will be enough to have a decisive result (LOS above 95% or 98%).
If this were the High Jump (athletics), there'ld be furious arguing over 0.005mm, variations in the gravitational constant and the phase of the moon.

User avatar
Laskos
Posts: 9449
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Laskos » Sat May 18, 2019 9:52 am

chrisw wrote:
Sat May 18, 2019 9:04 am
Laskos wrote:
Sat May 18, 2019 8:36 am
Nay Lin Tun wrote:
Sat May 18, 2019 8:20 am
Joost Buijs wrote:
Sat May 18, 2019 7:30 am
Nay Lin Tun wrote:
Sat May 18, 2019 6:45 am
Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
This doesn't mean anything, with just 41 games the error margin is much higher than 50 Elo.
Of course this public sample is small and it will always small forever.
If you dont believe those sample size, then TCEC champion for Stockfish 4 times, Houdini 3 times, Komodo 3 times, etc were just flukes.

And also, there were more than 30,000 games Leela vs SF with exactly same speed ratio in TCEC conditions in leela discords,
Those tests results shows T40 is +35 elo above previous T30.
I dont want to reference them cos most people dont know those stuff and they dont care those stuff.
It looks like unfair comeptition, Leela project has tons of tests against SF but Stockfish project has zero testing against Leela.
Come on, often no one needs more than 100 games. Say, if the match continues in this fashion, and the final result will be +16 -6 = 78 for Lc0 T40, then LOS will be a decisive 98.3% for Lc0, and the result will be outside 2 standard deviations error margins. Also, using pentanomial variance for paired games, error margins can be shrunk and LOS increased. We had 2 pairs of games with 1-0 result for White both sides. Roughly to get the LOS now using pentanomial variance, set each pair of won games to draws (4 more draws in total), and the "true wins result" as of now is +6 -1 for Leela, which gives a LOS of 96.4% for Leela even right now. The pentanomial computation is more complicated, but the result should be similar. And at this pace, in 100 paired games, the pentanomial error margins for paired games might well be much smaller than the difference between Lc0 and SF. I guess 100 games will be enough to have a decisive result (LOS above 95% or 98%).
If this were the High Jump (athletics), there'ld be furious arguing over 0.005mm, variations in the gravitational constant and the phase of the moon.
:lol:

Yes, exactly.

supersharp77
Posts: 753
Joined: Sat Jul 05, 2014 5:54 am
Location: Southwest USA

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by supersharp77 » Sat May 18, 2019 9:30 pm

JJJ wrote:
Wed May 15, 2019 5:09 pm
Stockfish is now the Komodo of lczero. It is first in rating list but loosing in direct confontation.
End of Era will be when Lczero will be the first in rating list as well.
LC0 Settings...Observe NN

Re: TCEC Lc0 Configuration
Quote
Post by arunsoorya1309 » Fri May 17, 2019 7:13 pm

{
"name": "MoveOverheadMs",
"value": "2000"
},
{
"name": "WeightsFile",
"value": "xxxx/T40.T8.610.pb.gz"
},
{
"name": "Threads",
"value": "3"
},
{
"name": "NNCacheSize", *************** :wink:
"value": "20000000" ========== :lol:
},
{
"name": "MinibatchSize",
"value": "256"
},
{
"name": "MaxCollisionEvents",
"value": "32"
},
{
"name": "CPuct",
"value": "3.4"
},
{
"name": "CPuctBase",
"value": "10000"
},
{
"name": "MaxPrefetch",
"value": "32"
},
{
"name": "RamLimitMb",
"value": "22528"
},
{
"name": "LogFile",
"value": "Sufi.txt"
},
{
"name": "SyzygyPath",
"value": "xxx"
},
{
"name": "Ponder",
"value": "false"
},
{
"name": "CommandLineOptions",
"value": "--backend=roundrobin --backend-opts=\"(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)\""


Neural Net... 20 Million......

User avatar
Ozymandias
Posts: 1128
Joined: Sun Oct 25, 2009 12:30 am

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Ozymandias » Mon May 20, 2019 4:58 am

Past the half-tournament, the advantage has been clearly diminished (to +3). Don't get me wrong, Lc0 will win, I predicted as much at the same time I said SF would still win TCEC14, but the bar is set very high, it doesn't seem feasible for anyone to steer clear of it by a wide margin.

jp
Posts: 820
Joined: Mon Apr 23, 2018 5:54 am

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by jp » Mon May 20, 2019 5:42 am

Lc0 is behind SF on CCC, but it's said they sent a different NN ID to that. That's another issue: not being able to tell which NN is best.

Post Reply