TCEC S15, END of an ERA event is much more Brutal than I thought!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
supersharp77
Posts: 577
Joined: Sat Jul 05, 2014 5:54 am
Location: Southwest USA

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by supersharp77 » Fri May 17, 2019 7:41 pm

glennsamuel32 wrote:
Fri May 17, 2019 6:05 am
Here's another comparison...

Leela used 32.2 billion transistors while SF used 14.4 billion :D :D
Well my friend if that "Hardware Comparison" is true "This Match Was Indeed RIGGED"
What opening book are they using? Whats the move cutoff? Thx AR :D :wink:



ps "We believe in Fair Matches..Not Handicap Matches"

Nay Lin Tun
Posts: 455
Joined: Mon Jan 16, 2012 5:34 am

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Nay Lin Tun » Sat May 18, 2019 6:45 am

Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
Capture (2).PNG (14.54 KiB) Viewed 394 times

Joost Buijs
Posts: 822
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Joost Buijs » Sat May 18, 2019 7:30 am

Nay Lin Tun wrote:
Sat May 18, 2019 6:45 am
Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
This doesn't mean anything, with just 41 games the error margin is much higher than 50 Elo.

Nay Lin Tun
Posts: 455
Joined: Mon Jan 16, 2012 5:34 am

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Nay Lin Tun » Sat May 18, 2019 8:20 am

Joost Buijs wrote:
Sat May 18, 2019 7:30 am
Nay Lin Tun wrote:
Sat May 18, 2019 6:45 am
Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
This doesn't mean anything, with just 41 games the error margin is much higher than 50 Elo.
Of course this public sample is small and it will always small forever.
If you dont believe those sample size, then TCEC champion for Stockfish 4 times, Houdini 3 times, Komodo 3 times, etc were just flukes.

And also, there were more than 30,000 games Leela vs SF with exactly same speed ratio in TCEC conditions in leela discords,
Those tests results shows T40 is +35 elo above previous T30.
I dont want to reference them cos most people dont know those stuff and they dont care those stuff.
It looks like unfair comeptition, Leela project has tons of tests against SF but Stockfish project has zero testing against Leela.

User avatar
Laskos
Posts: 8963
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Laskos » Sat May 18, 2019 8:36 am

Nay Lin Tun wrote:
Sat May 18, 2019 8:20 am
Joost Buijs wrote:
Sat May 18, 2019 7:30 am
Nay Lin Tun wrote:
Sat May 18, 2019 6:45 am
Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
This doesn't mean anything, with just 41 games the error margin is much higher than 50 Elo.
Of course this public sample is small and it will always small forever.
If you dont believe those sample size, then TCEC champion for Stockfish 4 times, Houdini 3 times, Komodo 3 times, etc were just flukes.

And also, there were more than 30,000 games Leela vs SF with exactly same speed ratio in TCEC conditions in leela discords,
Those tests results shows T40 is +35 elo above previous T30.
I dont want to reference them cos most people dont know those stuff and they dont care those stuff.
It looks like unfair comeptition, Leela project has tons of tests against SF but Stockfish project has zero testing against Leela.
Come on, often no one needs more than 100 games. Say, if the match continues in this fashion, and the final result will be +16 -6 = 78 for Lc0 T40, then LOS will be a decisive 98.3% for Lc0, and the result will be outside 2 standard deviations error margins. Also, using pentanomial variance for paired games, error margins can be shrunk and LOS increased. We had 2 pairs of games with 1-0 result for White both sides. Roughly to get the LOS now using pentanomial variance, set each pair of won games to draws (4 more draws in total), and the "true wins result" as of now is +6 -1 for Leela, which gives a LOS of 96.4% for Leela even right now. The pentanomial computation is more complicated, but the result should be similar. And at this pace, in 100 paired games, the pentanomial error margins for paired games might well be much smaller than the difference between Lc0 and SF. I guess 100 games will be enough to have a decisive result (LOS above 95% or 98%).

chrisw
Posts: 1415
Joined: Tue Apr 03, 2012 2:28 pm

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by chrisw » Sat May 18, 2019 9:04 am

Laskos wrote:
Sat May 18, 2019 8:36 am
Nay Lin Tun wrote:
Sat May 18, 2019 8:20 am
Joost Buijs wrote:
Sat May 18, 2019 7:30 am
Nay Lin Tun wrote:
Sat May 18, 2019 6:45 am
Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
This doesn't mean anything, with just 41 games the error margin is much higher than 50 Elo.
Of course this public sample is small and it will always small forever.
If you dont believe those sample size, then TCEC champion for Stockfish 4 times, Houdini 3 times, Komodo 3 times, etc were just flukes.

And also, there were more than 30,000 games Leela vs SF with exactly same speed ratio in TCEC conditions in leela discords,
Those tests results shows T40 is +35 elo above previous T30.
I dont want to reference them cos most people dont know those stuff and they dont care those stuff.
It looks like unfair comeptition, Leela project has tons of tests against SF but Stockfish project has zero testing against Leela.
Come on, often no one needs more than 100 games. Say, if the match continues in this fashion, and the final result will be +16 -6 = 78 for Lc0 T40, then LOS will be a decisive 98.3% for Lc0, and the result will be outside 2 standard deviations error margins. Also, using pentanomial variance for paired games, error margins can be shrunk and LOS increased. We had 2 pairs of games with 1-0 result for White both sides. Roughly to get the LOS now using pentanomial variance, set each pair of won games to draws (4 more draws in total), and the "true wins result" as of now is +6 -1 for Leela, which gives a LOS of 96.4% for Leela even right now. The pentanomial computation is more complicated, but the result should be similar. And at this pace, in 100 paired games, the pentanomial error margins for paired games might well be much smaller than the difference between Lc0 and SF. I guess 100 games will be enough to have a decisive result (LOS above 95% or 98%).
If this were the High Jump (athletics), there'ld be furious arguing over 0.005mm, variations in the gravitational constant and the phase of the moon.

User avatar
Laskos
Posts: 8963
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by Laskos » Sat May 18, 2019 9:52 am

chrisw wrote:
Sat May 18, 2019 9:04 am
Laskos wrote:
Sat May 18, 2019 8:36 am
Nay Lin Tun wrote:
Sat May 18, 2019 8:20 am
Joost Buijs wrote:
Sat May 18, 2019 7:30 am
Nay Lin Tun wrote:
Sat May 18, 2019 6:45 am
Update!
After 41 games, Leela leads with +5 scores!
Leela is approx +50 elo above SF now!!
Capture (2).PNG
This doesn't mean anything, with just 41 games the error margin is much higher than 50 Elo.
Of course this public sample is small and it will always small forever.
If you dont believe those sample size, then TCEC champion for Stockfish 4 times, Houdini 3 times, Komodo 3 times, etc were just flukes.

And also, there were more than 30,000 games Leela vs SF with exactly same speed ratio in TCEC conditions in leela discords,
Those tests results shows T40 is +35 elo above previous T30.
I dont want to reference them cos most people dont know those stuff and they dont care those stuff.
It looks like unfair comeptition, Leela project has tons of tests against SF but Stockfish project has zero testing against Leela.
Come on, often no one needs more than 100 games. Say, if the match continues in this fashion, and the final result will be +16 -6 = 78 for Lc0 T40, then LOS will be a decisive 98.3% for Lc0, and the result will be outside 2 standard deviations error margins. Also, using pentanomial variance for paired games, error margins can be shrunk and LOS increased. We had 2 pairs of games with 1-0 result for White both sides. Roughly to get the LOS now using pentanomial variance, set each pair of won games to draws (4 more draws in total), and the "true wins result" as of now is +6 -1 for Leela, which gives a LOS of 96.4% for Leela even right now. The pentanomial computation is more complicated, but the result should be similar. And at this pace, in 100 paired games, the pentanomial error margins for paired games might well be much smaller than the difference between Lc0 and SF. I guess 100 games will be enough to have a decisive result (LOS above 95% or 98%).
If this were the High Jump (athletics), there'ld be furious arguing over 0.005mm, variations in the gravitational constant and the phase of the moon.
:lol:

Yes, exactly.

supersharp77
Posts: 577
Joined: Sat Jul 05, 2014 5:54 am
Location: Southwest USA

Re: TCEC S15, END of an ERA event is much more Brutal than I thought!

Post by supersharp77 » Sat May 18, 2019 9:30 pm

JJJ wrote:
Wed May 15, 2019 5:09 pm
Stockfish is now the Komodo of lczero. It is first in rating list but loosing in direct confontation.
End of Era will be when Lczero will be the first in rating list as well.
LC0 Settings...Observe NN

Re: TCEC Lc0 Configuration
Quote
Post by arunsoorya1309 » Fri May 17, 2019 7:13 pm

{
"name": "MoveOverheadMs",
"value": "2000"
},
{
"name": "WeightsFile",
"value": "xxxx/T40.T8.610.pb.gz"
},
{
"name": "Threads",
"value": "3"
},
{
"name": "NNCacheSize", *************** :wink:
"value": "20000000" ========== :lol:
},
{
"name": "MinibatchSize",
"value": "256"
},
{
"name": "MaxCollisionEvents",
"value": "32"
},
{
"name": "CPuct",
"value": "3.4"
},
{
"name": "CPuctBase",
"value": "10000"
},
{
"name": "MaxPrefetch",
"value": "32"
},
{
"name": "RamLimitMb",
"value": "22528"
},
{
"name": "LogFile",
"value": "Sufi.txt"
},
{
"name": "SyzygyPath",
"value": "xxx"
},
{
"name": "Ponder",
"value": "false"
},
{
"name": "CommandLineOptions",
"value": "--backend=roundrobin --backend-opts=\"(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)\""


Neural Net... 20 Million......

Post Reply