TCEC Question

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Leo
Posts: 925
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

TCEC Question

Post by Leo » Sat Jun 27, 2020 11:54 pm

Why is SF doing so well against LCO in the latest TCEC?
Advanced Micro Devices fan.

Dann Corbit
Posts: 11169
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: TCEC Question

Post by Dann Corbit » Sun Jun 28, 2020 2:16 am

In less than 1000 games practically any outcome is possible amongst approximate equals.
I guess that they are very close to equal, but SF had some fortunate outcomes.
And if SF is stronger, it is not by an enormous margain, as evidenced by the draw count.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Leo
Posts: 925
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: TCEC Question

Post by Leo » Sun Jun 28, 2020 2:25 am

OK.
Advanced Micro Devices fan.

ernest
Posts: 1892
Joined: Wed Mar 08, 2006 7:30 pm

Re: TCEC Question

Post by ernest » Mon Jun 29, 2020 12:08 am

Dann Corbit wrote:
Sun Jun 28, 2020 2:16 am
And if SF is stronger, it is not by an enormous margain, as evidenced by the draw count.
Any idea if TCEC-hardware wise, the current Lc0-Stockfish contest is "fair" ?
Looking for some LeelaRatio clues, I noticed that going out of book, the Stockfish Knodes/sec were more than 2000 times those of Lc0. Isn't that a bit much (for fairness...) ?

Leo
Posts: 925
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: TCEC Question

Post by Leo » Mon Jun 29, 2020 12:24 am

ernest wrote:
Mon Jun 29, 2020 12:08 am
Dann Corbit wrote:
Sun Jun 28, 2020 2:16 am
And if SF is stronger, it is not by an enormous margain, as evidenced by the draw count.
Any idea if TCEC-hardware wise, the current Lc0-Stockfish contest is "fair" ?
Looking for some LeelaRatio clues, I noticed that going out of book, the Stockfish Knodes/sec were more than 2000 times those of Lc0. Isn't that a bit much (for fairness...) ?
I haven't heard anyone complaining about the fairness for a long time. I think they have it set up fair myself.

"The hardware for CPU engines was doubled to 88 cores, while the hardware for GPU engines was raised to 4 RTX 2080 Ti's."
Advanced Micro Devices fan.

Cornfed
Posts: 36
Joined: Sun Apr 26, 2020 9:40 pm
Full name: Brian D. Smith

Re: TCEC Question

Post by Cornfed » Mon Jun 29, 2020 12:28 am

Dann Corbit wrote:
Sun Jun 28, 2020 2:16 am
In less than 1000 games practically any outcome is possible amongst approximate equals.
I guess that they are very close to equal, but SF had some fortunate outcomes.
And if SF is stronger, it is not by an enormous margain, as evidenced by the draw count.
I think the proverbial 'sample size' answer just kind of begs the question.

What does "fortunate" mean? Did LZ0 stay out late partying the night before?

After 71 games SF leads 37.5 vs 33.5 (and game 72 looks like it will end in the Fish's favor as well...). A 4 to 5 pt lead at this point is actually reasonably significant. That said, there are more games to be played and Game 72 has LCZero defending the Latvian Counter Gambit...which is bad. Has SF yet to defend it? I don't know. The Devil is in the details.

EDIT: It has, the game before and SF lost...just as LCZero looks to at the moment.

ernest
Posts: 1892
Joined: Wed Mar 08, 2006 7:30 pm

Re: TCEC Question

Post by ernest » Mon Jun 29, 2020 12:36 am

Leo wrote:
Mon Jun 29, 2020 12:24 am
I haven't heard anyone complaining about the fairness for a long time.
Not complaining, just asking ! 8-)

(looking at the Knodes/sec)

User avatar
Ovyron
Posts: 4256
Joined: Tue Jul 03, 2007 2:30 am

Re: TCEC Question

Post by Ovyron » Mon Jun 29, 2020 12:53 am

ernest wrote:
Mon Jun 29, 2020 12:08 am
the Stockfish Knodes/sec were more than 2000 times those of Lc0
The "nodes" of Lc0 have absolutely nothing to do with the "nodes" of Stockfish because the architectures of both programs are completely different. Note the definition of NPS:

Nodes per second, including NNCache hits and terminal node hits, but excluding nodes carried over from tree reuse. In other words, total playouts generated per second.

Leela nodes = playouts (Stockfish doesn't do playouts at all)

ernest
Posts: 1892
Joined: Wed Mar 08, 2006 7:30 pm

Re: TCEC Question

Post by ernest » Mon Jun 29, 2020 1:03 am

Ovyron wrote:
Mon Jun 29, 2020 12:53 am
ernest wrote:
Mon Jun 29, 2020 12:08 am
the Stockfish Knodes/sec were more than 2000 times those of Lc0
The "nodes" of Lc0 have absolutely nothing to do with the "nodes" of Stockfish because the architectures of both programs are completely different. Note the definition of NPS:

Nodes per second, including NNCache hits and terminal node hits, but excluding nodes carried over from tree reuse. In other words, total playouts generated per second.

Leela nodes = playouts (Stockfish doesn't do playouts at all)
Yes, but LeelaRatio (one of the tools for "fairness") is precisely based on ratio of Knodes/sec...
Admittedly, not everybody likes LeelaRatio...

User avatar
Ovyron
Posts: 4256
Joined: Tue Jul 03, 2007 2:30 am

Re: TCEC Question

Post by Ovyron » Mon Jun 29, 2020 2:05 am

Yeah, we can't even have these discussions without putting quotes around the word "fairness."

It's like trying to make a Megahertz battle against a Megabyte "fair." People have to get into their heads these are different things.

Post Reply