ICGA WCCC and WCSC in Santiago de Compostela

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Viz
Posts: 108
Joined: Tue Apr 09, 2024 6:24 am
Full name: Michael Chaly

Re: ICGA WCCC and WCSC in Santiago de Compostela

Post by Viz »

Henk wrote: Fri Apr 19, 2024 3:13 pm So it is the last chance for Stockfish(developers) to finally become the real world champion. Reasons enough to participate this time.
Stockfish had become the real world champion more than dozen times. But will probably never become "champion" of bogus competition with operators, books and 8 games sample size indeed.
User avatar
hgm
Posts: 27925
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: ICGA WCCC and WCSC in Santiago de Compostela

Post by hgm »

The point isthat Stockfish is only strong on average, because that is what Elo is, an average. It is not reliably strong. Therefore it has a significant probability to loose, in a small number of games. And of course the Elo advantage on the competitors is so small that you would need many thousands of games to see it as an average.

The Stockfish developers don't like that. So they only want to participate in tournaments that measures average performance over an insanely large number of games. Because they know they only stand little chance to win a tournament over 8 games.
User avatar
Graham Banks
Posts: 41959
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: ICGA WCCC and WCSC in Santiago de Compostela

Post by Graham Banks »

hgm wrote: Fri Apr 19, 2024 10:05 pm The point isthat Stockfish is only strong on average, because that is what Elo is, an average. It is not reliably strong. Therefore it has a significant probability to loose, in a small number of games. And of course the Elo advantage on the competitors is so small that you would need many thousands of games to see it as an average.

The Stockfish developers don't like that. So they only want to participate in tournaments that measures average performance over an insanely large number of games. Because they know they only stand little chance to win a tournament over 8 games.
I think that you're underestimating Stockfish in the general context, but you're right that in a tournament based on 8 games, there is a slim chance that it might not win.
gbanksnz at gmail.com
Modern Times
Posts: 3573
Joined: Thu Jun 07, 2012 11:02 pm

Re: ICGA WCCC and WCSC in Santiago de Compostela

Post by Modern Times »

hgm wrote: Fri Apr 19, 2024 10:05 pm The point isthat Stockfish is only strong on average, because that is what Elo is, an average. It is not reliably strong. Therefore it has a significant probability to loose, in a small number of games. And of course the Elo advantage on the competitors is so small that you would need many thousands of games to see it as an average.

The Stockfish developers don't like that. So they only want to participate in tournaments that measures average performance over an insanely large number of games. Because they know they only stand little chance to win a tournament over 8 games.
I agree. They also like tightly controlled conditions like CCC and TCEC with same books for all engines (slightly unbalanced too), same hardware etc and a long run of games where they have a chance to get slightly more wins to tip the scales. It depends what other engines enter of course, but there is a very real chance Stockfish would not win. With freedom around books (which have a big impact) and hardware etc, they can't control the conditions and they don't like it. I get the impression that they don't enter because they are scared they will not win. Maybe I'm wrong on that but it is the feeling I get.
Modern Times
Posts: 3573
Joined: Thu Jun 07, 2012 11:02 pm

Re: ICGA WCCC and WCSC in Santiago de Compostela

Post by Modern Times »

Either that, the concern about the slightly random nature of the tournament, or else they just consider it completely irrelevant - like most of us.
Pali
Posts: 29
Joined: Wed Dec 01, 2021 12:23 pm
Full name: Doruk Sekercioglu

Re: ICGA WCCC and WCSC in Santiago de Compostela

Post by Pali »

Modern Times wrote: Fri Apr 19, 2024 10:52 pm
hgm wrote: Fri Apr 19, 2024 10:05 pm The point isthat Stockfish is only strong on average, because that is what Elo is, an average. It is not reliably strong. Therefore it has a significant probability to loose, in a small number of games. And of course the Elo advantage on the competitors is so small that you would need many thousands of games to see it as an average.

The Stockfish developers don't like that. So they only want to participate in tournaments that measures average performance over an insanely large number of games. Because they know they only stand little chance to win a tournament over 8 games.
I agree. They also like tightly controlled conditions like CCC and TCEC with same books for all engines (slightly unbalanced too), same hardware etc and a long run of games where they have a chance to get slightly more wins to tip the scales. It depends what other engines enter of course, but there is a very real chance Stockfish would not win. With freedom around books (which have a big impact) and hardware etc, they can't control the conditions and they don't like it. I get the impression that they don't enter because they are scared they will not win. Maybe I'm wrong on that but it is the feeling I get.
"with same books for all engines (slightly unbalanced too)"
This means fair conditions (game pairs)
"same hardware etc"
This means fair conditions
"and a long run of games"
This means more statistical significance
"With freedom around books (which have a big impact) and hardware etc"
Freedom around books means -> largest book made with the strongest engine with the most amount of search is the best -> you are likely still getting a draw fest.
Freedom around hardware -> this should simply not happen, a weaker engine running on 256 cores would be stronger than an otherwise stronger engine running on 1 core. It's the software competing, not the hardware. The one exception I can think of is cases like Leela where hardware requirements are wildly different.
User avatar
Antihelion
Posts: 30
Joined: Tue Mar 26, 2024 8:21 pm
Full name: Lyndon S. Sears

Re: ICGA WCCC and WCSC in Santiago de Compostela

Post by Antihelion »

hgm wrote: Fri Apr 19, 2024 10:05 pm The point isthat Stockfish is only strong on average, because that is what Elo is, an average. It is not reliably strong. Therefore it has a significant probability to loose, in a small number of games. And of course the Elo advantage on the competitors is so small that you would need many thousands of games to see it as an average.

The Stockfish developers don't like that. So they only want to participate in tournaments that measures average performance over an insanely large number of games. Because they know they only stand little chance to win a tournament over 8 games.
Then tell me what your 8-game tournament is supposed to measure? CPU noise?
Viz
Posts: 108
Joined: Tue Apr 09, 2024 6:24 am
Full name: Michael Chaly

Re: ICGA WCCC and WCSC in Santiago de Compostela

Post by Viz »

Last year software tournament was won by what, ginkgo(fritz)?
No offence to it author, he probably knows it himself, but this engine is nowhere of caliber of winning any engine tournament... Well, apart from abomination called WCCC/WCSC. If you play 10k games of it vs stockfish/torch I doubt it will win a single game pair on average, regardless of used book.
And you want SF to participate in this garbage, making it look legitimate? "Hey, look, great tournament, even stockfish participates there"?
How about no.
There is nothing bad in hanging out drinking beer or whatever with a bunch of devs, idiotic format that is outdated by 2 decades with pathos name "World tournament" while being your garage inside competition is not needed to do that.
User avatar
hgm
Posts: 27925
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: ICGA WCCC and WCSC in Santiago de Compostela

Post by hgm »

Antihelion wrote: Sat Apr 20, 2024 12:02 am Then tell me what your 8-game tournament is supposed to measure? CPU noise?
How reliably t he engine performs?
AndrewGrant
Posts: 1819
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: ICGA WCCC and WCSC in Santiago de Compostela

Post by AndrewGrant »

hgm wrote: Sat Apr 20, 2024 6:34 am
Antihelion wrote: Sat Apr 20, 2024 12:02 am Then tell me what your 8-game tournament is supposed to measure? CPU noise?
How reliably t he engine performs?
Can you tell me how reliably my coin works when I flip it 8 times?
Friendly reminder that stealing is a crime, is wrong, and makes you a thief.
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )