Carlsen withdrawal after loss to Niemann

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
AdminX
Posts: 6363
Joined: Mon Mar 13, 2006 2:34 pm
Location: Acworth, GA

Re: Carlsen withdrawal after loss to Niemann

Post by AdminX »

Stokfish on the subject! Enjoy.

"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers
Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Carlsen withdrawal after loss to Niemann

Post by Chessqueen »

AdminX wrote: Sun Sep 11, 2022 10:07 pm Stokfish on the subject! Enjoy.

So if GM Beat Neo, he is cheating again ?
CornfedForever
Posts: 648
Joined: Mon Jun 20, 2022 4:08 am
Full name: Brian D. Smith

Re: Carlsen withdrawal after loss to Niemann

Post by CornfedForever »

Chessqueen wrote: Sun Sep 11, 2022 10:37 pm
AdminX wrote: Sun Sep 11, 2022 10:07 pm Stokfish on the subject! Enjoy.

So if GM Beat Neo, he is cheating again ?
If I may butt in...given his pawn structure, yes, maybe I would join the 'cheating chorus' 8-)
Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Carlsen withdrawal after loss to Niemann

Post by Chessqueen »

CornfedForever wrote: Sun Sep 11, 2022 11:05 pm
Chessqueen wrote: Sun Sep 11, 2022 10:37 pm
AdminX wrote: Sun Sep 11, 2022 10:07 pm Stokfish on the subject! Enjoy.

So if GM Beat Neo, he is cheating again ?
If I may butt in...given his pawn structure, yes, maybe I would join the 'cheating chorus' 8-)
According to Stockfish it was a draw 5 moves before they agreed to draw :roll:
CornfedForever
Posts: 648
Joined: Mon Jun 20, 2022 4:08 am
Full name: Brian D. Smith

Re: Carlsen withdrawal after loss to Niemann

Post by CornfedForever »

Chessqueen wrote: Sun Sep 11, 2022 11:24 pm
CornfedForever wrote: Sun Sep 11, 2022 11:05 pm
Chessqueen wrote: Sun Sep 11, 2022 10:37 pm
AdminX wrote: Sun Sep 11, 2022 10:07 pm Stokfish on the subject! Enjoy.

So if GM Beat Neo, he is cheating again ?
If I may butt in...given his pawn structure, yes, maybe I would join the 'cheating chorus' 8-)
According to Stockfish it was a draw 5 moves before they agreed to draw :roll:
Well....most of that was the 'repetition'. Strange how Neop did not take the 'h' pawn...not sure what Fish would have said about that. Nepo simply said he blundered, thinking anything would win.
chrisw
Posts: 4624
Joined: Tue Apr 03, 2012 4:28 pm
Location: Midi-Pyrénées
Full name: Christopher Whittington

Re: Carlsen withdrawal after loss to Niemann

Post by chrisw »

DrCliche wrote: Sun Sep 11, 2022 6:07 pm
chrisw wrote: Sun Sep 11, 2022 4:53 pmTake a look back at the original tweet thread and you’ll find that the OP didn’t think to generate a control group and, incredibly, declines to provide one when challenged.
The OP is obviously not fit in a scientific/educated/skilled sense to be taken seriously, there’s not much point in wasting time challenging his data collection and selection exercise for cherry picking and manipulation until it could be wrapped into some confirmation bias conclusion.
Moral of the story - never accept data tables and conclusions at face value, always check first before repeating.
You don't know what a control group is. You mean to say that the original tweeter didn't control for confounding variables. Of course, outside of obscure techniques that you've never heard of (like synthetic control methods), controls aren't "generated". And it's hardly relevant that the original tweeter didn't control for confounding variables, because I did.

Lame and missing the point. To demonstrate Hans Nieman’s data line is somehow special, it is necessary to compare it with data lines of all players (or a fairly selected subset of comparable players), otherwise known as a Control Group. Chess engine testers/rating list compilers will confirm that Elo is highly volatile, especially when based on small game samples (substitute humans for chess engines and you add a whole new layer of volatility). Delta-Elo (which the OP is using) even more so. Only a control group can show what is normal and what is an outlier. Without it no conclusions can even begin to be drawn. Basic statistics.
Take the OP data and table back to the OP with a big red line through it and a mark of 0/10. Wrong.



(That is, I "checked first before repeating", which you would know if you'd bothered to follow your own advice in this thread. Alas, you appear to glide through life blissfully unburdened by a sense of irony.)

I gathered extra data in order to account for what I thought were the three most likely confounders, namely age, fatigue, and opponent strength. None of them were anywhere close to being statistically significant in my analysis. Andrew Grant suggested another (absurdly unlikely) confounder, that Niemann's strength increases (very) disproportionately compared to that of his opponents when players know their pairings ahead of time, and that advance knowledge of pairings is strongly and positively correlated with a tournament's broadcast status. But as of yet he's offered no plausible mechanism for such an effect, much less demonstrated that any data exists to support it.

If the broadcast data is accurate (still a big if, but I've seen no indications otherwise), the extent to which the original tweeter could have meaningfully cherry picked the data is very limited. The only thing he could have done is choose a particular interval of Niemann's classical games. But that's hardly relevant, as it would obviously be concerning if any player ever had a two year span in which their classical tournament performances were so strongly correlated with broadcast status, and to such a crazy degree of statistical significance.

Given a p-value of 0.0008, such a phenomenon is quite unlikely to occur due to random chance, even if you have the option of picking the most damning two year window in a given player's career. And if Niemann has other spans that don't follow the same pattern, it doesn't mean he never cheated, it just means he probably wasn't aggressively cheating then.
Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Carlsen withdrawal after loss to Niemann

Post by Chessqueen »

CornfedForever wrote: Sun Sep 11, 2022 11:48 pm
Chessqueen wrote: Sun Sep 11, 2022 11:24 pm
CornfedForever wrote: Sun Sep 11, 2022 11:05 pm
Chessqueen wrote: Sun Sep 11, 2022 10:37 pm
AdminX wrote: Sun Sep 11, 2022 10:07 pm Stokfish on the subject! Enjoy.

So if GM Beat Neo, he is cheating again ?
If I may butt in...given his pawn structure, yes, maybe I would join the 'cheating chorus' 8-)
According to Stockfish it was a draw 5 moves before they agreed to draw :roll:
Well....most of that was the 'repetition'. Strange how Neop did not take the 'h' pawn...not sure what Fish would have said about that. Nepo simply said he blundered, thinking anything would win.
What else do you expect from GM Hans, he beat the current World Champion and Drew against the Challenger of the Next World Chess Championship GM Neo :roll:
CornfedForever
Posts: 648
Joined: Mon Jun 20, 2022 4:08 am
Full name: Brian D. Smith

Re: Carlsen withdrawal after loss to Niemann

Post by CornfedForever »

Chessqueen wrote: Mon Sep 12, 2022 4:37 am

According to Stockfish it was a draw 5 moves before they agreed to draw :roll:
Well....most of that was the 'repetition'. Strange how Neop did not take the 'h' pawn...not sure what Fish would have said about that. Nepo simply said he blundered, thinking anything would win.
[/quote]

What else do you expect from GM Hans, he beat the current World Champion and Drew against the Challenger of the Next World Chess Championship GM Neo :roll:
[/quote]

I....seriously do not know what you mean by that. Perhaps you have just been up too late...happens to me.
DrCliche
Posts: 65
Joined: Sun Aug 19, 2018 10:57 pm
Full name: Nickolas Reynolds

Re: Carlsen withdrawal after loss to Niemann

Post by DrCliche »

chrisw wrote: Mon Sep 12, 2022 1:24 amLame and missing the point. To demonstrate Hans Nieman’s data line is somehow special, it is necessary to compare it with data lines of all players (or a fairly selected subset of comparable players), otherwise known as a Control Group. Chess engine testers/rating list compilers will confirm that Elo is highly volatile, especially when based on small game samples (substitute humans for chess engines and you add a whole new layer of volatility). Delta-Elo (which the OP is using) even more so. Only a control group can show what is normal and what is an outlier. Without it no conclusions can even begin to be drawn. Basic statistics.
Take the OP data and table back to the OP with a big red line through it and a mark of 0/10. Wrong.
Once again the poster boy for Dunning-Kruger leaps at the chance to show he remains free of the trammels of knowledge, experience, and the capacity for learning and self-reflection!

You don't understand what a control group is. You apparently don't understand much of anything when it comes to the "basic statistics" you hold so dear. Observational studies don't have control groups. Instead, they use various methods to control for confounding variables.

Your incoherent criticism seems to be clumsily circling around the idea that you believe it would be useful to run a regression on data from more players, with added terms interacting each independent variable with another variable indicating whether a given player is Hans Niemann.

That's a reasonable thing to do, and you're welcome to do that here if you like. (Though at this point it's clear you wouldn't know how.) But I don't believe it's necessary in this instance, because:
  • As I mentioned in my original analysis, and then again in later posts—you really must learn to read!—I already did this, albeit informally. I performed a cursory examination of a number of demographically similar players from the tournaments in the Niemann dataset to see if any of them displayed remotely similar patterns of performance. I found no one that did.
  • AndrewGrant's devil's advocacy notwithstanding, it appears that nobody realistically believes there's a mechanism (other than cheating) that could explain a correlation between broadcast status and performance. In other words, there appears to be general consensus that the prior probability of a non-cheating mechanism is quite low. So we ignore it. This same reasoning is why, for example, I didn't entertain the idea that Niemann can read minds on Sundays.
  • In effect, we ignore "possibilities" with evidently low prior probabilities, and are justified in doing so. This is common and reasonable in exploratory studies. You'll rarely be wrong to assume something is true when everyone believes it to be true, and nobody can even hypothesize a plausible alternate mechanism.
  • By far the biggest question mark is the actual broadcast status of the tournaments in the Niemann dataset. The methodology of the analysis is completely standard, and sufficiently controlled for its purposes.
If Carlsen or Chess.com were to release a report from a team of investigators, I would demand that they be much more thorough than my exploratory observational study. I would want them to contact every tournament director, confirm broadcast status and manner of broadcast, gather data on broadcast delays and glitches (intentional or not), perform statistical analysis on the level of individual games, show distributions of various statistics over different populations of players by demographic, etc. In other words, I would want them to dot all the i's and cross all the t's. Only then would I be comfortable acting on the conclusions of the report, and allowing the chess community to formally sanction Niemann for cheating.

But for the purposes of figuring out whether it looks like there might be something here? To see if there's any evidence that Niemann might be or have been a cheater? That question has been answered definitively, from multiple independent angles. Yeah, there's evidence. There's a lot of it. It's time for people with resources to do a real investigation.
User avatar
M ANSARI
Posts: 3719
Joined: Thu Mar 16, 2006 7:10 pm

Re: Carlsen withdrawal after loss to Niemann

Post by M ANSARI »

I think pretty much everyone now agrees that Hans was a prolific serial cheater online, and that he also cheated OTB in at least one tournament. This is enough evidence to require further investigation of ALL his previous online and OTB tournaments. The data of those tournaments is out there but there needs to be additional data added. That additional data would be what engines were being used at that time, what were the requirements to reach a certain norm that would motivate the player to take more risks to win a game to reach a certain norm. What was the anti cheating method used at each venue? How was the progression of the player in ELO strength compared to other players that had the same meteoric rise in ELO. I think once you put all the data and compare player A with player B etc... some pattern will begin to show. Who knows, maybe we will find out there are other serial cheaters out there.