Cheat testing

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Houdini
Posts: 1471
Joined: Tue Mar 16, 2010 12:00 am

Re: Cheat testing

Post by Houdini »

Mike S. wrote:What means "consistently", exactly? 100% agreement with either the 1st or with the 2nd Houdini choice? If less than 100%, which percentage is relevant?

I am not sure, but I am confident that I could find 10+ Capablanca games where he has chosen either the 1st or the 2nd choice of Houdini, all the time (of course except in the opening). This would be the clear refutation of this method.

Always, all the moves are there to choose from. - It is simply meaningless in terms of evidence.
"Consistently" is the exact opposite of cherry-picking 1, 5 or 10 games to make a point.
Even if you find 10 games in which Capablanca played exactly like Houdini, you will have MANY more games in which he didn't play exactly like Houdini.

Robert
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Cheat testing

Post by mwyoung »

Houdini wrote:
mwyoung wrote:I have quoted what you have said. And how it was used in chessbase. If you want to make in detail of what exactly you meant and revise your statement I am good with that. And to show your method works, and demanstrates how this proves BI is guilty with no further evidence being required. I will be very interested in your proof and results. So far I have seen nothing that shows your statement to be correct. And this should have never been said to say someone is guilty of cheating with no proof or test that it works.
Mark, over a month ago I've explained what I meant with my statement, in a direct response to one of your many posts. See http://www.talkchess.com/forum/viewtopi ... ht=#522545 stressing the statistical nature of the approach I recommend. I've also explained that I did not make any comment about the particular Ivanov case, nor that I support Mr. Lilov's rather sloppy and unscientific approach.

I have no further interest in all this nonsense, so please stop:
- (deliberately?) misrepresenting what I wrote.
- pretending Mr. Lilov followed the approach I recommended.
- using my name in this debate unless you actually make an effort to understand what I have written.

Thank you,

Robert
Robert you were the one that allowed yourself to be dragged into this mess. And it is a mess. I think it is wise to now distance yourself from FM Lilov. Yes he was sloppy, and I'm glad to see you clarify your opinion on BI guilt.

These comments are a bit late in my opinion. But I will not argue about the timing. I will honor your request since you have made a clarification.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Cheat testing

Post by bob »

mwyoung wrote:
Houdini wrote:
mwyoung wrote:Robert Houdart is not a GM player.
Robert Houdart claimed with no testing or proof of any kind.
Robert Houdart: "No human being can consistently play the #1 or #2 choice of Houdini. Detecting this pattern exposes the cheating, no further evidence is required."

and used his statement to support FM Lilov claim without proof or testing his theory that a player was 100% a cheater.
My statement is still 100% accurate: show me a human player who can consistently play the #1 or #2 choice of Houdini, and I will show you a cheater. Note the emphasis on consistently, as I explained to you before this implies statistical methods that rely on large number of games and sufficiently high confidence levels to avoid mistakes.

Contrary to what you suggest there is no such thing as a "Lilov-Houdart" method... you show a complete misunderstanding of my position and/or of what Mr. Lilov has done.
Mr. Lilov is not at all following what I wrote above - he has not demonstrated that Mr. Ivanov consistently plays like Houdini. In fact what you read as my "supporting" Mr. Lilov was in fact a critique: that it is completely pointless to comment individual moves as Mr. Lilov often does in his videos, and that only a statistical approach can provide insight.

Mark, please stop misrepresenting what I wrote about this topic.

Cheers,
Robert
I have quoted what you have said. And how it was used in chessbase. If you want to make in detail of what exactly you meant and revise your statement I am good with that. And to show your method works, and demanstrates how this proves BI is guilty with no further evidence being required. I will be very interested in your proof and results. So far I have seen nothing that shows your statement to be correct. And this should have never been said to say someone is guilty of cheating with no proof or test that it works.
The problem is that this "consistently matches" is too vague for any scientific method to use. The issue is catching a cheater in one tournament, where he may only cheat in a couple of key rounds. He is not going to "consistently match" any program for the tournament, and this type of cheat detection simply can not possibly work.

This is a MUCH harder problem that most appreciate. As I mentioned previously, any method should be applied to at least many hundreds of complete tournaments, preferably no later than the early 90's, so that there is no possibility of of any computer being usable for cheating. I wonder how many false cheater flags would be raised? Without that data, anything else is useless.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Cheat testing

Post by Don »

bob wrote:
mwyoung wrote:
Houdini wrote:
mwyoung wrote:Robert Houdart is not a GM player.
Robert Houdart claimed with no testing or proof of any kind.
Robert Houdart: "No human being can consistently play the #1 or #2 choice of Houdini. Detecting this pattern exposes the cheating, no further evidence is required."

and used his statement to support FM Lilov claim without proof or testing his theory that a player was 100% a cheater.
My statement is still 100% accurate: show me a human player who can consistently play the #1 or #2 choice of Houdini, and I will show you a cheater. Note the emphasis on consistently, as I explained to you before this implies statistical methods that rely on large number of games and sufficiently high confidence levels to avoid mistakes.

Contrary to what you suggest there is no such thing as a "Lilov-Houdart" method... you show a complete misunderstanding of my position and/or of what Mr. Lilov has done.
Mr. Lilov is not at all following what I wrote above - he has not demonstrated that Mr. Ivanov consistently plays like Houdini. In fact what you read as my "supporting" Mr. Lilov was in fact a critique: that it is completely pointless to comment individual moves as Mr. Lilov often does in his videos, and that only a statistical approach can provide insight.

Mark, please stop misrepresenting what I wrote about this topic.

Cheers,
Robert
I have quoted what you have said. And how it was used in chessbase. If you want to make in detail of what exactly you meant and revise your statement I am good with that. And to show your method works, and demanstrates how this proves BI is guilty with no further evidence being required. I will be very interested in your proof and results. So far I have seen nothing that shows your statement to be correct. And this should have never been said to say someone is guilty of cheating with no proof or test that it works.
The problem is that this "consistently matches" is too vague for any scientific method to use. The issue is catching a cheater in one tournament, where he may only cheat in a couple of key rounds. He is not going to "consistently match" any program for the tournament, and this type of cheat detection simply can not possibly work.

This is a MUCH harder problem that most appreciate. As I mentioned previously, any method should be applied to at least many hundreds of complete tournaments, preferably no later than the early 90's, so that there is no possibility of of any computer being usable for cheating. I wonder how many false cheater flags would be raised? Without that data, anything else is useless.
My test can show in a single game such a high percentage of matches that it's clearly suspicious, an event that has a truly small chance of happening. But it requires a game with a lot of matchable moves to be very convincing - and by itself I don't trust a single game, even with a 1/10000 chance. However I think a full tournament of these high match percentages can be pretty damning but as you hinted at you can cheat by just playing a few key moves of some strong program and that would be impossible to catch statistically.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
User avatar
Houdini
Posts: 1471
Joined: Tue Mar 16, 2010 12:00 am

Re: Cheat testing

Post by Houdini »

mwyoung wrote:
Houdini wrote:
mwyoung wrote:I have quoted what you have said. And how it was used in chessbase. If you want to make in detail of what exactly you meant and revise your statement I am good with that. And to show your method works, and demanstrates how this proves BI is guilty with no further evidence being required. I will be very interested in your proof and results. So far I have seen nothing that shows your statement to be correct. And this should have never been said to say someone is guilty of cheating with no proof or test that it works.
Mark, over a month ago I've explained what I meant with my statement, in a direct response to one of your many posts. See http://www.talkchess.com/forum/viewtopi ... ht=#522545 stressing the statistical nature of the approach I recommend. I've also explained that I did not make any comment about the particular Ivanov case, nor that I support Mr. Lilov's rather sloppy and unscientific approach.

I have no further interest in all this nonsense, so please stop:
- (deliberately?) misrepresenting what I wrote.
- pretending Mr. Lilov followed the approach I recommended.
- using my name in this debate unless you actually make an effort to understand what I have written.

Thank you,

Robert
Robert you were the one that allowed yourself to be dragged into this mess. And it is a mess. I think it is wise to now distance yourself from FM Lilov. Yes he was sloppy, and I'm glad to see you clarify your opinion on BI guilt.

These comments are a bit late in my opinion. But I will not argue about the timing. I will honor your request since you have made a clarification.
I'm not sure what you're doing here. First you misrepresent my position, then you attack this misrepresentation you created yourself, then you ask me to distance myself from the BS you created. Are you trolling me?
I'm not going to waste anymore time with this nonsense, bye...
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Cheat testing

Post by mwyoung »

bob wrote:
mwyoung wrote:
Houdini wrote:
mwyoung wrote:Robert Houdart is not a GM player.
Robert Houdart claimed with no testing or proof of any kind.
Robert Houdart: "No human being can consistently play the #1 or #2 choice of Houdini. Detecting this pattern exposes the cheating, no further evidence is required."

and used his statement to support FM Lilov claim without proof or testing his theory that a player was 100% a cheater.
My statement is still 100% accurate: show me a human player who can consistently play the #1 or #2 choice of Houdini, and I will show you a cheater. Note the emphasis on consistently, as I explained to you before this implies statistical methods that rely on large number of games and sufficiently high confidence levels to avoid mistakes.

Contrary to what you suggest there is no such thing as a "Lilov-Houdart" method... you show a complete misunderstanding of my position and/or of what Mr. Lilov has done.
Mr. Lilov is not at all following what I wrote above - he has not demonstrated that Mr. Ivanov consistently plays like Houdini. In fact what you read as my "supporting" Mr. Lilov was in fact a critique: that it is completely pointless to comment individual moves as Mr. Lilov often does in his videos, and that only a statistical approach can provide insight.

Mark, please stop misrepresenting what I wrote about this topic.

Cheers,
Robert
I have quoted what you have said. And how it was used in chessbase. If you want to make in detail of what exactly you meant and revise your statement I am good with that. And to show your method works, and demanstrates how this proves BI is guilty with no further evidence being required. I will be very interested in your proof and results. So far I have seen nothing that shows your statement to be correct. And this should have never been said to say someone is guilty of cheating with no proof or test that it works.
The problem is that this "consistently matches" is too vague for any scientific method to use. The issue is catching a cheater in one tournament, where he may only cheat in a couple of key rounds. He is not going to "consistently match" any program for the tournament, and this type of cheat detection simply can not possibly work.

This is a MUCH harder problem that most appreciate. As I mentioned previously, any method should be applied to at least many hundreds of complete tournaments, preferably no later than the early 90's, so that there is no possibility of of any computer being usable for cheating. I wonder how many false cheater flags would be raised? Without that data, anything else is useless.
I agree Robert,

Chess is already run like a third world country. And this kind of nonsense protocol some want to start using. To start calling other chess players cheaters will only make things worse.

When I saw intelligent people buying into this nonsense that should know better, I could not be silent. And what FM Lilov was doing in my opinion has nothing to do with catching cheaters. It is wrong on too many levels, sloppy and unscientific. It is much worse then that.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Cheat testing

Post by mwyoung »

Houdini wrote:
mwyoung wrote:
Houdini wrote:
mwyoung wrote:I have quoted what you have said. And how it was used in chessbase. If you want to make in detail of what exactly you meant and revise your statement I am good with that. And to show your method works, and demanstrates how this proves BI is guilty with no further evidence being required. I will be very interested in your proof and results. So far I have seen nothing that shows your statement to be correct. And this should have never been said to say someone is guilty of cheating with no proof or test that it works.
Mark, over a month ago I've explained what I meant with my statement, in a direct response to one of your many posts. See http://www.talkchess.com/forum/viewtopi ... ht=#522545 stressing the statistical nature of the approach I recommend. I've also explained that I did not make any comment about the particular Ivanov case, nor that I support Mr. Lilov's rather sloppy and unscientific approach.

I have no further interest in all this nonsense, so please stop:
- (deliberately?) misrepresenting what I wrote.
- pretending Mr. Lilov followed the approach I recommended.
- using my name in this debate unless you actually make an effort to understand what I have written.

Thank you,

Robert
Robert you were the one that allowed yourself to be dragged into this mess. And it is a mess. I think it is wise to now distance yourself from FM Lilov. Yes he was sloppy, and I'm glad to see you clarify your opinion on BI guilt.

These comments are a bit late in my opinion. But I will not argue about the timing. I will honor your request since you have made a clarification.
I'm not sure what you're doing here. First you misrepresent my position, then you attack this misrepresentation you created yourself, then you ask me to distance myself from the BS you created. Are you trolling me?
I'm not going to waste anymore time with this nonsense, bye...

A misrepresentation I created....Really.

http://www.chessbase.com/Home/TabId/211 ... 60613.aspx

Robert Houdart, the author of the chess engine Houdini, also expressed his opinion on the subject of chess cheating where ‘Houdini 3.0’ was involved. On YouTube, under the latest video analysis of Ivanov’s games by FM Valeri Lilov, Houdart states:

Robert Houdart: No human being can consistently play the #1 or #2 choice of Houdini. Detecting this pattern exposes the cheating, no further evidence is required. It's not very useful to focus on individual move choices that appear unlikely (like Qd1 in the blitz game), it's the combined evidence of ALL the moves that is compelling.


I believe you injected yourself into this discussion. When talking to FM Lilov and giving him advice on his videos. Did I miss the part were you told FM Lilov when giving him advice.

And I will quote you so not to misrepresent you. Robert Houdart "nor that I support Mr. Lilov's rather sloppy and unscientific approach."

Looks like to me you know exactly what FM Lilov was doing and his methods, and even gave FM Lilov friendly advice against BI.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
User avatar
Houdini
Posts: 1471
Joined: Tue Mar 16, 2010 12:00 am

Re: Cheat testing

Post by Houdini »

mwyoung wrote:A misrepresentation I created....Really.

http://www.chessbase.com/Home/TabId/211 ... 60613.aspx

Robert Houdart, the author of the chess engine Houdini, also expressed his opinion on the subject of chess cheating where ‘Houdini 3.0’ was involved. On YouTube, under the latest video analysis of Ivanov’s games by FM Valeri Lilov, Houdart states:

Robert Houdart: No human being can consistently play the #1 or #2 choice of Houdini. Detecting this pattern exposes the cheating, no further evidence is required. It's not very useful to focus on individual move choices that appear unlikely (like Qd1 in the blitz game), it's the combined evidence of ALL the moves that is compelling.


I believe you injected yourself into this discussion. When talking to FM Lilov and giving him advice on his videos. Did I miss the part were you told FM Lilov when giving him advice.

And I will quote you so not to misrepresent you. Robert Houdart "nor that I support Mr. Lilov's rather sloppy and unscientific approach."

Looks like to me you know exactly what FM Lilov was doing and his methods, and even gave FM Lilov friendly advice against BI.
I don't know Mr. Lilov, have never spoken to him and don't give him advice. I've just posted a very simple Youtube comment after watching a video, it's mostly a critic of Lilov's approach.

ChessBase simply quoted what I wrote, which is very much OK.
A lot smarter than what you did... you completely misread my comment and created your "Lilov-Houdart method" fantasy. Even now you continue to add to this fantasy by suggesting that I was "talking to FM Lilov and giving him advice on his videos".

Looking for windmills, Don Quichote?
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Cheat testing

Post by mwyoung »

Houdini wrote:
mwyoung wrote:A misrepresentation I created....Really.

http://www.chessbase.com/Home/TabId/211 ... 60613.aspx

Robert Houdart, the author of the chess engine Houdini, also expressed his opinion on the subject of chess cheating where ‘Houdini 3.0’ was involved. On YouTube, under the latest video analysis of Ivanov’s games by FM Valeri Lilov, Houdart states:

Robert Houdart: No human being can consistently play the #1 or #2 choice of Houdini. Detecting this pattern exposes the cheating, no further evidence is required. It's not very useful to focus on individual move choices that appear unlikely (like Qd1 in the blitz game), it's the combined evidence of ALL the moves that is compelling.


I believe you injected yourself into this discussion. When talking to FM Lilov and giving him advice on his videos. Did I miss the part were you told FM Lilov when giving him advice.

And I will quote you so not to misrepresent you. Robert Houdart "nor that I support Mr. Lilov's rather sloppy and unscientific approach."

Looks like to me you know exactly what FM Lilov was doing and his methods, and even gave FM Lilov friendly advice against BI.
I don't know Mr. Lilov, have never spoken to him and don't give him advice. I've just posted a very simple Youtube comment after watching a video, it's mostly a critic of Lilov's approach.

ChessBase simply quoted what I wrote, which is very much OK.
A lot smarter than what you did... you completely misread my comment and created your "Lilov-Houdart method" fantasy. Even now you continue to add to this fantasy by suggesting that I was "talking to FM Lilov and giving him advice on his videos".

Looking for windmills, Don Quichote?
No windmills Robert, only stopping you from making sh** up, saying I am the one who put you into this mess. That is the thing about the internet, your words stay there forever Robert.

Robert your comment is directed to FM Lilov, it is your opinion, and comment, and advice on FM Lilov's video, to FM Lilov, on FM Lilov's You Tube account. You were commenting on FM Lilov's methods, not your own methods. That is the context of your comment. So please don't BS us Robert.

Robert Houdart: No human being can consistently play the #1 or #2 choice of Houdini. Detecting this pattern exposes the cheating, no further evidence is required. It's not very useful to focus on individual move choices that appear unlikely (like Qd1 in the blitz game), it's the combined evidence of ALL the moves that is compelling.


If you really felt FM Lilov's You Tube video against BI was Sloppy and Unscientific. Why did you not say this to FM Lilov on his You Tube account.

Have a nice day Robert....
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.