Cheating detection algorithms - hints and ideas

[socoban]

You could sample games, and subject them to a blunder check by a strong engine with 1 sec/move. People that blunder most likely have not been using an engine, so you have filtered out those. (If needed, you could analyze the position just before and after their 'blunder candidate' at a larger depth, to make sure it is truly a blunder, rather than misevaluation by the engine because of the limited time.)

The games that pass the blunder test can be analyzed with somewhat larger time per move (say 10 sec), and remain suspect when there are almost no moves where the evaluation drops much. Moves that do decrease the evaluation can be analyzed with longer time, by a handfull of engines, to see if any of them suggests the move that was played. Games without such moves, or where such moves are all suggested by the same engine, remain suspect.

If you have reduced the number of suspect games enough, you can probably afford to analyze each position with the most-likely used engine at the actual time, to generate hard prove. The trick is to reduce the number of candidates first by much faster tests.

People that cheat, are likely to cheat consistently. If one of their games is sampled, and found suspect, you can subject all their earlier games that were not sampled to the test procedure.

In all cases it will remain very difficult to catch people that cheat with weak engines, such as (say) Fairy-Max. But thenature of cheaters is that when they cheat, they are likely to cheat with the strongest engine they can get their hands on. And would you really care when they do? If it plays like an average player on your server, who cares if it actually was a computer?

Thanks for ideas. Your algorithm seems fine to me. The pre-selection is the must. There are hundreds of games played at same time and I hope it will grow to thousands. I can't check them all. There will be needed multi level testing. So this is general idea. What I need to find out is which positions should be tested and which not. I don't want to create own representation of board and analyse position myself. Algorithm should be able to decide whether the position should be part of engine analysis or not based on UCI analyse results.

By that I mean,

1 step... fast check of all moves for big blunders.
2 step... if there are no blunders, than Which positions should be analysed deeper? Opening part is in my opinion lost time, no engine will give me good information about it. Cheater can or could not use Opening Database. Therefore if he used it, engines can mark some moves as small blunders. If he did not use it, and engine says it was small blunder, than It is confusing too.
If it is endgame, engines do blunders. So there is similar problem. I can't decide if user is using table bases, or not.

I need to select most important positions and conciser only them. Test of 1 position from difficult middle game can give me better results than testing 10 opening positions. I need to do such position selection based on UCI results. General setup like skip first 15 positions and consider only 15 - 40 moves seems to me very optimistic. I am sure that engine knows that the position is specific. Like if it is silent or there are lot changes. If it has to compute deep or it is considering only positional aspects. If it is using silent move, etc...

So are there any ideas how to sort positions by their importance using UCI results?

[Michel]

it can be solved be analyzisng every position by multiple engines and checking if it is best move in some of them

Well if a non cheating human is a good player it can be expected that his move choice will match the move choice of some engine (but not always the same engine). So that wouldn't tell you anything.

[socoban]

it can be solved be analyzisng every position by multiple engines and checking if it is best move in some of them

Well if a non cheating human is a good player it can be expected that his move choice will match the move choice of some engine (but not always the same engine). So that wouldn't tell you anything.

You would be surprised, but even very strong GMs play big blunders. If the cheater never do big blunders in multiple games than this can work. But yes, I agree, that very advanced skilled cheaters who are strong players themselves are almost impossible to be detected. But than those players will have big rating, will act like humans, so other people will not complain. And than we can analyze them by human, and trust me, human can detect cheaters easily. And if we ban them after they played like 50 games, it will be OK, because users will not think that we detected them too late. And if we ban strong player and he can prove he is IM or GM, we can give him his account back. The goal is to make cheating as hard as it can be, and at least kick of all non programming cheaters.

[phhnguyen]

There is no other way, we have to detect and ban users. Every server does it. The reason is simple, let say 90% never try to cheat, 9% will try it, but if we detect them, they stop. 1% will try it always, but, are all of them smart enougt to cheat undetected? The more complicated cheating is, the less people will do it. And yes, there will be people who will claim they are not cheating, doesn't matter they really cheat or not. Impotant is to create trust in our server, that we are in open war with all kind of cheating. It is more marketing problem, than technical problem. With good maketing you can create trust even if you detect few of them manually. We already anounce to everyone that we banned some account and why. It makes them feel saver and prevent many cheaters to try it.

You are so lucky with chess!

For Xiangqi (Chinese chess), we guess at least 90% top players on chess servers are cheaters.

I think we can reduce or make "newbie" cheaters be harder (for example, not allow auto playing, not allow players move cursor out of chess board to control other programs...) but "prof" cheaters always find ways to cheat again.

Specially, we see there is no way to against cheaters who use chess software for suggesting (sometimes he can make his own moves) only, or who combine some chess software (for using in different periods or for getting "average" moves). If you detect them by using move time or blur moves, you should give up because they make move themselves (but still copy from software - the move times are very reasonable); sometimes they may make "silly" moves because those are their own moves or simply they click/drag pieces wrongly.

Perhaps, leave with cheaters is better solution.

[socoban]

There is no other way, we have to detect and ban users. Every server does it. The reason is simple, let say 90% never try to cheat, 9% will try it, but if we detect them, they stop. 1% will try it always, but, are all of them smart enougt to cheat undetected? The more complicated cheating is, the less people will do it. And yes, there will be people who will claim they are not cheating, doesn't matter they really cheat or not. Impotant is to create trust in our server, that we are in open war with all kind of cheating. It is more marketing problem, than technical problem. With good maketing you can create trust even if you detect few of them manually. We already anounce to everyone that we banned some account and why. It makes them feel saver and prevent many cheaters to try it.

You are so lucky with chess!

For Xiangqi (Chinese chess), we guess at least 90% top players on chess servers are cheaters.

I think we can reduce or make "newbie" cheaters be harder (for example, not allow auto playing, not allow players move cursor out of chess board to control other programs...) but "prof" cheaters always find ways to cheat again.

Specially, we see there is no way to against cheaters who use chess software for suggesting (sometimes he can make his own moves) only, or who combine some chess software (for using in different periods or for getting "average" moves). If you detect them by using move time or blur moves, you should give up because they make move themselves (but still copy from software - the move times are very reasonable); sometimes they can make "silly" moves because that is their own moves or simply they click/drag pieces wrongly.

Perhaps, leave with cheaters is better solution.

I don't know what is the situation in China. But what we find out is, that cheaters are from countries, where is lack of democracy. We had very few cheaters from USA, Britain, Switzerland, Sweeden, etc.. Most of cheaters on our server is from Romania, Russia, Hungary. So in my oppinion cheaters are from countries, where chess is very popular, and people are used to act unfair in real live. They want to gain credit and act important in their poor lifes. I think this is same in China. Btw, we planed to enter Chines market, but we find out that hudge % of users is using Internet Explorer 6. It is extremly buggy, slow browser, which we simply can't support. So we desite not to enter the market.

And yes, I must agree, that users who are only using engines for suggestions are almost impossibe to be tracked. But those players has to be strong players themselve. So you can ask them to prove their personality according to FIDE rating, if they don't agree, you can ban them. If they care about server, they will prove it. If they don't care, they are almost always cheaters.

[phhnguyen]

I don't know what is the situation in China. But what we find out is, that cheaters are from countries, where is lack of democracy. We had very few cheaters from USA, Britain, Switzerland, Sweeden, etc.. Most of cheaters on our server is from Romania, Russia, Hungary. So in my oppinion cheaters are from countries, where chess is very popular, and people are used to act unfair in real live. They want to gain credit and act important in their poor lifes. I think this is same in China. Btw, we planed to enter Chines market, but we find out that hudge % of users is using Internet Explorer 6. It is extremly buggy, slow browser, which we simply can't support. So we desite not to enter the market.

Please don't add politic here. I am not Chinese nor leave in China.

As my observation, the situation for Xiangqi is the same as over 10 years ago, when Chinese from main land ware still rare to access Xiangqi servers which are mainly in US.

For chess I guess fewer cheaters from US, Britain... can be explained easily without democracy reason but other ones such as players started playing on Internet when software ware too weak for using to cheat, thus it formed a big group of players who have never cheated. However, younger players from anywhere seem to be easier to cheat: You see there are larger numbers of cheaters from other countries because their larger numbers are younger.

And yes, I must agree, that users who are only using engines for suggestions are almost impossibe to be tracked. But those players has to be strong players themselve. So you can ask them to prove their personality according to FIDE rating, if they don't agree, you can ban them. If they care about server, they will prove it. If they don't care, they are almost always cheaters.

I think you may face some troubles:
- Very hard to do that when number of players is big
- A lot of players just play for fun. They want to play as anonymous and don't want to give you any piece of their real information. Do you want your chess server becomes a "official" FIDE chess server or an enjoyable place?
- A "prof" cheater can do almost everything. He can simply give you a rating of someone. How can you verify that?

I have known some forums which cheaters can discuss everything, from how to use software to technique to cheat, or avoid detecting...

I have been developing my own chess server and very interested on that issue.

[socoban]

Please don't add politic here. I am not Chinese nor leave in China.

As my observation, the situation for Xiangqi is the same as over 10 years ago, when Chinese from main land ware still rare to access Xiangqi servers which are mainly in US.

Ok, it is not about politics. It is about society. But maybe there is simpler reason. We have huge amount of amateurs from those cheating countries, while from others we are focused on advanced chess players.

I do know nothing about Xiangqi , so I can't imagine how hard is to detect them.

I think you may face some troubles:
- Very hard to do that when number of players is big
- A lot of players just play for fun. They want to play as anonymous and don't want to give you any piece of their real information. Do you want your chess server becomes a "official" FIDE chess server or an enjoyable place?
- A "prof" cheater can do almost everything. He can simply give you a rating of someone. How can you verify that?

I have known some forums which cheaters can discuss everything, from how to use software to technique to cheat, or avoid detecting...

I have been developing my own chess server and very interested on that issue.

I am not sure about how strong player you are. But trust me, I myself can detect strong cheater who is seldom using engine. In chess there are sometimes such hard positions, that even very strong GM can't evaluate them correctly. And important is, that GM's will never play longer games. They always play blitz or bullet, so they make blunders. In 10 games of any GM, I can find positions, which will tell me if he is cheating or not. And the idea is not to ask all strong players to prove their origin, but only suspected. And if user accounts are paid, and GM/ IM can have it for free, than he will prove his origin to save money.

[phhnguyen]

I am not sure about how strong player you are. But trust me, I myself can detect strong cheater who is seldom using engine. In chess there are sometimes such hard positions, that even very strong GM can't evaluate them correctly. And important is, that GM's will never play longer games. They always play blitz or bullet, so they make blunders. In 10 games of any GM, I can find positions, which will tell me if he is cheating or not. And the idea is not to ask all strong players to prove their origin, but only suspected. And if user accounts are paid, and GM/ IM can have it for free, than he will prove his origin to save money.

I see that you (like me) can guess but can not confirm 100% nor have "enough" evidence that someone is a cheater. How can you ban him by feeling or guessing only?

As I have said above, I can guess and can confirm myself (100%) about cheaters (sometimes confirmations are not from chess but from their discussions). However, it is very hard to have enough evidence to do any action. And once we publish about any form of evidences, other cheaters can easily avoid that form later on.

From my experience, cheaters are usually ready to confront or be angry to anyone who wants to against them.

I don't think free/fee accounts can say who is cheating. Suppose there are some un-proved GMs playing on your server. You may doubt but can you ban them all because of that?

Perhaps, should you do a public notice such as "if you are a fee player, you can not exceed 2000 elo or you are a cheater"

[socoban]

I am not sure about how strong player you are. But trust me, I myself can detect strong cheater who is seldom using engine. In chess there are sometimes such hard positions, that even very strong GM can't evaluate them correctly. And important is, that GM's will never play longer games. They always play blitz or bullet, so they make blunders. In 10 games of any GM, I can find positions, which will tell me if he is cheating or not. And the idea is not to ask all strong players to prove their origin, but only suspected. And if user accounts are paid, and GM/ IM can have it for free, than he will prove his origin to save money.

I see that you (like me) can guess but can not confirm 100% nor have "enough" evidence that someone is a cheater. How can you ban him by feeling or guessing only?

As I have said above, I can guess and can confirm myself (100%) about cheaters (sometimes confirmations are not from chess but from their discussions). However, it is very hard to have enough evidence to do any action. And once we publish about any form of evidences, other cheaters can easily avoid that form later on.

From my experience, cheaters are usually ready to confront or be angry to anyone who wants to against them.

I don't think free/fee accounts can say who is cheating. Suppose there are some un-proved GMs playing on your server. You may doubt but can you ban them all because of that?

Perhaps, should you do a public notice such as "if you are a fee player, you can not exceed 2000 elo or you are a cheater"

I think you mixed 2 approaches. Democracy with human rights and the chess. In real life, you are not guilty of any crime, until court says something else. In chess, even on official FIDE tournaments you can't have mobile phone, go often to toilet, talk to other players, etc. You have only those right which are allowed by tournament rules. If you don't agree to those rules, you can simply go away. Same is with online chess. You can have only those right, that are given you by server. If server decide that you are cheating, you can prove that you are fair player or simple leave server. Those are the rules. Take it or go away. On Fide tournament, no one is checking if you received sms message with cheat move, you are banned because you have mobile turned on. So don't be crazy, the more crime is around, the less right you have. Every policy, every law which restricts crime, restricts also human rights.

So yes, I can ban GM who is doing no mistakes and rejects to prove his identity. In fact such GM is making bad blood, since others will suspect him as cheater too. And it will be better to have server without such players.

[jdart]

I don't think an automated method can replace human oversight. But the bottom line is, most reasonably strong computers now can't be beaten consistently even by GMs, at least at blitz. The main reason IMO is that human players overlook things - many games even at high level are decided by relatively simple blunders. So if a human player, especially not a titled one, is regularly scoring over 50% against computer opposition, that itself is a telltale sign.

In addition weak players do not know openings very well and are often busted in the opening. So a weak account that starts playing 20 moves straight from the latest theory is also suspect.

Accounts that rack up a lot of rating points rapidly against computers and low-rated accounts that magically play book moves are things you should flag for scrutiny at least.

I don't want to punish people for success, but basically, regular success in computer blitz games is only enjoyed by computers, nowadays.