False Positives on Chess Cheating

[CRoberson]

Today I played a game against NoonianChess. Afterwards, I used Komodo to review the game. At first, I was surprised at how many of the early moves that I made were predicted by Komodo. Then I realized the reasons.

I've been playing this particular opening (as white) exclusively for 2 years and played over 100 online games and OTB games with it. After each game, I use a computer to analyze the games and have learned from the computer over time. Yes, my moves in certain positions will be computer like.

In the late 80's, I memorized the Morphy - Consultation team game. About a year later, an opponent played the black side of it. So, I just copied Morphy's moves from my memory. It is not unusual for Chess players to learn from Masters and GM's, then just repeat the game.

Now, we can learn from computers, copy their moves from memory and be accused of cheating.

Chess.com's auto-cheat system notified me of one of my games being suspicious last summer. They didn't accuse me; it was clearly not a computer playing, but my game did match a high percentage of computer moves. The main reason was due to the opening and the endgame. The endgame was super easy, so computers matched it. The opening went as I explained above. The middle game had some mistakes on my part that made it obviously not computer play.

My point is that it is obvious to me that one game may be insufficient to prove cheating.

[M ANSARI]

Nothing is 100% and setting up an algo to catch cheaters online is one of these things ... but it still probably catches 99% of people who are cheating. By that I don't mean it catches 99% of people who are cheating but rather that the 99% of the people caught are cheating. A lot of people can circumvent the algo pretty easily by knowing "how" to cheat. If you are not cheating, I would not worry about it as this will quickly be seen as you play more and more games. If you are playing only a few opening that are computer moves that go deep, the algo will see that and will not flag you. Depending on the site, the cheating algo can be actually very sophisticated. It takes into account tons and tons of factors, and the algo is changed once someone has been flagged as suspicious. I think the algo takes into account what other programs are running on your system, whether you keep switching windows or tabbing during the game, how fast you are making your moves and relate that to the specific part of the game (makes no sense when an obvious only forced capture is made after 10 sec thought). But really what catches most cheaters is tactically it can be very obvious. Computers (even the strongest engines) play a rather different game than humans, you can even say they speak a different language. Engines play dramatically better than humans tactically (by a HUGE margin) and the errors they make are usually positional. A human is different, he is very weak tactically but rather strong when it comes to positional maneuvering and placing pieces. So when a 2000 rated player plays like a 3400 engine tactically consistently, then it is obvious.

But like most things, humans can circumvent any algo if they know what the algo is looking for and if they have the right tools. I have heard of software that is designed for cheating where it gives you a window with several moves and evaluation on them without having that information appear on the transmission. I can imagine that with something like that a human can crush Carlsen or Kramnik consistently and not be flagged by the algo by simply not playing the best moves and just playing natural human moves and waiting for a tactical mistake. I believe that even the top 10 chess players in the world play at a level of about 2500 ELO tactically, but their positional ability is probably around 3000 ELO. So even a 2000 ELO player that has the tactical optical seeing glasses to be able to raise his tactical ability to 3400 ELO will absolutely crush any human. All it would take is one tactical oversight and the game is over, and most humans do several tactical oversights in a game.

Here is a game that Garry Kasparov lost vs. Chess Genius in 1994. I don't even want to speculate how strong that engine was at that time with the hardware used, but my guess it would score 0 points out of a 100 games against an 8 core SF DD. Yet it was able to defeat Kasparov in his prime with black.

[pgn]1. c4 c6 2. d4 d5 3. Nf3 Nf6 4. Qc2 dxc4 5. Qxc4 Bf5 6. Nc3 Nbd7 7. g3 e6 8. Bg2 Be7 9. O-O O-O 10. e3 Ne4 11. Qe2 Qb6 12. Rd1 Rad8 13. Ne1 Ndf6 14. Nxe4 Nxe4 15. f3 Nd6 16. a4 Qb3 17. e4 Bg6 18. Rd3 Qb4 19. b3 Nc8 20. Nc2 Qb6 21. Bf4 c5 22. Be3 cxd4 23. Nxd4 Bc5 24. Rad1 e5 25. Nc2 Rxd3 26. Qxd3 Ne7 27. b4 Bxe3 28. Qxe3 Rd8 29. Rxd8 Qxd8 30. Bf1 b6 31. Qc3 f6 32. Bc4 Bf7 33. Ne3 Qd4 34. Bxf7 Kxf7 35. Qb3 Kf8 36. Kg2 Qd2 37. Kh3 Qe2 38. Ng2 h5 39. Qe3 Qc4 40. Qd2 Qe6 41. g4 hxg4 42. fxg4 Qc4 43. Qe1 Qb3 44. Ne3 Qd3 45. Kg3 Qxe4 46. Qd2 Qf4 47. Kg2 Qd4 48. Qxd4 exd4 49. Nc4 Nc6 50. b5 Ne5 51. Nd6 d3 52. Kf2 Nxg4 53. Ke1 Nxh2 54. Kd2 Nf3 55. Kxd3 Ke7 56. Nf5 Kf7 57. Ke4 Nd2 58. Kd5 g5 59. Nd6 Kg6 60. Kd4 Nb3[/pgn]


Humans have the ability of using a chess engine to make their "human" games reach masterful heights. I don't mean idiots like Ivanov that are obviously cheating, but imagine a 2600-2700 ELO player playing against any human while realizing he has to keep his game as human as possible and not play engine moves by rote. This is something that will really become a problem soon and I feel that if measures to prevent cheating are not implemented that soon we will have an Armstrong type scandal with the world chess championship.

[Uri Blass]

I do not agree that even a 2000 ELO player that has the tactical optical seeing glasses to be able to raise his tactical ability to 3400 ELO will absolutely crush any human.

Note that I do not know how you measure tactical ability but you basically need a good position to cause the top players to make tactical mistakes.

There are games when I made no tactical mistake because my opponent did not play well and there are also draws that I had with no tactical mistake of one of the players.

I have fide rating of 2004 and
If I can use a computer only to tell me the moves that lose at least 0.5 pawn in every move without suggesting me what to play then I am sure that I have no chance against the top human players.

[M ANSARI]

I do not agree that even a 2000 ELO player that has the tactical optical seeing glasses to be able to raise his tactical ability to 3400 ELO will absolutely crush any human.

Note that I do not know how you measure tactical ability but you basically need a good position to cause the top players to make tactical mistakes.

There are games when I made no tactical mistake because my opponent did not play well and there are also draws that I had with no tactical mistake of one of the players.

I have fide rating of 2004 and
If I can use a computer only to tell me the moves that lose at least 0.5 pawn in every move without suggesting me what to play then I am sure that I have no chance against the top human players.

A human player that is 2000 ELO and has computer assistance can crush any human (including Carlsen, Kramnik, Kasparov, Anand) if he has the "tactical glasses" of a 3400 ELO engine that is giving him say 5 different variations and he gets to choose what he thinks (according to his 2000 ELO ability) the best line the engine recommends. This would be so much of a "no contest" that it does not even need to be tested. Someone posted this link from GM Nakamura ... arguably one of the strongest tactical players ever!

http://www.youtube.com/watch?feature=pl ... hd7I#t=235

[Uri Blass]

I do not agree that even a 2000 ELO player that has the tactical optical seeing glasses to be able to raise his tactical ability to 3400 ELO will absolutely crush any human.

Note that I do not know how you measure tactical ability but you basically need a good position to cause the top players to make tactical mistakes.

There are games when I made no tactical mistake because my opponent did not play well and there are also draws that I had with no tactical mistake of one of the players.

I have fide rating of 2004 and
If I can use a computer only to tell me the moves that lose at least 0.5 pawn in every move without suggesting me what to play then I am sure that I have no chance against the top human players.

A human player that is 2000 ELO and has computer assistance can crush any human (including Carlsen, Kramnik, Kasparov, Anand) if he has the "tactical glasses" of a 3400 ELO engine that is giving him say 5 different variations and he gets to choose what he thinks (according to his 2000 ELO ability) the best line the engine recommends. This would be so much of a "no contest" that it does not even need to be tested. Someone posted this link from GM Nakamura ... arguably one of the strongest tactical players ever!

http://www.youtube.com/watch?feature=pl ... hd7I#t=235

It is clear that computers are stronger than humans but it is not clear that 2000 player can beat super GM's in case of seeing 5 different variation of the computer that are ordered in random order.

Note that this option gives the human also some positional knowledge of the engines because there are cases when the human does not know that a different move is bad based on his positional understanding.

[CRoberson]

It has been well shown that class players even as low as 1400 can beat GM's with computer assistance. This was done many times in the Freestyle Chess tournaments. I used to look at the top computer moves from several computers and programs and choose the move I liked based on my personal playing style. I didn't pick the move that had the highest value. On several occasions, I didn't agree with the computers and gave them more think time which led to them producing a better move. Eventually, the GM makes a deep tactical error and the computers see it. When that happens, I let the computers take over in Freestyle events.

But, my real issue in this thread is false positives in cheating claims. As far as I am concerned (and I understand Algorithms quite well), there shouldn't be any false positives. The algorithms should be cut back so that nobody is falsely found guilty.

Generally, I don't trust the mathematical and algorithmic competence of most people running the various Chess sites. I've had several discussions with some of them over the past 20 years (mostly about Glickman system #1) and they lack the competence to understand the issues (despite Glickman stating his first system has issues), then they tell me they don't have anybody that can reprogram that code.

[M ANSARI]

It has been well shown that class players even as low as 1400 can beat GM's with computer assistance. This was done many times in the Freestyle Chess tournaments. I used to look at the top computer moves from several computers and programs and choose the move I liked based on my personal playing style. I didn't pick the move that had the highest value. On several occasions, I didn't agree with the computers and gave them more think time which led to them producing a better move. Eventually, the GM makes a deep tactical error and the computers see it. When that happens, I let the computers take over in Freestyle events.

But, my real issue in this thread is false positives in cheating claims. As far as I am concerned (and I understand Algorithms quite well), there shouldn't be any false positives. The algorithms should be cut back so that nobody is falsely found guilty.

Generally, I don't trust the mathematical and algorithmic competence of most people running the various Chess sites. I've had several discussions with some of them over the past 20 years (mostly about Glickman system #1) and they lack the competence to understand the issues (despite Glickman stating his first system has issues), then they tell me they don't have anybody that can reprogram that code.

If there are false positives in murder convictions where the stakes are much higher, then I think chances are that you will have false positives in chess cheating ... it just simply a matter of trying to reduce this to the absolute minimum possible. Reaching a 100% accuracy rate is most likely not possible.

[CRoberson]

It comes down to which of two systems is best:
1) You convict 110% of the cheaters.
2) You convict 90% of the cheaters.

Both systems are off by 10%, but the first one falsely convicts some innocents while the second case doesn't quite catch all the cheaters.

I say system #2 is best. Don't convict the innocent.
If you use system #1, eventually over time you will convict all people of cheating.
If you use system #2, eventually over time you will catch all cheaters without convicting the innocent.

[bob]

It has been well shown that class players even as low as 1400 can beat GM's with computer assistance. This was done many times in the Freestyle Chess tournaments. I used to look at the top computer moves from several computers and programs and choose the move I liked based on my personal playing style. I didn't pick the move that had the highest value. On several occasions, I didn't agree with the computers and gave them more think time which led to them producing a better move. Eventually, the GM makes a deep tactical error and the computers see it. When that happens, I let the computers take over in Freestyle events.

But, my real issue in this thread is false positives in cheating claims. As far as I am concerned (and I understand Algorithms quite well), there shouldn't be any false positives. The algorithms should be cut back so that nobody is falsely found guilty.

You DO realize that this is simple statistical analysis? Here's a question to ponder...

"what does it take to make P(false_positive) == 0"???

The ONLY answer is "do not do the test". You can get false positives on copyright analysis. A monkey COULD type Lincoln's Gettysburg Address. If you tell your doctor to ONLY report diagnoses that he is 100% certain about, with absolutely zero probability of being wrong, you are going to shorten your lifetime by decades. I don't think anyone would use a single game as a serious accusation. Perhaps as a "need to look a lot closer at this guy's games" perhaps.

Generally, I don't trust the mathematical and algorithmic competence of most people running the various Chess sites. I've had several discussions with some of them over the past 20 years (mostly about Glickman system #1) and they lack the competence to understand the issues (despite Glickman stating his first system has issues), then they tell me they don't have anybody that can reprogram that code.

That is a completely different issue. Someone with no technical competence could accuse you of being an alien from outer space. No way to prevent it.

[bob]

It comes down to which of two systems is best:
1) You convict 110% of the cheaters.
2) You convict 90% of the cheaters.

Both systems are off by 10%, but the first one falsely convicts some innocents while the second case doesn't quite catch all the cheaters.

I say system #2 is best. Don't convict the innocent.
If you use system #1, eventually over time you will convict all people of cheating.
If you use system #2, eventually over time you will catch all cheaters without convicting the innocent.

Bad argument. Are you CERTAIN that 90% is only cheaters? Probability is zero that is the case. Even if you drop it to catching only 10% of the cheaters, the probability that they are all cheaters is NOT 100%. Only when you get down to the point where you accuse no one do you reach the point where you will never falsely accuse anyone...