hgm wrote:You could sample games, and subject them to a blunder check by a strong engine with 1 sec/move. People that blunder most likely have not been using an engine, so you have filtered out those. (If needed, you could analyze the position just before and after their 'blunder candidate' at a larger depth, to make sure it is truly a blunder, rather than misevaluation by the engine because of the limited time.)
The games that pass the blunder test can be analyzed with somewhat larger time per move (say 10 sec), and remain suspect when there are almost no moves where the evaluation drops much. Moves that do decrease the evaluation can be analyzed with longer time, by a handfull of engines, to see if any of them suggests the move that was played. Games without such moves, or where such moves are all suggested by the same engine, remain suspect.
If you have reduced the number of suspect games enough, you can probably afford to analyze each position with the most-likely used engine at the actual time, to generate hard prove. The trick is to reduce the number of candidates first by much faster tests.
People that cheat, are likely to cheat consistently. If one of their games is sampled, and found suspect, you can subject all their earlier games that were not sampled to the test procedure.
In all cases it will remain very difficult to catch people that cheat with weak engines, such as (say) Fairy-Max. But thenature of cheaters is that when they cheat, they are likely to cheat with the strongest engine they can get their hands on. And would you really care when they do? If it plays like an average player on your server, who cares if it actually was a computer?
Thanks for ideas. Your algorithm seems fine to me. The pre-selection is the must. There are hundreds of games played at same time and I hope it will grow to thousands. I can't check them all. There will be needed multi level testing. So this is general idea. What I need to find out is which positions should be tested and which not. I don't want to create own representation of board and analyse position myself. Algorithm should be able to decide whether the position should be part of engine analysis or not based on UCI analyse results.
By that I mean,
1 step... fast check of all moves for big blunders.
2 step... if there are no blunders, than Which positions should be analysed deeper? Opening part is in my opinion lost time, no engine will give me good information about it. Cheater can or could not use Opening Database. Therefore if he used it, engines can mark some moves as small blunders. If he did not use it, and engine says it was small blunder, than It is confusing too.
If it is endgame, engines do blunders. So there is similar problem. I can't decide if user is using table bases, or not.
I need to select most important positions and conciser only them. Test of 1 position from difficult middle game can give me better results than testing 10 opening positions. I need to do such position selection based on UCI results. General setup like skip first 15 positions and consider only 15 - 40 moves seems to me very optimistic. I am sure that engine knows that the position is specific. Like if it is silent or there are lot changes. If it has to compute deep or it is considering only positional aspects. If it is using silent move, etc...
So are there any ideas how to sort positions by their importance using UCI results?