Searching crazy positions

[AxolotlFever]

Hello,

When my engine searches from the starting position, I noticed that it considers board positions which are completely terrible, such as
[d]rnb1k1nr/pppp1ppp/8/2K5/4q3/8/PPP3PP/RNBQ1B1R b kq - 1 15
or
[d]rnbq2nr/pppp2pp/5k2/8/1b2P3/2N5/PPP2PPP/R1BQKBNR w KQ - 0 1

these are searched at a depth of 11, from the starting position.

Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?

Thank you in advance,
Louis

[Michael Sherwin]

This is entering the realm of vagueness. We humans can see these positions as being sort of ridiculous. A computer cannot see. A computer can only calculate within the search tree. How did it arrive at these positions. What was the first ridiculous move? If there was an easy way to forward prune the first ridiculous move Engines would have an 8000 elo. As it stands now moves that lead to positions as these can be limited by techniques like LMR, i.e. spending less time in lines like these. I think in a hundred more years algorithms might be developed that can in a way "see" that the line has become ridiculous and stop the line cold. But we are not at that point yet. Maybe a point system for ridiculous moves can be developed and if a threshold is exceeded the line can be stopped scored or ignored by not scoring it. However, it will be like giving a computer a simile to human vision.

[Michael Sherwin]

deleted

[Michael Sherwin]

deleted

[Michael Sherwin]

Here is a game that came to mind with a crazy exposed king. How many ridiculous positions were searched in this game? How would one root them out without losing the winning line? This is an example why the problem is so vague.

[pgn] ["67th Amateur D7 (12.5)"] [Site "ChessGUI"] [Date "2018.01.03"] [Round "?"] [White "Jumbo 0.6.10 64-bit"] [Black "RomiChess P3n 64-bit"] [Result "0-1"] [BlackElo "2400"] [ECO "E12"] [Opening "Queen's Indian"] [Time "16:21:52"] [Variation "Petrosian, 5.Nc3"] [WhiteElo "2400"] [Termination "unterminated"] [PlyCount "111"] [WhiteType "computer"] [BlackType "computer"] 1. c4 b6 2. d4 e6 3. a3 Bb7 4. d5 Nf6 5. Nc3 Bd6 6. Nf3 O-O 7. g3 exd5 8. cxd5 c6 9. Bg5 Re8 10. Qd4 Bc5 11. Bxf6 Bxd4 12. Bxd8 Bxc3+ 13. bxc3 Rxd8 14. d6 Re8 15. Bh3 c5 16. O-O Rxe2 17. Rfe1 Rxe1+ 18. Rxe1 f6 19. Re8+ Kf7 20. Re7+ Kf8 21. Bg2 Bd5 22. c4 Bc6 23. h4 h5 24. Kf1 a6 25. Kg1 Ra7 26. Re3 Kf7 27. Re7+ Kg6 28. Re8 Rb7 29. Rf8 b5 30. Bf1 Kf5 31. Bh3+ Ke4 32. Re8+ Kd3 33. Bf1+ Kc3 34. Re3+ Kb2 35. Nd2 b4 36. axb4 cxb4 37. Nb3 Ra7 38. Na5 Ba4 39. c5 Nc6 40. Nc4+ Kb1 41. Nb6 Bb5 42. Bh3 a5 43. Bxd7 a4 44. Bf5+ Kb2 45. Re1 a3 46. Rb1+ Kc3 47. Rc1+ Kd4 48. Nc8 Ra6 49. d7 a2 50. Ra1 b3 51. Ne7 Nd8 52. c6 Bxc6 53. Nxc6+ Rxc6 54. g4 Kc3 55. Kh1 Kb2 56. Rd1 * [/pgn]

[Uri Blass]

This is entering the realm of vagueness. We humans can see these positions as being sort of ridiculous. A computer cannot see. A computer can only calculate within the search tree. How did it arrive at these positions. What was the first ridiculous move? If there was an easy way to forward prune the first ridiculous move Engines would have an 8000 elo. As it stands now moves that lead to positions as these can be limited by techniques like LMR, i.e. spending less time in lines like these. I think in a hundred more years algorithms might be developed that can in a way "see" that the line has become ridiculous and stop the line cold. But we are not at that point yet. Maybe a point system for ridiculous moves can be developed and if a threshold is exceeded the line can be stopped scored or ignored by not scoring it. However, it will be like giving a computer a simile to human vision.

I do not agree that engines could get 8000 elo by pruning ridiculous moves.
I believe that humans consider some good moves as ridiculous so trying to use the same algorithm as humans is not a solution.

[Michael Sherwin]

My use of the word ridiculous was probably an oversimplification. I only used that word because the positions given look that way to humans such as the original poster. I was just trying to make a point. The number 8000 was just a number that I pulled out of thin air because I do not think we know what the true number is. But that number was also making a point. The point being if an engine knew what moves were worth searching deeper and which moves were not worth searching deeper it would approach solving chess. And an 8000 elo represents to me the virtual solving of chess. But once again we do not know the elo of an engine that solves chess. So that number is just a guess.

[hgm]

Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?

This is absolutely normal; >99.9% of the positions conventional alpha-beta engines consider belong in this totally ridiculous category. This is why a smart engine like AlphaZero can be competitive while searching 1000 times fewer nodes.

In strong engines it is often even worse than in weak engines, because many pruning techniques promote 'ridiculous' behavior. E.g. in an engine that uses null-move pruning almost all branches of the tree look like a random mover is playing against a player that only passes its turn, even in the face of being able to grab a hanging Queen.

This is why evaluation ideas that intuitively sound reasonable often just weaken an engine; most positions that it evaluates do not look anything like what we are used to in chess games, so our intuition just isn't any good.

[petero2]

When my engine searches from the starting position, I noticed that it considers board positions which are completely terrible
...
Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?

It is probably likely to search some such positions. If the engine searches millions of positions, some of them are likely going to be really ridiculous.

Making your engine stronger (better move ordering, pruning and evaluation) will likely make it search fewer "terrible" positions.

Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?

This is absolutely normal; >99.9% of the positions conventional alpha-beta engines consider belong in this totally ridiculous category.

Did you actually test this and if so what engine did you use?

I get completely different results with Texel. I added code to the search and q-search functions that with probability 1/65536 logs the current position. I let it search from the starting position and when I stopped the search after a while it had logged 3139 positions.

The average ply value among the logged positions was 17.1 with a standard deviation of 4.4.

The average remaining depth among the logged positions was 0.08 with a standard deviation of 2.1. In my engine depth 0 is the last ply before q-search, depth -1 is the first q-search level, depth -2 is the second q-search level, etc.

I then analyzed all logged positions using Texel and 100ms search time per position. I then calculated the number of positions having abs(score) < s for various values of s:

Code: Select all

    s  nPos  nPos/totPos
    1    12       0.0038
   10   144       0.0459
   25   341       0.1086
   50   619       0.1972
  100  1178       0.3753
  200  1648       0.5250
  500  2435       0.7757
 1000  2818       0.8977

If we consider positions with a score >= 100cp "ridiculous", it means that about 1-0.3753 = 62.5% of the positions are ridiculous. Even if we consider all positions having a score != 0 ridiculous, only 99.6% of the positions are ridiculous.

A more reasonable definition of ridiculous might be abs(score)>=500, in which case about 22% of the searched positions were ridiculous.

[Michael Sherwin]

The caveat to that is even positions that are equal can be ridiculous or were arrived at by a ridiculous move sequence.