Page 1 of 2

Searching crazy positions

Posted: Thu May 09, 2019 8:15 pm
by AxolotlFever
Hello,

When my engine searches from the starting position, I noticed that it considers board positions which are completely terrible, such as

or


these are searched at a depth of 11, from the starting position.

Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?

Thank you in advance,
Louis

Re: Searching crazy positions

Posted: Thu May 09, 2019 9:19 pm
by Michael Sherwin
This is entering the realm of vagueness. We humans can see these positions as being sort of ridiculous. A computer cannot see. A computer can only calculate within the search tree. How did it arrive at these positions. What was the first ridiculous move? If there was an easy way to forward prune the first ridiculous move Engines would have an 8000 elo. As it stands now moves that lead to positions as these can be limited by techniques like LMR, i.e. spending less time in lines like these. I think in a hundred more years algorithms might be developed that can in a way "see" that the line has become ridiculous and stop the line cold. But we are not at that point yet. Maybe a point system for ridiculous moves can be developed and if a threshold is exceeded the line can be stopped scored or ignored by not scoring it. However, it will be like giving a computer a simile to human vision.

Re: Searching crazy positions

Posted: Thu May 09, 2019 9:21 pm
by Michael Sherwin
deleted

Re: Searching crazy positions

Posted: Thu May 09, 2019 9:35 pm
by Michael Sherwin
deleted

Re: Searching crazy positions

Posted: Thu May 09, 2019 10:48 pm
by Michael Sherwin
Here is a game that came to mind with a crazy exposed king. How many ridiculous positions were searched in this game? How would one root them out without losing the winning line? This is an example why the problem is so vague.


Re: Searching crazy positions

Posted: Fri May 10, 2019 2:57 am
by Uri Blass
Michael Sherwin wrote:
Thu May 09, 2019 9:19 pm
This is entering the realm of vagueness. We humans can see these positions as being sort of ridiculous. A computer cannot see. A computer can only calculate within the search tree. How did it arrive at these positions. What was the first ridiculous move? If there was an easy way to forward prune the first ridiculous move Engines would have an 8000 elo. As it stands now moves that lead to positions as these can be limited by techniques like LMR, i.e. spending less time in lines like these. I think in a hundred more years algorithms might be developed that can in a way "see" that the line has become ridiculous and stop the line cold. But we are not at that point yet. Maybe a point system for ridiculous moves can be developed and if a threshold is exceeded the line can be stopped scored or ignored by not scoring it. However, it will be like giving a computer a simile to human vision.
I do not agree that engines could get 8000 elo by pruning ridiculous moves.
I believe that humans consider some good moves as ridiculous so trying to use the same algorithm as humans is not a solution.

Re: Searching crazy positions

Posted: Fri May 10, 2019 5:31 am
by Michael Sherwin
My use of the word ridiculous was probably an oversimplification. I only used that word because the positions given look that way to humans such as the original poster. I was just trying to make a point. The number 8000 was just a number that I pulled out of thin air because I do not think we know what the true number is. But that number was also making a point. The point being if an engine knew what moves were worth searching deeper and which moves were not worth searching deeper it would approach solving chess. And an 8000 elo represents to me the virtual solving of chess. But once again we do not know the elo of an engine that solves chess. So that number is just a guess.

Re: Searching crazy positions

Posted: Fri May 10, 2019 5:38 am
by hgm
AxolotlFever wrote:
Thu May 09, 2019 8:15 pm
Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?
This is absolutely normal; >99.9% of the positions conventional alpha-beta engines consider belong in this totally ridiculous category. This is why a smart engine like AlphaZero can be competitive while searching 1000 times fewer nodes.

In strong engines it is often even worse than in weak engines, because many pruning techniques promote 'ridiculous' behavior. E.g. in an engine that uses null-move pruning almost all branches of the tree look like a random mover is playing against a player that only passes its turn, even in the face of being able to grab a hanging Queen.

This is why evaluation ideas that intuitively sound reasonable often just weaken an engine; most positions that it evaluates do not look anything like what we are used to in chess games, so our intuition just isn't any good.

Re: Searching crazy positions

Posted: Fri May 10, 2019 8:02 pm
by petero2
AxolotlFever wrote:
Thu May 09, 2019 8:15 pm
When my engine searches from the starting position, I noticed that it considers board positions which are completely terrible
...
Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?
It is probably likely to search some such positions. If the engine searches millions of positions, some of them are likely going to be really ridiculous.

Making your engine stronger (better move ordering, pruning and evaluation) will likely make it search fewer "terrible" positions.
hgm wrote:
Fri May 10, 2019 5:38 am
AxolotlFever wrote:
Thu May 09, 2019 8:15 pm
Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?
This is absolutely normal; >99.9% of the positions conventional alpha-beta engines consider belong in this totally ridiculous category.
Did you actually test this and if so what engine did you use?

I get completely different results with Texel. I added code to the search and q-search functions that with probability 1/65536 logs the current position. I let it search from the starting position and when I stopped the search after a while it had logged 3139 positions.

The average ply value among the logged positions was 17.1 with a standard deviation of 4.4.

The average remaining depth among the logged positions was 0.08 with a standard deviation of 2.1. In my engine depth 0 is the last ply before q-search, depth -1 is the first q-search level, depth -2 is the second q-search level, etc.

I then analyzed all logged positions using Texel and 100ms search time per position. I then calculated the number of positions having abs(score) < s for various values of s:

Code: Select all

    s  nPos  nPos/totPos
    1    12       0.0038
   10   144       0.0459
   25   341       0.1086
   50   619       0.1972
  100  1178       0.3753
  200  1648       0.5250
  500  2435       0.7757
 1000  2818       0.8977
If we consider positions with a score >= 100cp "ridiculous", it means that about 1-0.3753 = 62.5% of the positions are ridiculous. Even if we consider all positions having a score != 0 ridiculous, only 99.6% of the positions are ridiculous.

A more reasonable definition of ridiculous might be abs(score)>=500, in which case about 22% of the searched positions were ridiculous.

Re: Searching crazy positions

Posted: Fri May 10, 2019 9:15 pm
by Michael Sherwin
The caveat to that is even positions that are equal can be ridiculous or were arrived at by a ridiculous move sequence.