Searching crazy positions

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
AxolotlFever
Posts: 46
Joined: Sun Nov 11, 2018 8:26 pm
Location: Germany
Full name: Louis Mackenzie-Smith
Contact:

Searching crazy positions

Post by AxolotlFever » Thu May 09, 2019 8:15 pm

Hello,

When my engine searches from the starting position, I noticed that it considers board positions which are completely terrible, such as

or


these are searched at a depth of 11, from the starting position.

Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?

Thank you in advance,
Louis

Michael Sherwin
Posts: 3024
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: Searching crazy positions

Post by Michael Sherwin » Thu May 09, 2019 9:19 pm

This is entering the realm of vagueness. We humans can see these positions as being sort of ridiculous. A computer cannot see. A computer can only calculate within the search tree. How did it arrive at these positions. What was the first ridiculous move? If there was an easy way to forward prune the first ridiculous move Engines would have an 8000 elo. As it stands now moves that lead to positions as these can be limited by techniques like LMR, i.e. spending less time in lines like these. I think in a hundred more years algorithms might be developed that can in a way "see" that the line has become ridiculous and stop the line cold. But we are not at that point yet. Maybe a point system for ridiculous moves can be developed and if a threshold is exceeded the line can be stopped scored or ignored by not scoring it. However, it will be like giving a computer a simile to human vision.
Last edited by Michael Sherwin on Thu May 09, 2019 9:34 pm, edited 1 time in total.
I hate if statements. Pawns demand if statements. Therefore I hate pawns.

Michael Sherwin
Posts: 3024
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: Searching crazy positions

Post by Michael Sherwin » Thu May 09, 2019 9:21 pm

deleted
Last edited by Michael Sherwin on Thu May 09, 2019 9:35 pm, edited 1 time in total.
I hate if statements. Pawns demand if statements. Therefore I hate pawns.

Michael Sherwin
Posts: 3024
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: Searching crazy positions

Post by Michael Sherwin » Thu May 09, 2019 9:35 pm

deleted
I hate if statements. Pawns demand if statements. Therefore I hate pawns.

Michael Sherwin
Posts: 3024
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: Searching crazy positions

Post by Michael Sherwin » Thu May 09, 2019 10:48 pm

Here is a game that came to mind with a crazy exposed king. How many ridiculous positions were searched in this game? How would one root them out without losing the winning line? This is an example why the problem is so vague.

I hate if statements. Pawns demand if statements. Therefore I hate pawns.

Uri Blass
Posts: 8506
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Searching crazy positions

Post by Uri Blass » Fri May 10, 2019 2:57 am

Michael Sherwin wrote:
Thu May 09, 2019 9:19 pm
This is entering the realm of vagueness. We humans can see these positions as being sort of ridiculous. A computer cannot see. A computer can only calculate within the search tree. How did it arrive at these positions. What was the first ridiculous move? If there was an easy way to forward prune the first ridiculous move Engines would have an 8000 elo. As it stands now moves that lead to positions as these can be limited by techniques like LMR, i.e. spending less time in lines like these. I think in a hundred more years algorithms might be developed that can in a way "see" that the line has become ridiculous and stop the line cold. But we are not at that point yet. Maybe a point system for ridiculous moves can be developed and if a threshold is exceeded the line can be stopped scored or ignored by not scoring it. However, it will be like giving a computer a simile to human vision.
I do not agree that engines could get 8000 elo by pruning ridiculous moves.
I believe that humans consider some good moves as ridiculous so trying to use the same algorithm as humans is not a solution.

Michael Sherwin
Posts: 3024
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: Searching crazy positions

Post by Michael Sherwin » Fri May 10, 2019 5:31 am

My use of the word ridiculous was probably an oversimplification. I only used that word because the positions given look that way to humans such as the original poster. I was just trying to make a point. The number 8000 was just a number that I pulled out of thin air because I do not think we know what the true number is. But that number was also making a point. The point being if an engine knew what moves were worth searching deeper and which moves were not worth searching deeper it would approach solving chess. And an 8000 elo represents to me the virtual solving of chess. But once again we do not know the elo of an engine that solves chess. So that number is just a guess.
I hate if statements. Pawns demand if statements. Therefore I hate pawns.

User avatar
hgm
Posts: 23379
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Searching crazy positions

Post by hgm » Fri May 10, 2019 5:38 am

AxolotlFever wrote:
Thu May 09, 2019 8:15 pm
Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?
This is absolutely normal; >99.9% of the positions conventional alpha-beta engines consider belong in this totally ridiculous category. This is why a smart engine like AlphaZero can be competitive while searching 1000 times fewer nodes.

In strong engines it is often even worse than in weak engines, because many pruning techniques promote 'ridiculous' behavior. E.g. in an engine that uses null-move pruning almost all branches of the tree look like a random mover is playing against a player that only passes its turn, even in the face of being able to grab a hanging Queen.

This is why evaluation ideas that intuitively sound reasonable often just weaken an engine; most positions that it evaluates do not look anything like what we are used to in chess games, so our intuition just isn't any good.

petero2
Posts: 580
Joined: Mon Apr 19, 2010 5:07 pm
Location: Sweden
Contact:

Re: Searching crazy positions

Post by petero2 » Fri May 10, 2019 8:02 pm

AxolotlFever wrote:
Thu May 09, 2019 8:15 pm
When my engine searches from the starting position, I noticed that it considers board positions which are completely terrible
...
Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?
It is probably likely to search some such positions. If the engine searches millions of positions, some of them are likely going to be really ridiculous.

Making your engine stronger (better move ordering, pruning and evaluation) will likely make it search fewer "terrible" positions.
hgm wrote:
Fri May 10, 2019 5:38 am
AxolotlFever wrote:
Thu May 09, 2019 8:15 pm
Is it normal to search positions that are this unlikely to happen in a real game? Or do your engines find a reason not to go anywhere near them?
This is absolutely normal; >99.9% of the positions conventional alpha-beta engines consider belong in this totally ridiculous category.
Did you actually test this and if so what engine did you use?

I get completely different results with Texel. I added code to the search and q-search functions that with probability 1/65536 logs the current position. I let it search from the starting position and when I stopped the search after a while it had logged 3139 positions.

The average ply value among the logged positions was 17.1 with a standard deviation of 4.4.

The average remaining depth among the logged positions was 0.08 with a standard deviation of 2.1. In my engine depth 0 is the last ply before q-search, depth -1 is the first q-search level, depth -2 is the second q-search level, etc.

I then analyzed all logged positions using Texel and 100ms search time per position. I then calculated the number of positions having abs(score) < s for various values of s:

Code: Select all

    s  nPos  nPos/totPos
    1    12       0.0038
   10   144       0.0459
   25   341       0.1086
   50   619       0.1972
  100  1178       0.3753
  200  1648       0.5250
  500  2435       0.7757
 1000  2818       0.8977
If we consider positions with a score >= 100cp "ridiculous", it means that about 1-0.3753 = 62.5% of the positions are ridiculous. Even if we consider all positions having a score != 0 ridiculous, only 99.6% of the positions are ridiculous.

A more reasonable definition of ridiculous might be abs(score)>=500, in which case about 22% of the searched positions were ridiculous.

Michael Sherwin
Posts: 3024
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: Searching crazy positions

Post by Michael Sherwin » Fri May 10, 2019 9:15 pm

The caveat to that is even positions that are equal can be ridiculous or were arrived at by a ridiculous move sequence.
I hate if statements. Pawns demand if statements. Therefore I hate pawns.

Post Reply