A position to troll your engine :-)

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: A position to troll your engine :-)

Post by zullil »

lech wrote: Tue Sep 10, 2019 5:51 pm
zullil wrote: Tue Sep 10, 2019 5:10 pm
lech wrote: Mon Sep 09, 2019 3:40 pm [d]kB5K/2P4B/P4rP1/4r3/2N1n3/1RP4N/3b4/6RQ b - - 0 1

It is a wonderful example (thanks!) to prove that "auto-playing" can be the best way to get such long-string-solution positions.

Sting 16++ returns the full solution in 5 second per move (old dual core!)

f6f8 h8g7 d2h6 g7h6 e5h5 h6h5 f8f5 h3g5 f5g5 h5h6 g5g6 h6h5 g6g5 h5h4 g5g4 h4h3 g4g3 h3h2 g3h3 h2g2 h3g3 g2f1 g3f3 f1e1 f3e3 e1d1 e3d3 d1c1 d3d1 c1b2 d1d2 b2a3 d2a2 a3b4 a2a4 b4b5 a4a5 b5c6 a5c5 c6d7 c5d5 d7e8 d5e5 e8f7 e5f5 f7g7 f5g5 h7g6 g5g6 g7h7 g6g7 h7h6 g7g6 h6h5 g6g5 h5h4 g5g4 h4h3 g4g3 h3h2 g3h3 h2g2 h3g3 g2f1 g3f3 f1e1 f3e3 e2d1 e3d3 d1c1 d3d1 c1b2 d1d2 b2a3 d2a2 a3b4 a2a4 b4b5 a4a5 b5c6 a5c5 c6d7 c5d5 c4d6 d5d6 d7e7 d6e6 e7f7 e6f6 f7g7 f6g6 g7h7 g6g7 h7h6 g7g6 h6h5 g6g5 h5h4 g5g4 h4h3 g4g3 h3h2 g3h3 h2g2 h3g3 g2f1 g3f3 f1e1 f3e3 e2d1 e3d3 d1c1 d3d1 c1b2 d1d2 b2a3 d2a2 a3b4 a2a4 b4b5 a4a5 b5c4 a5c5 c4d3 c5c4 c7c8q c4d4 d3e3 d4d3 e3e4 d3d7 c8c6 d7b7 c6b7 Black is mated :D
Is this "auto-playing" done after you have run your search, so that the hash table is already filled and a (roughly) correct PV is already determined?
Not, I started all the items with clear hashes.
That seems very impressive then. Starting directly from the initial position, each move in the line is found in no more than 5 seconds per move? And resulting line is a minimal mating sequence?
lech
Posts: 1136
Joined: Sun Feb 14, 2010 10:02 pm

Re: A position to troll your engine :-)

Post by lech »

zullil wrote: Tue Sep 10, 2019 8:54 pm
lech wrote: Tue Sep 10, 2019 5:51 pm
zullil wrote: Tue Sep 10, 2019 5:10 pm
lech wrote: Mon Sep 09, 2019 3:40 pm kB5K/2P4B/P4rP1/4r3/2N1n3/1RP4N/3b4/6RQ b - - 0 1

It is a wonderful example (thanks!) to prove that "auto-playing" can be the best way to get such long-string-solution positions.

Sting 16++ returns the full solution in 5 second per move (old dual core!)

f6f8 h8g7 d2h6 g7h6 e5h5 h6h5 f8f5 h3g5 f5g5 h5h6 g5g6 h6h5 g6g5 h5h4 g5g4 h4h3 g4g3 h3h2 g3h3 h2g2 h3g3 g2f1 g3f3 f1e1 f3e3 e1d1 e3d3 d1c1 d3d1 c1b2 d1d2 b2a3 d2a2 a3b4 a2a4 b4b5 a4a5 b5c6 a5c5 c6d7 c5d5 d7e8 d5e5 e8f7 e5f5 f7g7 f5g5 h7g6 g5g6 g7h7 g6g7 h7h6 g7g6 h6h5 g6g5 h5h4 g5g4 h4h3 g4g3 h3h2 g3h3 h2g2 h3g3 g2f1 g3f3 f1e1 f3e3 e2d1 e3d3 d1c1 d3d1 c1b2 d1d2 b2a3 d2a2 a3b4 a2a4 b4b5 a4a5 b5c6 a5c5 c6d7 c5d5 c4d6 d5d6 d7e7 d6e6 e7f7 e6f6 f7g7 f6g6 g7h7 g6g7 h7h6 g7g6 h6h5 g6g5 h5h4 g5g4 h4h3 g4g3 h3h2 g3h3 h2g2 h3g3 g2f1 g3f3 f1e1 f3e3 e2d1 e3d3 d1c1 d3d1 c1b2 d1d2 b2a3 d2a2 a3b4 a2a4 b4b5 a4a5 b5c4 a5c5 c4d3 c5c4 c7c8q c4d4 d3e3 d4d3 e3e4 d3d7 c8c6 d7b7 c6b7 Black is mated :D
Is this "auto-playing" done after you have run your search, so that the hash table is already filled and a (roughly) correct PV is already determined?
Not, I started all the items with clear hashes.
That seems very impressive then. Starting directly from the initial position, each move in the line is found in no more than 5 seconds per move? And resulting line is a minimal mating sequence?
All the 4 studies should be very easy for engines but, not by searching, and playing.
Please try the great van Breukelen's study:

[d]1r4Q1/7p/2N1B2k/7P/3Pp1P1/K1N1P2R/PP2PR2/8 b - - 0 1
to play as match eg. Stockfish vs Stockfish and, if the used engine is ok, you always (9/10) will see the result 1:0 and full solution.
It seems to be a very good position and a method to test engines (PLY_MAX and rule-50).
Maybe, I can't be friendly, but let me be useful.
Vinvin
Posts: 5228
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: A position to troll your engine :-)

Post by Vinvin »

lech wrote: Thu Sep 12, 2019 8:40 am ...
All the 4 studies should be very easy for engines but, not by searching, and playing.
Please try the great van Breukelen's study:

[d]1r4Q1/7p/2N1B2k/7P/3Pp1P1/K1N1P2R/PP2PR2/8 b - - 0 1
to play as match eg. Stockfish vs Stockfish and, if the used engine is ok, you always (9/10) will see the result 1:0 and full solution.
It seems to be a very good position and a method to test engines (PLY_MAX and rule-50).
Stockfish still has troubles here, even after 5 billions nodes :

FEN: 1r4Q1/7p/2N1B2k/7P/3Pp1P1/K1N1P2R/PP2PR2/8 b - - 0 1

Code: Select all

Stockfish_19082608_x64_modern:
Found 511 tablebases
...
 140/32-	03:33	2 481 914 646	11 611 948	+47,32	1. ... Rb3+
 140/32-	03:33	2 481 934 870	11 611 934	+45,27	1. ... Rb3+
 140/54-	03:42	2 571 660 415	11 570 556	+4,27	1. ... Rb3+
 140/54	03:46	2 639 572 092	11 636 067	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+ 8.Kc8 Rc7+ 9.Kd8 Rd7+ 10.Ke8 Rd8+ 11.Kf7 Rd7+ 12.Kf6 Rf7+ 13.Ke5 Rf5+ 14.Kd6 Rd5+ 15.Kc7 Rd7+ 16.Kb6 Rb7+ 17.Ka5
 141/46	06:01	3 941 284 105	10 893 694	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+ 8.Kc8 Rc7+ 9.Kd8 Rd7+ 10.Ke8 Re7+ 11.Kf8 Re8+ 12.Kf7 Re7+ 13.Kf6 Rf7+ 14.Ke5 Rf5+ 15.Kd6 Rd5+ 16.Kc7 Rd7+ 17.Kb6 Rb7+ 18.Kc5 Rb5+ 19.Kc4 Rc5+ 20.Kb3 Rb5+
 142/46	06:01	3 941 481 434	10 893 637	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+
 143/46	06:02	3 949 691 942	10 893 237	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+
 144/48+	06:56	4 503 906 995	10 814 169	+0,11	1. ... Rb3+ 2.Ka4
 144/78+	07:28	4 775 745 035	10 643 206	+0,22	1. ... Rb3+ 2.Ka4
 144/78+	08:24	5 279 264 218	10 464 598	+1,26	1. ... Rb3+ 2.Ka4
lech
Posts: 1136
Joined: Sun Feb 14, 2010 10:02 pm

Re: A position to troll your engine :-)

Post by lech »

Vinvin wrote: Fri Sep 13, 2019 2:07 am
lech wrote: Thu Sep 12, 2019 8:40 am ...
All the 4 studies should be very easy for engines but, not by searching, and playing.
Please try the great van Breukelen's study:

[d]1r4Q1/7p/2N1B2k/7P/3Pp1P1/K1N1P2R/PP2PR2/8 b - - 0 1
to play as match eg. Stockfish vs Stockfish and, if the used engine is ok, you always (9/10) will see the result 1:0 and full solution.
It seems to be a very good position and a method to test engines (PLY_MAX and rule-50).
Stockfish still has troubles here, even after 5 billions nodes :

FEN: 1r4Q1/7p/2N1B2k/7P/3Pp1P1/K1N1P2R/PP2PR2/8 b - - 0 1

Code: Select all

Stockfish_19082608_x64_modern:
Found 511 tablebases
...
 140/32-	03:33	2 481 914 646	11 611 948	+47,32	1. ... Rb3+
 140/32-	03:33	2 481 934 870	11 611 934	+45,27	1. ... Rb3+
 140/54-	03:42	2 571 660 415	11 570 556	+4,27	1. ... Rb3+
 140/54	03:46	2 639 572 092	11 636 067	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+ 8.Kc8 Rc7+ 9.Kd8 Rd7+ 10.Ke8 Rd8+ 11.Kf7 Rd7+ 12.Kf6 Rf7+ 13.Ke5 Rf5+ 14.Kd6 Rd5+ 15.Kc7 Rd7+ 16.Kb6 Rb7+ 17.Ka5
 141/46	06:01	3 941 284 105	10 893 694	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+ 8.Kc8 Rc7+ 9.Kd8 Rd7+ 10.Ke8 Re7+ 11.Kf8 Re8+ 12.Kf7 Re7+ 13.Kf6 Rf7+ 14.Ke5 Rf5+ 15.Kd6 Rd5+ 16.Kc7 Rd7+ 17.Kb6 Rb7+ 18.Kc5 Rb5+ 19.Kc4 Rc5+ 20.Kb3 Rb5+
 142/46	06:01	3 941 481 434	10 893 637	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+
 143/46	06:02	3 949 691 942	10 893 237	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+
 144/48+	06:56	4 503 906 995	10 814 169	+0,11	1. ... Rb3+ 2.Ka4
 144/78+	07:28	4 775 745 035	10 643 206	+0,22	1. ... Rb3+ 2.Ka4
 144/78+	08:24	5 279 264 218	10 464 598	+1,26	1. ... Rb3+ 2.Ka4
At first, I very like you Vincent.
You very help me (project Sting). Thanks!!!

The problem why engines have some problems here is:
PLY_MAX should be: minimum 100 (rule 50) + X (for qsearch),
in qsearch if PLY > PLY_MAX return VALUE_ZERO seems to be OK, but in search, if PLY > PLY_MAX - X && !Root return qsearch (never VALUE_ZERO).

In the next version of Sting I am tring PLY_MAX = 104 (100 + 4) and Sting doesn't crash now and solves it by auto-playing (1 engine) or playing (2 engines).
Maybe, I can't be friendly, but let me be useful.
lech
Posts: 1136
Joined: Sun Feb 14, 2010 10:02 pm

Re: A position to troll your engine :-)

Post by lech »

lech wrote: Fri Sep 13, 2019 10:12 am
Vinvin wrote: Fri Sep 13, 2019 2:07 am
lech wrote: Thu Sep 12, 2019 8:40 am ...
All the 4 studies should be very easy for engines but, not by searching, and playing.
Please try the great van Breukelen's study:

[d]1r4Q1/7p/2N1B2k/7P/3Pp1P1/K1N1P2R/PP2PR2/8 b - - 0 1
to play as match eg. Stockfish vs Stockfish and, if the used engine is ok, you always (9/10) will see the result 1:0 and full solution.
It seems to be a very good position and a method to test engines (PLY_MAX and rule-50).
Stockfish still has troubles here, even after 5 billions nodes :

FEN: 1r4Q1/7p/2N1B2k/7P/3Pp1P1/K1N1P2R/PP2PR2/8 b - - 0 1

Code: Select all

Stockfish_19082608_x64_modern:
Found 511 tablebases
...
 140/32-	03:33	2 481 914 646	11 611 948	+47,32	1. ... Rb3+
 140/32-	03:33	2 481 934 870	11 611 934	+45,27	1. ... Rb3+
 140/54-	03:42	2 571 660 415	11 570 556	+4,27	1. ... Rb3+
 140/54	03:46	2 639 572 092	11 636 067	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+ 8.Kc8 Rc7+ 9.Kd8 Rd7+ 10.Ke8 Rd8+ 11.Kf7 Rd7+ 12.Kf6 Rf7+ 13.Ke5 Rf5+ 14.Kd6 Rd5+ 15.Kc7 Rd7+ 16.Kb6 Rb7+ 17.Ka5
 141/46	06:01	3 941 284 105	10 893 694	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+ 8.Kc8 Rc7+ 9.Kd8 Rd7+ 10.Ke8 Re7+ 11.Kf8 Re8+ 12.Kf7 Re7+ 13.Kf6 Rf7+ 14.Ke5 Rf5+ 15.Kd6 Rd5+ 16.Kc7 Rd7+ 17.Kb6 Rb7+ 18.Kc5 Rb5+ 19.Kc4 Rc5+ 20.Kb3 Rb5+
 142/46	06:01	3 941 481 434	10 893 637	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+
 143/46	06:02	3 949 691 942	10 893 237	 0,00	1. ... Rb3+ 2.Ka4 Rb4+ 3.Ka5 Rb5+ 4.Ka6 Rb6+ 5.Ka7 Rb7+ 6.Ka8 Ra7+ 7.Kb8 Rb7+
 144/48+	06:56	4 503 906 995	10 814 169	+0,11	1. ... Rb3+ 2.Ka4
 144/78+	07:28	4 775 745 035	10 643 206	+0,22	1. ... Rb3+ 2.Ka4
 144/78+	08:24	5 279 264 218	10 464 598	+1,26	1. ... Rb3+ 2.Ka4
At first, I very like you Vincent.
You very help me (project Sting). Thanks!!!

The problem why engines have some problems here is:
PLY_MAX should be: minimum 100 (rule 50) + X (for qsearch),
in qsearch if PLY > PLY_MAX return VALUE_ZERO seems to be OK, but in search, if PLY > PLY_MAX - X && !Root return qsearch (never VALUE_ZERO).

In the next version of Sting I am tring PLY_MAX = 104 (100 + 4) and Sting doesn't crash now and solves it by auto-playing (1 engine) or playing (2 engines).
How to read it?
PLY_MAX is a maximum count of moves in a chain (WHITE-NLACK-WHITE-BLACK...) of moves for engines in searching.
If engines try to analyze the next move, they return ZERO.
I hope not all engines!

Abnormality!!!

Why? Who is the author of this "great" idea? :lol:
Maybe, I can't be friendly, but let me be useful.