"Poor Fish": very hard test suite

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

BBauer
Posts: 658
Joined: Wed Mar 08, 2006 8:58 pm

Re: "Poor Fish": very hard test suite

Post by BBauer »

Code: Select all

peter wrote:
Or you simply switch off nullmove- pruning.
Smile

Exactly, adding a simple UCI option for this is trivial (I have one in my engine)
I do the something similar:

if depth > number

so it is more flexibel.

Perhaps Marco will read this. :D
kind regards
Bernhard
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: "Poor Fish": very hard test suite

Post by mcostalba »

mcostalba wrote:
Vinvin wrote: I ran an overnight analyze on this 7 positions.
Great job!

I have just set-up a site under GitHub pages:

https://mcostalba.github.io/PoorFish/
I have added support for interactive pgn thanks to pgn4web, unfortunately this forced me to move hosting to

http://poorfish.bitballoon.com/

Because GitHub pages do not support iframe embedding that is needed for the PGN viewer.
BBauer
Posts: 658
Joined: Wed Mar 08, 2006 8:58 pm

Re: "Poor Fish": very hard test suite

Post by BBauer »

By adding

Code: Select all

&&  pos.non_pawn_material(pos.side_to_move()) > RookValueEg
to step 7 and step 8 in search.cpp
I got

Code: Select all

  20/26	00:01	 2.172k	2.115k	+1,07	a2-a4 Bg5-d2 f3-f4 Bd2xf4 a4-a5 b6xa5 Ke8-e7 h6-h5 b5-b6 h5-h4 Bc2-f5 Kg8-g7 b6-b7 Bf4-h2 Ke7-e6 Bh2-b8 Bf5-h3 Kg7-g6 Bh3-f1 Kg6-g5 Ke6-d5 Kg5-f4 Bf1-h3 Kf4-f3
 21/26	00:01	 2.298k	2.097k	+1,08	a2-a4 Bg5-d2 f3-f4 Bd2xf4 a4-a5 b6xa5 Ke8-e7 h6-h5 b5-b6 h5-h4 Bc2-f5 Kg8-g7 b6-b7 Bf4-h2 Ke7-e6 Bh2-b8 Bf5-h3 Kg7-g6 Ke6-d5 Kg6-g5 Kd5xc5 Kg5-f4 Kc5-d5
 22/27	00:01	 2.701k	2.079k	+1,08	a2-a4 Bg5-d2 f3-f4 Bd2xf4 a4-a5 b6xa5 b5-b6 h6-h5 Ke8-e7 Kg8-g7 Bc2-f5 h5-h4 Ke7-e6 Bf4-h2 Ke6-d5 Kg7-f6 Bf5-h3 Kf6-e7 Kd5xc5 Bh2-f4 Bh3-g4 Ke7-d8 Bg4-f5
 23/27	00:01	 3.909k	2.036k	+1,04	Ke8-d7 Bg5-d2 Kd7-c6 Bd2-a5 f3-f4 h6-h5 Bc2-e4 Kg8-g7 a2-a3 Kg7-f6 Be4-f3 Kf6-g6 Bf3-e2 Kg6-f6 Be2-d1 Kf6-g6 Bd1-f3 Kg6-f6 b3-b4 c5xb4 a3xb4 Ba5xb4 Kc6xb6
 24/28	00:03	 7.159k	2.114k	+1,00	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 Bf5-g6 h5-h4 Bg6-f5 Kg8-f7 Bf5-c8 Kf7-e7 f3-f4 Ke7-f6 Bc8-h3 Kf6-e7 Bh3-f5 Ke7-f6 Bf5-c8 Kf6-f7 Bc8-d7 Kf7-f6 a2-a3 Kf6-f7 Bd7-h3 Kf7-f6
 25/28	00:03	 8.278k	2.135k	+1,00	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 Bf5-g6 h5-h4 Bg6-f5 Kg8-f7 Bf5-c8 Kf7-e7 f3-f4 Ke7-f6 Bc8-d7 Kf6-e7 Bd7-f5 Ke7-f7 Bf5-h3 Kf7-f6 Bh3-c8 Kf6-f7 Bc8-g4 Kf7-f6 Bg4-h3 Kf6-f7
 26/29	00:04	 9.435k	2.162k	+1,00	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 Bf5-g6 h5-h4 Bg6-f5 Kg8-f7 Bf5-c8 Kf7-f6 f3-f4 Kf6-e7 Bc8-f5 Ke7-f6 Bf5-h3 Kf6-e7 Bh3-c8 Ke7-f6 Bc8-d7 Kf6-f7 Kc6-c7 Kf7-e7 Bd7-c8 Ke7-f6 Bc8-g4 Kf6-e7 Kc7-c6
 27/29	00:05	 11.071k	2.166k	+1,00	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 Bf5-g6 h5-h4 Bg6-f5 Kg8-f7 Bf5-c8 Kf7-f6 f3-f4 Kf6-e7 Bc8-f5 Ke7-f6 Bf5-g4 Kf6-e7 Kc6-c7 Ke7-f6 Bg4-c8 Kf6-e7 Kc7-c6 Ke7-f6 Bc8-d7 Kf6-e7 Bd7-g4 Ke7-f6 Kc6-c7
 28/32	00:12	 28.091k	2.267k	+1,07	Ke8-d7 Bg5-d2 a2-a3 Kg8-f7 Kd7-c7 Bd2-a5 Kc7-c6 h6-h5 f3-f4 Kf7-f6 Bc2-d1 h5-h4 Bd1-g4 Kf6-e7 Bg4-c8 Ke7-f6 Bc8-h3 Kf6-e7 Bh3-g4 Ke7-f6 Kc6-c7 Kf6-e7 b3-b4 c5xb4 a3xb4 Ba5xb4 Kc7xb6 Ke7-f6 Kb6-c6
 29/37+	00:16	 39.780k	2.346k	+1,18	Ke8-d7
 29/37+	00:17	 40.068k	2.348k	+1,25	Ke8-d7
 29/37	00:17	 40.709k	2.356k	+1,18	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 a2-a3 Kg8-f7 f3-f4 Kf7-e7 Bf5-h3 h5-h4 Bh3-f5 Ke7-f7 Bf5-c8 Kf7-e7 Bc8-h3 Ke7-d8 f2-f3 Kd8-e7 b3-b4 c5xb4 a3xb4 Ba5xb4 Kc6xb6 Ke7-d6 Kb6-a6 Kd6-d5 b5-b6
 30/37+	00:17	 40.824k	2.356k	+1,26	Ke8-d7
 30/37	00:17	 41.541k	2.362k	+1,27	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 a2-a3 Kg8-f7 f3-f4 Kf7-e7 Bf5-h3 h5-h4 Bh3-f5 Ke7-f7 Bf5-c8 Kf7-e7 Bc8-h3 Ke7-f6 Bh3-g4 Kf6-g7 Kc6-b7 Kg7-f6 Kb7-a7 Kf6-f7 b3-b4 c5xb4 a3xb4 Ba5xb4 Ka7xb6 Kf7-f6
 31/37+	00:19	 45.566k	2.394k	+1,35	Ke8-d7
 31/37+	00:19	 47.071k	2.399k	+1,42	Ke8-d7
 31/37-	00:20	 49.617k	2.390k	+1,35	Ke8-d7 Bg5-d2
 31/39+	00:21	 51.261k	2.393k	+1,43	Ke8-d7
 31/39	00:26	 62.277k	2.377k	+1,48	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f7 a2-a3 h5-h4 Bf5-h3 Kf7-f6 Bh3-g4 Kf6-f7 Kc6-c7 Kf7-f6 Kc7-b7 Kf6-f7 Kb7-c6 Kf7-f6 Kc6-c7 Kf6-f7 b3-b4 Ba5xb4 a3xb4 c5-c4 Bg4-f3 c4-c3 Bf3-e4 h4-h3 f4-f5
 32/39+	00:32	 75.162k	2.346k	+1,55	Ke8-d7
 32/39-	00:38	 90.854k	2.342k	+1,48	Ke8-d7 Bg5-d2
 32/39	00:41	 97.194k	2.346k	+1,45	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f8 a2-a3 Kf8-e7 Kc6-c7 h5-h4 Bf5-h3 Ke7-f6 Kc7-b7 Kf6-e7 Kb7-c6 Ke7-f7 Bh3-g4 Kf7-f6 Kc6-c7 Kf6-f7 Kc7-b7 Kf7-f6 Kb7-c6 Kf6-f7 Bg4-h3 Kf7-e7 Bh3-c8 Ke7-f6 Kc6-c7 Kf6-e7 Bc8-f5 Ke7-f6
 33/39	00:45	 108.145k	2.353k	+1,41	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f8 a2-a3 Kf8-e7 Kc6-c7 h5-h4 Bf5-h3 Ke7-f6 Kc7-b7 Kf6-e7 Kb7-c6 Ke7-f7 Bh3-f5 Kf7-f6 Bf5-g4 Kf6-f7 Bg4-h3 Kf7-e7 Bh3-c8 Ke7-f6 Kc6-c7 Kf6-e7 Bc8-g4 Ke7-f6 Bg4-h3 Kf6-e7 b3-b4 Ba5xb4 a3xb4 c5xb4
 34/39	00:50	 119.444k	2.368k	+1,35	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f8 Kc6-b7 Kf8-f7 a2-a3 Kf7-f6 Bf5-h3 Kf6-e7 Kb7-c7 h5-h4 Bh3-f5 Ke7-f6 Bf5-c8 Kf6-e7 Bc8-g4 Ke7-f6 Bg4-d7 Kf6-e7 Bd7-h3 Ke7-f6 Bh3-g4 Kf6-f7 b3-b4 Ba5xb4 a3xb4 c5-c4 Bg4-f5 c4-c3 Bf5-e4 h4-h3
 35/39	01:06	 160.503k	2.418k	+1,31	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f8 a2-a3 Kf8-e7 Bf5-h3 h5-h4 Kc6-c7 Ke7-f6 Bh3-g4 Kf6-f7 Bg4-f5 Kf7-f6 Bf5-h3 Kf6-e7 Kc7-c6 Ke7-f7 Bh3-g4 Kf7-f6 Kc6-b7 Kf6-f7 Kb7-c7 Kf7-f6 Kc7-c6 Kf6-f7 Kc6-b7 Kf7-f6 Kb7-a6 Kf6-f7 b3-b4 Ba5xb4 a3xb4 c5-c4
 36/40+	01:07	 162.652k	2.423k	+1,39	Ke8-d7
 36/40-	01:12	 175.431k	2.435k	+1,31	Ke8-d7 Bg5-d2
 36/40	01:15	 185.403k	2.441k	+1,31	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f8 a2-a3 Kf8-e7 Kc6-c7 h5-h4 Bf5-h3 Ke7-f7 Bh3-c8 Kf7-f6 Bc8-g4 Kf6-f7 Kc7-c6 Kf7-g6 Bg4-d7 Kg6-f6 Bd7-c8 Kf6-e7 Bc8-f5 Ke7-f6 Bf5-d7 Kf6-g6 Kc6-b7 Kg6-g7 Bd7-h3 Kg7-f7 Kb7-c7 Kf7-e7 b3-b4 Ba5xb4 a3xb4
 37/40	01:42	 256.685k	2.495k	+1,30	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f7 a2-a3 Kf7-f6 Bf5-h3 h5-h4 Bh3-d7 Kf6-f7 Kc6-c7 Kf7-e7 Bd7-c8 Ke7-f6 Bc8-g4 Kf6-f7 Bg4-f5 Kf7-e7 Bf5-h3 Ke7-f7 Bh3-g4 Kf7-f6 Bg4-c8 Kf6-e7 Kc7-c6 Ke7-f6 Bc8-h3 Kf6-e7 b3-b4 c5xb4 a3xb4 Ba5xb4 Kc6xb6 Ke7-d6
 38/40	01:53	 285.108k	2.506k	+1,27	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c7 Bd2-a5 f3-f4 Kg8-f7 Bf5-d7 Kf7-e7 Bd7-h3 h5-h4 a2-a3 Ke7-f6 Bh3-c8 Kf6-e7 Kc7-c6 Ke7-f6 Bc8-h3 Kf6-e7 Bh3-g4 Ke7-f6 Bg4-c8 Kf6-e7 Kc6-c7 Ke7-f6 Kc7-b7 Kf6-e7 Bc8-g4 Ke7-f6 Kb7-c6 Kf6-e7 Bg4-f5 Ke7-f7 b3-b4 c5xb4 a3xb4 Ba5xb4 Kc6xb6
 39/40	02:15	 340.814k	2.522k	+1,25	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f7 Bf5-h3 Kf7-e7 Bh3-g2 Ke7-f6 Bg2-f3 h5-h4 Bf3-g4 Kf6-e7 Kc6-c7 Ke7-f7 Bg4-c8 Kf7-f6 Bc8-h3 Kf6-f7 Bh3-g4 Kf7-f6 Bg4-d7 Kf6-g7 Kc7-c6 Kg7-f6 Bd7-c8 Kf6-g6 Bc8-g4 Kg6-f6 Kc6-b7 Kf6-f7 Bg4-h3 Kf7-f6 Bh3-c8 Kf6-g6 Kb7-c6
 40/40	02:26	 370.469k	2.524k	+1,25	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f7 Bf5-h3 Kf7-e7 Bh3-g2 Ke7-f6 Bg2-f3 h5-h4 Bf3-g4 Kf6-e7 Kc6-c7 Ke7-f7 a2-a3 Kf7-e7 Kc7-c6 Ke7-f6 Bg4-h3 Kf6-e7 Kc6-c7 Ke7-f6 Bh3-c8 Kf6-e7 Bc8-g4 Ke7-f7 Kc7-b7 Kf7-f6 Bg4-h3 Kf6-f7 Bh3-c8 Kf7-e7 Kb7-c6 Ke7-f6 Bc8-g4 Kf6-e7
 41/45	03:04	 466.712k	2.526k	+1,24	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f7 Bf5-h3 Kf7-e7 Bh3-g2 Ke7-f6 Bg2-f3 h5-h4 Bf3-g4 Kf6-f7 Bg4-c8 Kf7-g6 a2-a3 Kg6-f6 Bc8-g4 Kf6-e7 Bg4-h3 Ke7-f6 Bh3-d7 Kf6-f7 Kc6-c7 Kf7-e7 Bd7-g4 Ke7-f7 Bg4-c8 Kf7-f6 Bc8-h3 Kf6-e7 Kc7-b7 Ke7-d6 Bh3-g4 Kd6-e7 Kb7-a7 Ke7-d6 Ka7-a6 Kd6-d5 f4-f5
 42/45	03:29	 528.614k	2.521k	+1,24	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f7 Bf5-h3 Kf7-e7 Bh3-g2 Ke7-f6 Bg2-f3 h5-h4 Bf3-g4 Kf6-f7 a2-a3 Kf7-f6 Bg4-d7 Kf6-f7 Kc6-c7 Kf7-e7 Bd7-h3 Ke7-f6 Bh3-c8 Kf6-e7 Kc7-c6 Ke7-f6 Bc8-g4 Kf6-e7 Kc6-c7 Ke7-f7 Kc7-b7 Kf7-e7 Bg4-c8 Ke7-d6 Bc8-h3 c5-c4 b3-b4 Ba5xb4 a3xb4 Kd6-e7 Kb7xb6
 43/45	04:24	 665.817k	2.513k	+1,24	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f7 Bf5-h3 Kf7-e7 Bh3-g2 Ke7-f6 Bg2-f3 h5-h4 Bf3-g4 Kf6-f7 a2-a3 Kf7-f6 Kc6-c7 Kf6-f7 Bg4-d7 Kf7-e7 Bd7-f5 Ke7-f6 Bf5-c8 Kf6-e7 Bc8-h3 Ke7-f6 Kc7-b7 Kf6-f7 Kb7-c6 Kf7-f6 Bh3-c8 Kf6-f7 Bc8-d7 Kf7-e7 Bd7-f5 Ke7-f6 Bf5-g4 Kf6-e7 b3-b4 Ba5xb4 a3xb4 c5-c4 Kc6xb6
 44/46	06:17	 942.943k	2.499k	+1,24	Ke8-d7 Bg5-d2 Bc2-f5 h6-h5 Kd7-c6 Bd2-a5 f3-f4 Kg8-f7 Bf5-h3 Kf7-e7 Kc6-c7 h5-h4 Bh3-c8 Ke7-f6 Kc7-c6 Kf6-g6 Bc8-h3 Kg6-f7 Bh3-g4 Kf7-f6 a2-a3 Kf6-e7 Bg4-h3 Ke7-f7 Bh3-d7 Kf7-e7 Bd7-g4 Ke7-f6 Kc6-c7 Kf6-f7 Bg4-d7 Kf7-e7 Bd7-h3 Ke7-f6 Bh3-c8 Kf6-e7 Kc7-c6 Ke7-f7 b3-b4 Ba5xb4 a3xb4 c5-c4 Kc6xb6 c4-c3 Bc8-f5
 45/52	34:05	 5.366.587k	2.624k	+1,25	Ke8-d7 Bg5-d2 Kd7-e6 h6-h5 Ke6-f5 Bd2-e1 Kf5-g5 Be1xf2 Kg5xh5 Kg8-f7 Kh5-g5 Kf7-e6 f3-f4 Bf2-e3 Bc2-e4 Be3-d2 Be4-g2 Bd2-e3 Bg2-c6 Ke6-d6 Bc6-e4 Kd6-d7 Kg5-f5 Kd7-d6 a2-a4 Be3-d2 Be4-d3 Kd6-e7 Kf5-g5 Ke7-e6 Bd3-e4 Ke6-e7 Be4-f3 Ke7-e6 Bf3-g2 Ke6-d7 Kg5-f5 Kd7-e7 Bg2-f3 Ke7-d6 Bf3-e2 Kd6-d7 Be2-f1 Kd7-e7 Bf1-g2 Ke7-f7
 46/52	36:54	 5.800.321k	2.620k	+1,25	Ke8-d7 Bg5-d2 Kd7-e6 h6-h5 Ke6-f5 Bd2-e1 Kf5-g5 Be1xf2 Kg5xh5 Kg8-f7 Kh5-g5 Kf7-e6 f3-f4 Bf2-e3 Bc2-e4 Be3-d2 Be4-g2 Bd2-e3 Bg2-f3 Be3-c1 a2-a4 Bc1-d2 Bf3-g2 Ke6-f7 Kg5-g4 Kf7-g6 Bg2-e4+ Kg6-f6 Be4-d5 Bd2-c3 Bd5-b7 Kf6-e6 Kg4-g5 Bc3-d2 Bb7-f3 Ke6-d6 Kg5-f5 Kd6-e7 Bf3-e4 Ke7-d6 Be4-d3 Kd6-e7 Kf5-e5 Ke7-f7 Ke5-d5 Kf7-f6 Kd5-c6 Bd2-a5
 47/52	39:30	 6.211.921k	2.621k	+1,25	Ke8-d7 Bg5-d2 Kd7-e6 h6-h5 Ke6-f5 Bd2-e1 Kf5-g5 Be1xf2 Kg5xh5 Kg8-f7 Kh5-g5 Kf7-e6 f3-f4 Bf2-e3 Bc2-e4 Be3-d2 Be4-g2 Bd2-e3 a2-a4 Be3-d2 Kg5-g4 Ke6-d6 Bg2-e4 Kd6-e7 Kg4-f5 Ke7-f7 Be4-d5+ Kf7-e7 Bd5-f3 Ke7-d7 Kf5-e5 Kd7-e7 Bf3-h5 Bd2-c3+ Ke5-d5 Ke7-f6 Kd5-c6 Bc3-a5 Bh5-g4 Kf6-f7 Bg4-f5 Kf7-f6 Bf5-d3 Kf6-e6 Bd3-h7 Ke6-f6 Bh7-b1 Kf6-e6 Bb1-e4 Ke6-f6
 48/59+	46:04	 7.239.959k	2.618k	+1,32	f3-f4
 48/59+	46:26	 7.295.348k	2.618k	+1,39	f3-f4
 48/59+	58:04	 9.013.761k	2.587k	+5,44	f3-f4
 48/59-	1:03:17	 9.774.235k	2.574k	+4,60	f3-f4 Bg5-f6
 48/59+	1:04:23	 9.930.272k	2.570k	+5,66	f3-f4
 48/66	2:08:11	 20.370.680k	2.648k	+10,39	f3-f4 
So it took 46:30 min to find f4 on a small labtop running on 4 threads with 2 GB hash.
Kind regards
Bernhard
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: "Poor Fish": very hard test suite

Post by mcostalba »

Here are the result of my final test DBT 480+480seconds on 3 cores 1024 MB of hash.

Test has been performed on hard2016 and 100brillante testsuites:

Code: Select all

1b6/2p5/1kB5/1PN5/8/3K4/8/q7 w - - bm Nb3; id "hard2016.1";
1r3rk1/3bbppp/1qn2P2/p2pP1P1/3P4/2PB1N2/6K1/qNBQ1R2 w - - bm Bxh7+; id "arasan19.96"; c0 "nick-Fritz 11, playchess.com 2011";
1r6/4k3/r2p2p1/2pR1p1p/2P1pP1P/pPK1P1P1/P7/1B6 b - - 0 48 bm Rxb3+; id "hard2016.10"; Brzozka-Bronstein, Miskolc 1963;
2b1r3/r2ppN2/8/1p1p1k2/pP1P4/2P3R1/PP3PP1/2K5 w - - bm Nd6+; id "hard2016.19";
2r1r1k1/4bppp/1q4n1/3pP3/p1pP4/PnR2NBP/1P4P1/1B1QR2K w - - bm e6; id "W.Eigenmann: Brillant 057";c2 "Gaprindashvili-Bliss, CorrGame 1994";
3k4/4pp2/n2n4/5BN1/3N4/b7/4K3/8 w - - bm Nxf7+; id "hard2016.40";
4k1br/1K1p1n1r/2p2pN1/P2p1N2/2P3pP/5B2/P2P4/8 w - - bm Kc8; id "hard2016.49";
4q1kr/p6p/1prQPppB/4n3/4P3/2P5/PP2B2P/R5K1 w - - 1 24 bm Qxe5; id "gusev"; c0 "Gusev-Auerbach, Chelyabinsk, 1946";
6b1/5p2/1p3Bp1/1P3pP1/5P2/pp1B2K1/pr4P1/k7 w - - bm Kh2; id "hard2016.65";
7q/b1p5/1p1Npkb1/pPP2ppP/P1P5/3B2P1/5P1R/K3R3 w - - bm hxg6; id "hard2016.74";
8/1p2b3/6k1/5p2/p2NbP1p/P1B1P1p1/1P5P/6K1 w - - 0 47 bm h3; id "hard2016.78"; Smirin-HIARCS 8, 2002;
8/3P3k/n2K3p/2p3n1/1b4N1/2p1p1P1/8/3B4 w - - bm Nf6+; id "hard2016.82"; Study by Van Breukelen";
N7/8/2KQ2rp/6k1/3p3p/2p4P/4PP2/5N2 w - - bm f4+; c1 "diff=8";
r1b2rk1/2q1bppp/p2pp3/2n3PQ/1p1BP3/1BN5/PPP2P1P/2KR2R1 w - - 0 16 bm Bf6; id "hard2016.108"; Hennings-Moehring, Colditz 1967;
r2qrbk1/1b3p2/3p2pB/1p1P1N2/1np1P3/5NRP/1P3PP1/1B1Qn1K1 w - - bm N3h4; id "hard2016.129;
r5k1/p1pb2bp/3p3r/P1pPp1p1/2B1Pq2/1R2QPp1/1P4PP/5RBK b - - 3 28 bm Rxh2+; id "hard2016.144"; Summerscale-Jones, Torquay 2009;
r6r/4k3/p2bP2p/2n2PP1/2BQn2P/1pP1p3/1q1BK3/3R3R b - - bm Qc2; id "hard2016.148";


8/3P3k/n2K3p/2p1n3/1b4N1/2p1p1P1/8/3B4 w - - bm Nf6+; id "W.Eigenmann: Brillant 021";
8/8/8/1pR1P1K1/p2P2P1/rp1B1P2/pk6/b7 w - - bm Bh7; id "W.Eigenmann: Brillant 042";
4rrk1/7p/p2b4/1p1p1ppq/3P1P2/2P1B1Pb/PP1N1Q1P/R3R1K1 b - - bm Re4; id "W.Eigenmann: Brillant 046";
N7/8/2KQ2rp/6k1/3p3p/2p4P/4PP2/5N2 w - - bm f4+; id "W.Eigenmann: Brillant 052";
1rbr2k1/4q1p1/p2bp2p/8/R6P/3QB1P1/4PPB1/2R3K1 w - - bm Bxh6; id "W.Eigenmann: Brillant 064";
6q1/2N3p1/8/k7/8/1p1p4/1P1Pp1Q1/1n2K3 w - - bm Qc6; id "W.Eigenmann: Brillant 078";
1rb1nrk1/1p1qp2p/p1np1ppQ/3N1P2/2P1P3/3B1N1P/P2B2P1/R6K w - - bm e5; id "W.Eigenmann: Brillant 079";
7K/3p4/8/3p1B2/1BpP4/8/ppp1p3/qk1bR3 w - - bm Bf8; id "W.Eigenmann: Brillant 086";
4bkrb/1K2p1r1/4P2B/8/8/8/8/7R w - - bm Ra1; id "W.Eigenmann: Brillant 088";
1n6/Pp1p1p1p/1P1P1P1P/4K3/2p5/p7/rpp1P3/qkb1R3 w - - bm axb8=N; id "W.Eigenmann: Brillant 100";
These should be the most difficult positions and the only ones to survive such hard test. Just to give you an idea, the full Arasan suite (minus one) did not pass it! The only Arasan position that passed is this one (already included in the hard2016):

Code: Select all

1r3rk1/3bbppp/1qn2P2/p2pP1P1/3P4/2PB1N2/6K1/qNBQ1R2 w - - bm Bxh7+; id "arasan19.96"; c0 "nick-Fritz 11, playchess.com 2011";
In the next weeks I will update the suite here

http://poorfish.bitballoon.com/

Adding the above positions, if I am able to find the pgn and sources (I have already of some of them, thanks to Vincent), I will ask here for the missing ones...
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: "Poor Fish": very hard test suite

Post by mcostalba »

mcostalba wrote: In the next weeks I will update the suite here

http://poorfish.bitballoon.com/
I have started a huge effort to DBT test a very large epd collection.

I have written a Python script to automate the DBT testing, so that I can leave it running overnight without supervision. So far so well.

This collection is quite old, so I expected to find very few successful positions, but with my surprised I found that many position survived the DBT test at 30 seconds (this seems small, but is already more then enough to filter out almost any position).

So I have doubts that perhaps these positions are false positives, i.e. are wrong.

Here are the positions of the first few collections that I found so far and that I suppose are wrong:

Code: Select all

5rk1/pbqn2pp/1p1bP3/8/3B4/1P5P/P2N1rP1/R2QRBK1 b - - bm Bh2+ Rxg2+; id "Emms.833";
5r2/3brp1R/p2p2p1/1p1Pp1k1/6P1/P2BKP2/1PP5/4R3 w - - bm Bxg6; id "Emms.835";
2q1rr1k/3bbnnp/p2p1pp1/2pPp3/PpP1P1P1/1P2BNNP/2BQ1PRK/7R b - - bm f5; id "position 03";
6k1/3b1p2/3pp1p1/2p1n3/2q1P2P/2PnQP1B/1r1B4/5NRK w - - bm Rg3; id "CCC-III No.6";
r1b1k2r/2qnbppp/pppp1n2/4p3/P1BPP3/2N2N2/1PP1QPPP/R1BR2K1 w kq - bm dxe5; id "CCC-III No.11";
r3k2r/p2n1pp1/2p1pn1p/1p5P/2PP4/5N2/PP1BRPP1/2K4R w kq - bm b3; id "CCC-III No.15";
r1bqrbk1/2n4p/3p1pp1/pppP3n/4P3/P1N2NPP/1P1B1PB1/R2QR1K1 w - - bm Nh2; id "CCC-III No.16";
r1bqr1k1/3nbppp/p2p4/3N1PP1/1p1B1P2/1P6/1PP1Q2P/2KR2R1 b - - bm Bf8; id "CCC-III No.18";
r1bb1rk1/pp3p1p/7q/3N2p1/Q2N2nP/6P1/PP2PPB1/R4RK1 w - - bm Rfc1; id "CCC-III No.25";
2rqkb1r/3n1p1p/p3p1pn/1p1pP1N1/5P2/2N5/PPP3PP/R1BQ1R1K w kq - bm a4; id "CCC-III No.45";
2n5/1k6/4pP2/4P3/2K5/7p/6P1/8 w - - bm f7; id "MES.100";
8/8/7p/3KNN1k/2p4p/8/3P2p1/8 w - - bm Kc6; id "MES.112";
8/2P2n1B/8/5K2/8/3p4/3Pkp2/8 w - - bm Ke6; id "MES.250";
B1k5/5n1P/3P3P/4p2K/4p3/4p3/8/8 w - - bm Bb7+; id "MES.253";
nK6/2p3pk/1pp5/1P3NP1/5P2/1P4P1/p1B5/8 w - - bm Ne3+; id "MES.361";
b6k/7p/2p5/8/8/2N5/3B4/1K6 w - - bm Bh6; id "MES.367";
8/8/7k/8/7P/2p5/4KN2/3B2b1 w - - bm Nd3; id "MES.368";
7B/1k6/7P/8/1K6/6n1/b2PPn2/6b1 w - - bm h7; id "MES.421";
8/6k1/4K3/8/3R4/pp6/8/8 w - - bm Rd7+; id "MES.452";
5K2/2k2P2/7R/8/8/5r1p/8/8 w - - bm Kg8; id "MES.477";
4r1RR/kp6/1p5P/1P6/K7/7P/8/8 w - - bm Rg2; id "MES.527";
R7/p7/5p2/2N5/1K5p/1p5p/3k4/8 w - - bm Ka3; id "MES.571";
8/1p6/4Pp2/8/r2N4/8/5N2/1k3K2 w - - bm Ne4; id "MES.601";
5k2/8/8/3R4/6K1/8/3b2pP/8 w - - bm Rf5+; id "MES.676";
4r2k/4P2p/8/3pP3/3P1K2/8/R7/6b1 w - - bm Rg2; id "MES.729";
8/2P5/p4k2/2r4p/3pB3/3P3K/rP1P4/7R w - - bm Bd5; id "MES.784";
8/8/8/1B6/6p1/8/4KPpp/3N2kr w - - bm Kd3; id "MES.831";
8/8/8/3B2K1/8/8/5p1p/2k2Nbr w - - bm Kf6; id "MES.869";
1r3k2/7p/1P1K3P/8/1P6/8/2RN1p2/6b1 w - - bm Rc8+; id "MES.903";
3B4/1r2p3/r2p1p2/bkp1P1p1/1p1P1PPp/p1P1K2P/PPB5/8 w - - bm Ba4+; id "MES.941";
7r/1P5B/k2Pr3/1n1K4/8/2R5/8/8 w - - bm b8=N+; id "MES.946";
3K2B1/8/6R1/P1n5/8/r7/N7/7k w - - bm a6; id "MES.961";
bN2N3/7k/1Pr1rp2/2P3P1/8/5K2/B7/8 w - - bm b7; id "MES.969";
8/4p2K/pp6/5k2/3bp2P/P4N2/1P4QP/3q4 w - - bm Qe2; id "MES.1069";
N3kb2/pq4p1/4B1P1/1p1P2B1/p7/3P4/2P5/2K5 w - - bm Bf7+; id "MES.1091";
4kr2/5p1K/3p1Q2/1p4P1/4P3/1PP5/7b/8 w - - bm Kh6; id "MES.1159";
8/4q3/5p2/B1N1b2p/P6k/P4P1p/8/4R2K w - - bm Re4+; id "MES.1203";
8/K6N/3k2P1/8/2P5/8/2N5/5r2 w - - bm c5+; id "MES.1254";
Can someone confirm these positions?
Vinvin
Posts: 5228
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: "Poor Fish": very hard test suite

Post by Vinvin »

mcostalba wrote:
mcostalba wrote: In the next weeks I will update the suite here

http://poorfish.bitballoon.com/
I have started a huge effort to DBT test a very large epd collection.

I have written a Python script to automate the DBT testing, so that I can leave it running overnight without supervision. So far so well.

This collection is quite old, so I expected to find very few successful positions, but with my surprised I found that many position survived the DBT test at 30 seconds (this seems small, but is already more then enough to filter out almost any position).

So I have doubts that perhaps these positions are false positives, i.e. are wrong.

Here are the positions of the first few collections that I found so far and that I suppose are wrong:

Code: Select all

5rk1/pbqn2pp/1p1bP3/8/3B4/1P5P/P2N1rP1/R2QRBK1 b - - bm Bh2+ Rxg2+; id "Emms.833";
5r2/3brp1R/p2p2p1/1p1Pp1k1/6P1/P2BKP2/1PP5/4R3 w - - bm Bxg6; id "Emms.835";
2q1rr1k/3bbnnp/p2p1pp1/2pPp3/PpP1P1P1/1P2BNNP/2BQ1PRK/7R b - - bm f5; id "position 03";
6k1/3b1p2/3pp1p1/2p1n3/2q1P2P/2PnQP1B/1r1B4/5NRK w - - bm Rg3; id "CCC-III No.6";
r1b1k2r/2qnbppp/pppp1n2/4p3/P1BPP3/2N2N2/1PP1QPPP/R1BR2K1 w kq - bm dxe5; id "CCC-III No.11";
r3k2r/p2n1pp1/2p1pn1p/1p5P/2PP4/5N2/PP1BRPP1/2K4R w kq - bm b3; id "CCC-III No.15";
r1bqrbk1/2n4p/3p1pp1/pppP3n/4P3/P1N2NPP/1P1B1PB1/R2QR1K1 w - - bm Nh2; id "CCC-III No.16";
r1bqr1k1/3nbppp/p2p4/3N1PP1/1p1B1P2/1P6/1PP1Q2P/2KR2R1 b - - bm Bf8; id "CCC-III No.18";
r1bb1rk1/pp3p1p/7q/3N2p1/Q2N2nP/6P1/PP2PPB1/R4RK1 w - - bm Rfc1; id "CCC-III No.25";
2rqkb1r/3n1p1p/p3p1pn/1p1pP1N1/5P2/2N5/PPP3PP/R1BQ1R1K w kq - bm a4; id "CCC-III No.45";
2n5/1k6/4pP2/4P3/2K5/7p/6P1/8 w - - bm f7; id "MES.100";
8/8/7p/3KNN1k/2p4p/8/3P2p1/8 w - - bm Kc6; id "MES.112";
8/2P2n1B/8/5K2/8/3p4/3Pkp2/8 w - - bm Ke6; id "MES.250";
B1k5/5n1P/3P3P/4p2K/4p3/4p3/8/8 w - - bm Bb7+; id "MES.253";
nK6/2p3pk/1pp5/1P3NP1/5P2/1P4P1/p1B5/8 w - - bm Ne3+; id "MES.361";
b6k/7p/2p5/8/8/2N5/3B4/1K6 w - - bm Bh6; id "MES.367";
8/8/7k/8/7P/2p5/4KN2/3B2b1 w - - bm Nd3; id "MES.368";
7B/1k6/7P/8/1K6/6n1/b2PPn2/6b1 w - - bm h7; id "MES.421";
8/6k1/4K3/8/3R4/pp6/8/8 w - - bm Rd7+; id "MES.452";
5K2/2k2P2/7R/8/8/5r1p/8/8 w - - bm Kg8; id "MES.477";
4r1RR/kp6/1p5P/1P6/K7/7P/8/8 w - - bm Rg2; id "MES.527";
R7/p7/5p2/2N5/1K5p/1p5p/3k4/8 w - - bm Ka3; id "MES.571";
8/1p6/4Pp2/8/r2N4/8/5N2/1k3K2 w - - bm Ne4; id "MES.601";
5k2/8/8/3R4/6K1/8/3b2pP/8 w - - bm Rf5+; id "MES.676";
4r2k/4P2p/8/3pP3/3P1K2/8/R7/6b1 w - - bm Rg2; id "MES.729";
8/2P5/p4k2/2r4p/3pB3/3P3K/rP1P4/7R w - - bm Bd5; id "MES.784";
8/8/8/1B6/6p1/8/4KPpp/3N2kr w - - bm Kd3; id "MES.831";
8/8/8/3B2K1/8/8/5p1p/2k2Nbr w - - bm Kf6; id "MES.869";
1r3k2/7p/1P1K3P/8/1P6/8/2RN1p2/6b1 w - - bm Rc8+; id "MES.903";
3B4/1r2p3/r2p1p2/bkp1P1p1/1p1P1PPp/p1P1K2P/PPB5/8 w - - bm Ba4+; id "MES.941";
7r/1P5B/k2Pr3/1n1K4/8/2R5/8/8 w - - bm b8=N+; id "MES.946";
3K2B1/8/6R1/P1n5/8/r7/N7/7k w - - bm a6; id "MES.961";
bN2N3/7k/1Pr1rp2/2P3P1/8/5K2/B7/8 w - - bm b7; id "MES.969";
8/4p2K/pp6/5k2/3bp2P/P4N2/1P4QP/3q4 w - - bm Qe2; id "MES.1069";
N3kb2/pq4p1/4B1P1/1p1P2B1/p7/3P4/2P5/2K5 w - - bm Bf7+; id "MES.1091";
4kr2/5p1K/3p1Q2/1p4P1/4P3/1PP5/7b/8 w - - bm Kh6; id "MES.1159";
8/4q3/5p2/B1N1b2p/P6k/P4P1p/8/4R2K w - - bm Re4+; id "MES.1203";
8/K6N/3k2P1/8/2P5/8/2N5/5r2 w - - bm c5+; id "MES.1254";
Can someone confirm these positions?
I don't understand a lot of key moves in this list ...
Why did you selected this positions ??
example for this one, Ke6 is an obvious blunder : 8/2P2n1B/8/5K2/8/3p4/3Pkp2/8 w - - bm Ke6; id "MES.250";
jdart
Posts: 4366
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: "Poor Fish": very hard test suite

Post by jdart »

MES is junk. In fact, most endgame analysis that is pre-computer is junk.

--Jon
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: "Poor Fish": very hard test suite

Post by mcostalba »

Vinvin wrote: I don't understand a lot of key moves in this list ...
Why did you selected this positions ??
example for this one, Ke6 is an obvious blunder : 8/2P2n1B/8/5K2/8/3p4/3Pkp2/8 w - - bm Ke6; id "MES.250";
Stockfish does not understand them too!

These are positions where the suggested best move is different from the SF one, even after a deep search (it is 30 secs search but with DBT method, equivalent at a very deep search in standard mode).

I have found the collection here:

http://computer-chess.org/doku.php?id=c ... load:index


I have started to filter them all with DBT, to squeeze out the few good ones, but many of them, as reported above passed the DBT test, so I expect them to be wrong (I expect 1 out of 10000 to pass DBT test).
Jouni
Posts: 3279
Joined: Wed Mar 08, 2006 8:15 pm

Re: "Poor Fish": very hard test suite

Post by Jouni »

All testsuites before 2010 are junk for today programs! 99% solved instantly or are wrong. 1% of positions are usefull at most maybe. Good sign of progress actually :) .
Jouni
User avatar
MikeB
Posts: 4889
Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania

Re: "Poor Fish": very hard test suite

Post by MikeB »

mcostalba wrote:
tpoppins wrote:Or is this test supposed to be MultiPV=1-only?
Actually multi pv gives even more help to these already super strong engines, anyhow I have removed the position you found from the list and here is the final one (some of them are very difficult to find even with multi pv):

Code: Select all

rn1q1rk1/2pbb3/pn2p3/1p1pPpp1/3P4/1PNBBN2/P1P1Q1PP/R4R1K w - - bm Nxg5; id "arasan19.16"; c0 "Dann Corbit, CCC 2015";
r5n1/p1p1q2k/4b2p/3pB3/3PP1pP/8/PPPQ2P1/5RK1 w - - bm Qf4; id "arasan19.23"; c0 "Shulman-Marciano, Ubeda 1997";
rn1qr1k1/1p2bppp/p3p3/3pP3/P2P1B2/2RB1Q1P/1P3PP1/R5K1 w - - bm Bxh7+; id "arasan19.42"; c0 "Polugaevsky-Torre, London 1984";
1n3rk1/3rbppp/p2p4/4pP2/Ppq1P3/1N2B3/1PP3PP/R2Q1R1K w - - bm f6; id "arasan19.44"; c0 "Kuzmin-Groszpeter, Kusadasi 1990";
8/5p2/3p2p1/1bk4p/p2pBNnP/P5P1/1P3P2/4K3 b - - bm d3; id "arasan19.115"; c0 "Bondarevsky-Petrosian, Soviet ch-19 Moscow 1951";
8/4nk2/1p3p2/1r1p2pp/1P1R1N1P/6P1/3KPP2/8 w - - bm Nd3; id "arasan19.116"; c0 "Grandelius-Raznikov, EU Youth Chess Ch B18, Albena 2011";
3r1r1k/pp5p/4b1pb/6q1/3P4/4p1BP/PP2Q1PK/3RRB2 b - - bm Qxg3+; id "arasan19.137"; c0 "Apicella-Lautier, Clichy 2001";
r3k3/1p4p1/1Bb1Bp1p/P1p1bP1P/2Pp2P1/3P4/5K2/4R3 w - - bm g5; id "arasan19.154"; c0 "Nakamura-Giri, FIDE Grand Prix, London 2012";
1r1rb1k1/5ppp/4p3/1p1p3P/1q2P2Q/pN3P2/PPP4P/1K1R2R1 w - - bm Rxg7+; id "arasan19.155"; c0 "Vescovi-Gschwendtner, Groningen op 1994 (Aemis Test Suite #30)";
r1bq1rk1/pp2bppp/1n2p3/3pP3/8/2RBBN2/PP2QPPP/2R3K1 w - - bm Bxh7+; id "arasan19.157"; c0 "Trent-Deslandes, Cap d'Agde CCAS op 2008";
r1b2rk1/1pq1nppp/pbn1p3/8/3N4/3BBN2/PPP1QPPP/3R1RK1 w - - bm Bxh7+; id "arasan19.159"; c0 "Leu-Muck, DDR-ch H137 corr 1973";
8/4bBpp/3p4/P6P/2PN2p1/3k1b2/P7/6K1 w - - bm h6; id "arasan19.188"; c0 "Timman, 1c Corus {m} 2008";
r1b3r1/5p1k/p1n2P1p/P1qpp1P1/1p1p4/3P2Q1/BPPB2P1/R4RK1 w - - bm Kf2; id "arasan19.179"; c0 "Efremov-Fischer, Danube/4 ICCF 2012";
4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - - bm f4; id "arasan19.189"; c0 "Šachová skladba 2011-12, 1st Prize";
The last 2 are special, the last one perhaps or is not correct or is super hard, the one before the last:

[D]r1b3r1/5p1k/p1n2P1p/P1qpp1P1/1p1p4/3P2Q1/BPPB2P1/R4RK1 w - -

Is the one I like most :-)
this one , the last one , is just super hard...
some back and forth using large hash tables with 12 cores , and a modified stockfish with mpv==2

[d]4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - - bm f4

Code: Select all

info depth 41 seldepth 76 multipv 1 score cp 2196 nodes 12639422838 nps 35086604 hashfull 999 tbhits 13800868 time 360235 pv f3f4 g5f6 a2a4 f6c3 f4f5 g8g7 e8e7 c3f6 e7e6 f6c3 c2d1 g7f8 f5f6 c3a5 d1h5 a5c3 h5g6 c3e1 e6d5 h6h5 g6h5 e1a5 d5c6 f8g8 c6d6 g8f8 d6d7 a5c3 d7c6 c3d4 h5e2 f8f7 e2c4 f7e8 c6b6 d4f6 b6b7 f6c3 b7a7 e8d8 b5b6 c3d4 b6b7 d8d7 b7b8q d4b2 b8g8 b2a1 g8d5 d7e7 d5f7 e7d8 f7f8 d8c7 f8e7 c7c6 e7e8 c6c7 e8b8 c7d7 a4a5 a1d4 b8b7 d7e8
 info depth 41 seldepth 76 multipv 2 score cp 239 nodes 12639422838 nps 35086604 hashfull 999 tbhits 13800868 time 360235 pv c2d1 g5h4 a2a4 h4g5 a4a5 b6a5 d1e2 g8g7 b5b6 g5f4 e8d7 g7f6 d7c6 f6e7 b6b7 h6h5 c6c5 f4c7 e2d3 e7d8 d3g6 h5h4 g6f5 c7e5 c5b6 e5d4 b6c6 d4e5 f5h3 e5b8 c6b6 d8e7 f3f4 e7d6 h3f5 d6d5 f5g4 d5d6 g4h3 d6e7 h3f5 e7f8 f5e6 b8f4 b6a7 f8e7 e6g4 e7f6 b7b8q f4b8 a7b8 f6g5