A test position to test your NN

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Uri Blass
Posts: 11148
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

A test position to test your NN

Post by Uri Blass »

[fen]2r2bk1/p4r1p/5pp1/q7/1p1PP1PP/1P3Q2/BP1B4/1K2R3 w - - 0 31 [/fen]

White is totally losing but for some reason both stockfish and dragon suggest initially h5 with equal or positive score for white.

Note that chess engines that do not use NNUE usually do not have this problem and see that white is worse even at very small depth.
Top
Spliffjiffer
Posts: 436
Joined: Thu Aug 02, 2012 7:48 pm
Location: Germany

Re: A test position to test your NN

Post by Spliffjiffer »

didnt test other engines but here is the output of SugaR AI 2.40 avx2 "Peter-settings" ["use classical evaluation"=disabled(means pure NNUE used)..."materialistic evaluation strategy":-12..."positional evaluation strategy":12...8192mb hash, no TB's...NNUE:nn-13406b1dcbe0] on a ryzen 3600, 12 threads used:

Code: Select all

FEN: 2r2bk1/p4r1p/5pp1/q7/1p1PP1PP/1P1Q4/BP1B4/1K2R3 b - - 1 31

SugaR AI 2.40 avx2:
Binding thread 0 to group 0
Binding thread 1 to group 0
Binding thread 2 to group 0
Binding thread 3 to group 0
Binding thread 4 to group 0
Binding thread 5 to group 0
Binding thread 6 to group 0
Binding thread 7 to group 0
Binding thread 8 to group 0
Binding thread 9 to group 0
Binding thread 10 to group 0
Binding thread 11 to group 0
Found 165 tablebases
NNUE evaluation using C:\Program Files\Arena 3.5.1\Engines\NNUE\nn-13406b1dcbe0.nnue enabled
NNUE evaluation (using C:\Program Files\Arena 3.5.1\Engines\NNUE\nn-13406b1dcbe0.nnue) is enabled
 1/1	00:00	 492	246k	+3,40	31.h4-h5
 2/2	00:00	 868	434k	+6,12	31.h4-h5 g6xh5
 3/3	00:00	 2k	753k	+3,99	31.Te1-d1 a7-a6 32.h4-h5
 4/4	00:00	 5k	2.276k	+3,69	31.Te1-d1 g6-g5 32.Ld2-e1
 5/5	00:00	 5k	2.746k	+3,69	31.Te1-d1 g6-g5 32.Ld2-e1 g5xh4 33.Le1xh4
 6/6	00:00	 12k	3.865k	+13,03	31.Df3-d3 Tc8-d8 32.Dd3-c4
 7/7	00:00	 19k	6.411k	+3,39	31.Df3-e2 Tc8-d8 32.Ld2-e3 Da5-b6 33.h4-h5
 8/9	00:00	 49k	8.161k	+2,07	31.Te1-f1 Da5-b5 32.h4-h5 Tc8-d8 33.e4-e5 Td8xd4 34.Ld2-e3 Db5-d3+ 35.Kb1-a1
 9/11	00:00	 142k	10.148k	+1,81	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-d1 Db5-c6 34.Df3-d3 a7-a5
 10/16	00:00	 232k	11.070k	+1,34	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-d1 Db5-c6 34.Df3-d3 a7-a5 35.g4-g5 f6-f5 36.e4xf5 Tf7xf5
 11/17	00:00	 622k	11.305k	+1,09	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-h1 Tf7-h7 34.Th1-d1 Kg8-h8 35.e4-e5 f6xe5
 12/19	00:00	 1.589k	11.517k	+0,54	31.Df3-d3 Kg8-h8 32.Te1-f1 Tf7-d7 33.Ld2-e3 Tc8-e8 34.h4-h5 g6xh5 35.Tf1-f5 Da5-d8 36.Tf5xh5 a7-a5 37.Kb1-a1
 13/19	00:00	 2.026k	11.512k	+0,47	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-h1 Tf7-h7 34.Th1-f1 Th7-h2 35.Ld2-f4 Th2-c2 36.g4-g5 f6-f5 37.Kb1-a1 Lf8-g7
 14/22	00:00	 2.758k	11.541k	+0,44	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Ld2-f4 Db5-d7 34.Te1-d1 a7-a5 35.Lf4-g3 Dd7-c6 36.Df3-d3 Lf8-h6
 15/24	00:00	 2.965k	11.493k	+0,54	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Ld2-f4 Db5-d7 34.Te1-d1 Tc8-e8 35.Lf4-g3 a7-a5 36.Kb1-a1 Dd7-e6 37.La2-b1 Tf7-a7 38.d4-d5 De6-a6
 16/24	00:00	 5.601k	11.719k	+0,27	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-h1 Db5-d7 34.Kb1-a1 Tc8-e8 35.g4-g5 Dd7xd4 36.Ld2-e3 Dd4-e5 37.g5xf6 Tf7xf6 38.Df3-h3 Lf8-g7 39.Le3-h6
 17/26	00:00	 8.458k	11.539k	+0,42	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-h1 Db5-d7 34.Ld2-e3 Tc8-e8 35.Kb1-a1 Tf7-h7 36.Th1xh7 Kg8xh7 37.g4-g5 f6xg5 38.e4-e5 g5-g4 39.Df3-h1+ Kh7-g7
 18/31	00:01	 12.061k	11.541k	+0,06	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-h1 Db5-d7 34.Ld2-e3 Tc8-e8 35.Kb1-a1 Tf7-h7 36.Th1xh7 Kg8xh7 37.g4-g5 f6xg5 38.e4-e5 Lf8-h6 39.La2-b1 Te8-f8 40.Df3-e4 Dd7-e6 41.Lb1-d3 De6-f5 42.De4-b7+ Tf8-f7
 19/30	00:01	 14.193k	11.530k	+0,06	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Kb1-a1 Tc8-e8 34.Te1-h1 Tf7-h7 35.Th1-d1 Db5-b7 36.Td1-e1 Kg8-g7 37.g4-g5 f6-f5 38.d4-d5 Lf8-d6 39.e4xf5 Te8xe1+ 40.Ld2xe1 Th7-h2 41.Le1-g3 Ld6xg3
 20/26	00:01	 15.308k	11.501k	+0,18	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-h1 Db5-d7 34.Ld2-e3 Tf7-h7 35.Th1-f1 Dd7-e6 36.d4-d5 De6-d6 37.g4-g5 f6xg5 38.Le3xg5 Lf8-g7 39.Lg5-f4 Lg7-e5 40.Lf4xe5 Dd6xe5
 21/38	00:02	 25.184k	11.329k	+0,10	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Ld2-f4 Db5-d7 34.Lf4-e3 Dd7-c6 35.Kb1-a1 Tc8-e8 36.Le3-d2 Dc6-e6 37.Df3-h3 De6-d7 38.Dh3-d3 Dd7xg4 39.La2-b1 Tf7-h7 40.Ld2-e3 Lf8-h6 41.Te1-g1 Dg4-h5 42.Le3xh6 Th7xh6
 22/31	00:02	 30.706k	11.310k	 0,00	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-d1 Db5-d7 34.Ld2-f4 Tc8-e8 35.Kb1-a1 Dd7-e6 36.La2-b1 De6-a6+ 37.Lb1-a2
 23/14	00:02	 33.812k	11.274k	 0,00	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-d1 Db5-d7 34.Ld2-f4 Tc8-e8 35.Kb1-a1 Dd7-e6 36.La2-b1 De6-a6+ 37.Lb1-a2
 24/14	00:03	 36.189k	11.246k	 0,00	31.h4-h5 Da5-b5 32.h5xg6 h7xg6 33.Te1-d1 Db5-d7 34.Ld2-f4 Tc8-e8 35.Kb1-a1 Dd7-e6 36.La2-b1 De6-a6+ 37.Lb1-a2
 25/23-	00:03	 43.351k	11.124k	-0,08	31.h4-h5 Da5-b5
 25/23-	00:04	 48.663k	11.169k	-0,16	31.h4-h5 Da5-b5
 25/32-	00:07	 77.976k	11.017k	-0,71	31.h4-h5 Da5-b5
 25/36-	00:07	 86.028k	10.980k	-1,04	31.h4-h5 Tc8-c6
 25/42-	00:12	 133.655k	10.684k	-2,76	31.h4-h5 Tc8-c6
 25/42	00:16	 175.831k	10.699k	-2,97	31.Df3-d3 Tc8-c6 32.Kb1-a1 Tf7-d7 33.Ld2-e3 Tc6-a6 34.Dd3-b1 Kg8-f7 35.Te1-f1 Kf7-e8 36.h4-h5 Lf8-e7 37.h5xg6 h7xg6 38.Tf1-h1 Td7-c7 39.Le3-f4 Tc7-c2 40.e4-e5 f6xe5 41.d4xe5 Tc2-e2 42.Lf4-g3 Te2-g2 43.e5-e6 Da5xa2+ 44.Db1xa2 Ta6xa2+ 45.Ka1xa2 Tg2xg3
 26/42+	00:17	 182.132k	10.705k	-2,89	31.Te1-c1
 26/42-	00:17	 188.214k	10.712k	-3,05	31.Te1-c1 Tc8xc1+
 26/42-	00:21	 229.716k	10.785k	-3,36	31.Te1-c1 Tc8xc1+
 26/48	00:23	 250.389k	10.797k	-3,23	31.Te1-c1 Tc8xc1+ 32.Ld2xc1 Da5-b5 33.Kb1-a1 Tf7-c7 34.Df3-d1 Tc7-e7 35.Dd1-c2 Kg8-h8 36.g4-g5 f6xg5 37.h4xg5 Db5-f1 38.La2-b1 Df1-a6+ 39.Lb1-a2 Te7-e8 40.e4-e5 Te8-c8 41.Dc2-d1 Lf8-g7 42.Dd1-e1 Da6-d3 43.Lc1-e3 Tc8-f8 44.Le3-f2 Dd3-f5 45.Lf2-g3 Df5-f1 46.De1xf1 Tf8xf1+ 47.La2-b1 Tf1-g1 48.Lg3-h2 Tg1xg5 49.Lb1-d3 Tg5-h5 50.e5-e6 Lg7xd4
 27/41-	00:24	 265.055k	10.802k	-3,31	31.Te1-c1 Tc8xc1+
 27/41-	00:25	 276.131k	10.819k	-3,39	31.Te1-c1 Tc8xc1+
 27/41+	00:26	 287.946k	10.823k	-3,31	31.Kb1-a1
 27/41	00:27	 297.018k	10.831k	-3,16	31.Kb1-a1 Tc8-c6 32.Df3-d3 Tf7-d7 33.Ld2-e3 Kg8-f7 34.h4-h5 Tc6-a6 35.Dd3-b1 Kf7-e8 36.Te1-g1 Td7-e7 37.Le3-c1 Te7-f7 38.Lc1-e3 Ke8-d8 39.Tg1-f1 Tf7-d7 40.g4-g5 Td7-f7 41.Le3-d2 f6-f5 42.e4xf5 Tf7xf5 43.Tf1xf5 g6xf5 44.g5-g6 h7xg6 45.h5-h6 g6-g5 46.Ld2xg5+ Kd8-e8 47.h6-h7 Lf8-g7 48.Db1-e1+ Ke8-f7 49.h7-h8S+ Lg7xh8 50.De1-e7+ Kf7-g6
 28/36-	00:28	 305.179k	10.836k	-3,24	31.Kb1-a1 Tc8-c6
 28/38-	00:28	 311.984k	10.845k	-3,32	31.Kb1-a1 Tc8-c6
 28/40-	00:31	 347.688k	10.897k	-3,62	31.Kb1-a1 Tc8-c6
 26/42	00:31	 347.688k	10.897k	-3,58	31.Df3-d3 Tc8-c6 32.Kb1-a1 Tf7-d7 33.Ld2-e3 Tc6-a6 34.Dd3-b1 Kg8-f7 35.h4-h5 Kf7-e8 36.Te1-h1 Td7-c7 37.Th1-d1 Lf8-e7 38.Le3-f4 Tc7-c2 39.h5xg6 h7xg6 40.Lf4-d2 Ke8-d8 41.e4-e5 Tc2-c7 42.e5-e6 g6-g5 43.Td1-h1 Tc7-c6 44.Th1-h8+ Kd8-c7 45.Th8-h7 Tc6xe6
it needs around 3-4 sec to see that sth is wrong with white's position
Wahrheiten sind Illusionen von denen wir aber vergessen haben dass sie welche sind.
Top
amanjpro
Posts: 883
Joined: Sat Mar 13, 2021 1:47 am
Full name: Amanj Sherwany

Re: A test position to test your NN

Post by amanjpro »

Zahak (dev, NNUE):

Using: 10 Threads, 12G Hash
On: AMD Ryzen 7 4800H, 8 Cores, 16 Threads, 2.9GHz each

Needed little time to find d4d5, but 2 minutes to decide it is losing:

Code: Select all

setoption name Hash value 12000
setoption name Threads value 10
position fen 2r2bk1/p4r1p/5pp1/q7/1p1PP1PP/1P3Q2/BP1B4/1K2R3 w - - 0 31
go
info depth 1 seldepth 1 hashfull 0 tbhits 0 nodes 83 nps 486250 score cp 102 time 0 pv h4h5
info depth 2 seldepth 3 hashfull 0 tbhits 0 nodes 706 nps 236396 score cp 69 time 2 pv e1c1 c8c1 d2c1
info depth 4 seldepth 5 hashfull 0 tbhits 0 nodes 1979 nps 302293 score cp 94 time 6 pv e1c1 c8c1 d2c1 f6f5 g4f5
info depth 5 seldepth 5 hashfull 0 tbhits 0 nodes 10919 nps 861856 score cp 117 time 12 pv e1f1 c8c2 b1c2 a5a2 h4h5
info depth 6 seldepth 6 hashfull 0 tbhits 0 nodes 15624 nps 1087818 score cp 93 time 14 pv h4h5 a5b5 e1f1 a7a5 e4e5 f6f5
info depth 7 seldepth 7 hashfull 0 tbhits 0 nodes 27012 nps 2435728 score cp 68 time 11 pv e1f1 a5b5 h4h5 f8g7 h5g6 h7g6 g4g5
info depth 8 seldepth 8 hashfull 0 tbhits 0 nodes 30414 nps 1516264 score cp 87 time 20 pv h4h5 c8d8 d2e3 a5b5 h5g6 h7g6 e1h1 a7a5
info depth 9 seldepth 9 hashfull 0 tbhits 0 nodes 37202 nps 1459692 score cp 60 time 25 pv h4h5 a5b5 h5g6 h7g6 d2c1 f8d6 e1d1 g8g7 d1h1
info depth 10 seldepth 10 hashfull 0 tbhits 0 nodes 61447 nps 1227738 score cp 44 time 50 pv h4h5 a5b5 h5g6 h7g6 e1d1 f8g7 f3h3 f7d7 d2e3 a7a5
info depth 11 seldepth 13 hashfull 0 tbhits 0 nodes 76926 nps 1320056 score cp 51 time 58 pv h4h5 a5b5 h5g6 h7g6 e1h1 c8d8 d2e3 b5d3 b1a1 a7a5 h1d1 d3b5 d1c1
info depth 12 seldepth 13 hashfull 0 tbhits 0 nodes 162173 nps 1017461 score cp 43 time 159 pv h4h5 a5b5 h5g6 h7g6 e1d1 a7a5 d2e3 c8d8 e3c1 b5b6 d4d5 f7e7 c1e3
info depth 13 seldepth 15 hashfull 0 tbhits 0 nodes 211207 nps 1281708 score cp 49 time 164 pv h4h5 a5b5 h5g6 h7g6 e1d1 a7a5 d2e3 c8d8 b1a1 g8h8 d1h1 h8g8 a2b1 a5a4 b1a2
info depth 14 seldepth 14 hashfull 2 tbhits 0 nodes 442573 nps 1274318 score cp 38 time 347 pv h4h5 a5b5 h5g6 h7g6 e1d1 a7a5 d2e3 c8b8 e3c1 b8d8 c1e3 f8d6 e3c1 g8h8
info depth 15 seldepth 18 hashfull 10 tbhits 0 nodes 1594280 nps 1229682 score cp -14 time 1296 pv d4d5 f8d6 h4h5 d6e5 d2e3 c8c2 e1e2 c2e2 f3e2 g8g7 h5h6 g7h8 e2c4 f7c7 c4d3 c7d7 g4g5 f6g5
info depth 16 seldepth 16 hashfull 11 tbhits 0 nodes 2774313 nps 1977548 score cp -5 time 1402 pv d4d5 f8d6 h4h5 d6e5 d2e3 c8c2 e1e2 c2e2 f3e2 a5c7 e2a6 e5f4 e3d4 f4e5 h5g6 h7g6
info depth 17 seldepth 19 hashfull 18 tbhits 0 nodes 4631925 nps 2127779 score cp -1 time 2176 pv d4d5 f8d6 e1f1 d6e5 h4h5 c8c2 b1c2 a5a2 f3d3 a2b2 c2d1 g8g7 d5d6 f7d7 h5h6 g7h8 d2b4 a7a5 b4c5
info depth 18 seldepth 22 hashfull 24 tbhits 0 nodes 7450325 nps 2548172 score cp -42 time 2923 pv d4d5 f8d6 h4h5 d6e5 d2e3 a5b5 f3d1 f7c7 e1e2 b5d7 e2f2 g6h5 g4h5 d7h3 d5d6 h3e3 d6c7 e3e4 d1c2 e4c2 f2c2 e5c7
info depth 19 seldepth 21 hashfull 33 tbhits 0 nodes 11176692 nps 2931346 score cp -54 time 3812 pv d4d5 f8d6 f3h3 a5b5 h4h5 g6g5 h3f3 d6e5 e1d1 f7c7 d5d6 c7c2 d2c3 c8c3 b2c3 c2c3 f3f1 b5f1 d1f1 e5d6 f1f6
info depth 21 seldepth 25 hashfull 54 tbhits 0 nodes 17764121 nps 2952218 score cp -58 time 6017 pv d4d5 f8d6 d2e3 d6e5 f3d1 f7c7 e1e2 a5b5 h4h5 g6g5 h5h6 a7a5 b1a1 b5a6 a1b1 g8h8 e2f2 a6d6 d1e2 d6d7 f2f5 a5a4 d5d6 d7d6 b3a4
info depth 23 seldepth 27 hashfull 88 tbhits 0 nodes 27926061 nps 2914830 score cp -63 time 9580 pv f3d3 c8c6 b1a1 c6a6 d3b1 f7c7 d2e3 c7c2 g4g5 f6f5 h4h5 f5e4 h5g6 h7g6 e3f4 a6c6 e1e4 a5f5 b1h1 f8e7 a2b1 c6a6 b1a2 e7g5 e4e8 g8g7 f4g5
info depth 24 seldepth 32 hashfull 138 tbhits 0 nodes 43638707 nps 2930094 score cp -61 time 14893 pv d4d5 f8d6 f3h3 a5b5 g4g5 c8e8 g5f6 f7f6 h4h5 g6h5 b1a1 d6f4 h3h4 e8f8 d2f4 f6f4 h4h5 f4f1 h5g5 g8h8 g5e5 f8f6 e1b1 b5d3 d5d6 f1b1 a2b1 d3a6 b1a2 a6f1 a2b1 h8g7
info depth 25 seldepth 31 hashfull 239 tbhits 0 nodes 76505319 nps 2809581 score cp -89 time 27230 pv d4d5 f8d6 f3h3 a5b5 g4g5 c8e8 g5f6 f7f6 h4h5 g6h5 b1a1 g8h8 d2c1 e8e5 h3c8 b5e8 c8c4 h5h4 a2b1 h4h3 e1h1 f6f3 a1a2 a7a5 c4a6 e8f8 c1d2 e5e8 a6a5 h3h2 b1c2
info depth 27 seldepth 27 hashfull 331 tbhits 0 nodes 111625499 nps 2729334 score cp -91 time 40898 pv d4d5 f8d6 e1f1 d6e5 d2f4 a5c7 f1c1 c7b8 c1c8 b8c8 f4e3 c8a6 f3d1 g6g5 h4h5 g8g7 b1a1 e5f4 e3f4 g5f4 d1f3 a6d6 f3h3 g7h6 a2b1 d6e5 b1c2
info depth 28 seldepth 33 hashfull 417 tbhits 0 nodes 157620115 nps 2823724 score cp -81 time 55819 pv e1c1 c8c1 d2c1 f7d7 f3d3 g8g7 c1e3 d7d6 d3c4 d6a6 b1c2 f8d6 a2b1 a5c7 c2d3 a6a1 b1c2 d6f4 c4c7 f4c7 h4h5 a7a5 h5g6 h7g6 g4g5 f6f5 e4e5 g7f8 e3d2 f8e7 d3c4 e7e6 d2e3
info depth 30 seldepth 34 hashfull 656 tbhits 0 nodes 299175771 nps 2538493 score cp -120 time 117855 pv d4d5 f8d6 f3d3 d6e5 d2e3 f7d7 e1c1 c8c1 e3c1 d7d6 d3c4 a5b6 b1a1 d6d8 a2b1 a7a5 g4g5 g8g7 c4d3 f6g5 c1e3 b6c7 h4g5 d8f8 d3e2 g7g8 b1c2 f8c8 c2d3 e5f4 d3c4 c7e5 e3f4 e5f4
Top
Joerg Oster
Posts: 990
Joined: Fri Mar 10, 2006 4:29 pm
Location: Germany
Full name: Jörg Oster

Re: A test position to test your NN

Post by Joerg Oster »

Uri Blass wrote: ↑Fri Oct 22, 2021 9:50 am [fen]2r2bk1/p4r1p/5pp1/q7/1p1PP1PP/1P3Q2/BP1B4/1K2R3 w - - 0 31 [/fen]

White is totally losing but for some reason both stockfish and dragon suggest initially h5 with equal or positive score for white.

Note that chess engines that do not use NNUE usually do not have this problem and see that white is worse even at very small depth.
Very probably an effect of not training by the game outcome.
A shallow search can't see the position is bad for White.
Jörg Oster
Top
peter
Posts: 3520
Joined: Sat Feb 16, 2008 7:38 am
Full name: Peter Martan

Re: A test position to test your NN

Post by peter »

Spliffjiffer wrote: ↑Fri Oct 22, 2021 11:19 am didnt test other engines but here is the output of SugaR AI 2.40 avx2 "Peter-settings"
Too much honour, dear Peer!
:)
Here classic eval seems to work best as for SugaR, took single core for better discrimination of time to eval and better reproducibility:

Code: Select all

First trial with default classical and NNUE checked:

2r2bk1/p4r1p/5pp1/q7/1p1PP1PP/1P3Q2/BP1B4/1K2R3 w - - 0 1

Analysis by SugaR AI 2.40 avx2:

31.h5 Db5 32.Th1 g5 33.Td1 Td7 34.Le3 
  White is better: +/- (1.10)  Depth: 10/15   00:00:00
31.Tf1 Db5 32.h5 Tfc7 33.hxg6 Lg7 34.Dh3 hxg6 35.g5 f5 36.exf5 
  White is better: +/- (1.06)  Depth: 11/15   00:00:00
31.h5 Db5 32.hxg6 hxg6 33.Th1 Dc6 34.Dd3 Th7 35.Tc1 Dd7 36.Txc8 Dxc8 37.Dc4+ Dxc4 
  White is better: +/- (0.81)  Depth: 12/17   00:00:00
...
31.h5 Tc6 
  Black is clearly better: -+ (-2.21 --)  Depth: 26/34   00:00:36  55475kN
31.d5 Ld6 32.h5 Le5 33.Lf4 Db6 34.hxg6 hxg6 35.Td1 Dd6 36.Lxe5 fxe5 37.Dh3 Th7 38.Dd3 Kg7 39.Ka1 Th4 40.De3 Tf8 41.Dxa7+ Tf7 42.De3 Tf4 43.g5 Da6 
  Black has a decisive advantage: -+ (-2.59)  Depth: 26/36   00:00:42  63774kN

Classical only:

Analysis by SugaR AI 2.40 avx2:

31.d5 Ld6 32.Le3 Le5 33.Tc1 Txc1+ 34.Lxc1 
  Black is clearly better: -+ (-2.46)  Depth: 10/16   00:00:00
...
31.d5 Ld6 32.h5 Le5 33.Lf4 Dc7 34.Lxe5 Dxe5 35.Dd3 Tfc7 36.Dd2 Kg7 37.d6 Tc2 38.Dxc2 Txc2 39.Kxc2 Dxd6 40.Kb1 Dg3 41.Tc1 Kh6 42.Ka1 g5 43.Lb1 Df4 44.Tc4 Dxg4 45.Tc7 De6 
  Black has a decisive advantage: -+ (-3.24 --)  Depth: 25/38   00:00:15  32335kN

And NNUE only (pure):

Analysis by SugaR AI 2.40 avx2:

31.Dd3 Kg7 32.h5 Tfc7 33.hxg6 hxg6 
  White has a decisive advantage: +- (2.91)  Depth: 10/13   00:00:00
31.Dd3 Kg7 32.h5 gxh5 33.g5 fxg5 34.e5 Db6 35.Lxg5 Dg6 36.Lf6+ Txf6 37.exf6+ Kh8 
  White is better: +/- (1.04)  Depth: 11/15   00:00:00
31.h5 Db5 32.Tf1 Td8 33.Le3 Dd3+ 34.Ka1 Da6 35.hxg6 hxg6 
  White is clearly better: +/- (1.35)  Depth: 12/18   00:00:00
31.Lf4 Db5 32.Tf1 a5 33.h5 Dd7 34.hxg6 hxg6 35.Lg3 Dc6 
  White is better: +/- (0.76)  Depth: 13/20   00:00:00  259kN
31.h5 Db5 32.hxg6 hxg6 33.Lf4 Dd7 34.Td1 a5 35.Ka1 Te8 36.Lb1 g5 37.Lg3 Kg7 38.e5 fxe5 39.Lxe5+ Txe5 40.dxe5 Txf3 41.Txd7+ Kg8 
  White is slightly better: +/= (0.62)  Depth: 14/20   00:00:01  546kN
...
31.h5 Db5 32.hxg6 hxg6 33.Lf4 Dd7 34.Td1 Te8 35.Ka1 De6 36.Lb1 Da6+ 37.La2 De6 
  The position is equal: = (-0.16 --)  Depth: 25/27   00:00:15  10397kN
31.h5 Db5 32.hxg6 hxg6 33.Lf4 Dd7 34.Td1 Te8 35.Ka1 De6 36.Lb1 Da6+ 37.La2 De6 
  Black has an edge: = (-0.29 --)  Depth: 25/27   00:00:17  11894kN
31.h5 Db5 32.hxg6 hxg6 33.Lf4 Dd7 34.Td1 Te8 35.Ka1 De6 36.Lb1 Da6+ 37.La2 De6 
  Black is slightly better: =/+ (-0.47 --)  Depth: 25/35   00:00:20  14999kN
31.Lf4 
  Black is slightly better: =/+ (-0.32 ++)  Depth: 25/35   00:00:25  19281kN
31.Lf4 Db6 
  Black is better: -/+ (-0.71 --)  Depth: 25/35   00:00:31  23970kN
31.Lf4 Db5 
  Black is better: -/+ (-1.14 --)  Depth: 25/35   00:00:39  30126kN
31.Lf4 Tc6 
  Black is clearly better: -+ (-1.71 --)  Depth: 25/35   00:00:44  33768kN
31.Lf4 Tc6 
  Black is clearly better: -+ (-2.43 --)  Depth: 25/35   00:00:50  37941kN
With SF MOJO (Modification Jörg Oster) I guess that should be about the same, at this one branch there's the parameter of pure NNUE too.
Didn't try, because latest version doesn't support latest NNsUE, what a pity
(You are reading me, Jörg? :)) regards
Peter.
Top
User avatar
yurikvelo
Posts: 710
Joined: Sat Dec 06, 2014 1:53 pm

Re: A test position to test your NN

Post by yurikvelo »

Lc0 192x15 instantly gives -1.41, and very fast -2.11 d5
SF-Dev reach -0.70 @ 18 MN; -1.30 @ 30 MN; -1.77 @ 42 MN; -2.51 @ 115 MN, -3.39 @ 389 MN; -4.00 @ 2779 MN
Positive eval for white is only up to 2.4 MN
Last edited by yurikvelo on Fri Oct 22, 2021 7:10 pm, edited 1 time in total.
Top
amanjpro
Posts: 883
Joined: Sat Mar 13, 2021 1:47 am
Full name: Amanj Sherwany

Re: A test position to test your NN

Post by amanjpro »

Funny enough, Zahak 6.2 (not NNUE), has zero issue announcing that black is winning starting from depth 3 or something like this
Top
dkappe
Posts: 1632
Joined: Tue Aug 21, 2018 7:52 pm
Full name: Dietrich Kappe

Re: A test position to test your NN

Post by dkappe »

Looks like the NNUE engines mostly “get it” around depth 25. Now Dragon does train on game outcome (otherwise adversarial training wouldn’t make much sense) and uses different data, game generation and net architecture from Stockfish, yet both engines seem to have the same blind spot. I suspect that it has something to do with this long term trapped bishop. These are the types of positions that often give SF and Dragon trouble.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
Top
peter
Posts: 3520
Joined: Sat Feb 16, 2008 7:38 am
Full name: Peter Martan

Re: A test position to test your NN

Post by peter »

peter wrote: ↑Fri Oct 22, 2021 6:49 pm With SF MOJO (Modification Jörg Oster) I guess that should be about the same, at this one branch there's the parameter of pure NNUE too.
Didn't try, because latest version doesn't support latest NNsUE,
Yet of course I couldn't resist for long, this time with an even weaker CPU of 2GHz and single core again. Net of MOJO was nn-62ef826d1a6d.nnue way back then.

Code: Select all


2r2bk1/p4r1p/5pp1/q7/1p1PP1PP/1P3Q2/BP1B4/1K2R3 w - - 0 1

Analysis by Stockfish-MOJO 020521 Classical Eval first:

1.Tc1 Txc1+ 2.Lxc1 Dc7 3.h5 Dh2 4.hxg6 hxg6 5.Le3 
  -/+ (-1.50)  Depth: 10/15   00:00:00
...
1.Tc1 Txc1+ 2.Lxc1 Da6 3.Ld2 Td7 4.d5 Tc7 5.h5 g5 6.e5 fxe5 7.Df5 h6 8.Ka1 De2 9.Lb1 Dxd2 10.Dg6+ Tg7 11.De6+ Tf7 12.Lg6 Df4 13.Lxf7+ Dxf7 
  -+ (-2.71)  Depth: 18/28   00:00:02  1240kN
1.d5 Ld6 2.Lf4 Da6 3.Lxd6 Dxd6 4.h5 Tfc7 5.Dd1 gxh5 6.gxh5 Kh8 7.Tf1 De5 8.Dd2 Dxe4+ 9.Ka1 Tc2 10.Lb1 Txd2 11.Lxe4 
  -+ (-2.75)  Depth: 19/31   00:00:02  1659kN
...
1.d5 Ld6 2.Lf4 Da6 3.Lxd6 Dxd6 4.Df2 Tfc7 5.h5 De5 6.Dd2 a5 7.hxg6 hxg6 8.d6 Td7 9.Dh6 Dxd6 10.Dxg6+ Tg7 11.Dh6 Dd4 12.Dh5 
  -+ (-2.87)  Depth: 23/31   00:00:08  6904kN

Hybrid:

2r2bk1/p4r1p/5pp1/q7/1p1PP1PP/1P3Q2/BP1B4/1K2R3 w - - 0 1

Analysis by Stockfish-MOJO 020521:

1.h5 Db6 2.Le3 Da6 3.Tg1 g5 4.Df5 Tfc7 5.e5 
  +/- (1.48)  Depth: 10/15   00:00:00
1.h5 Db6 2.hxg6 hxg6 3.Le3 Da6 4.Ka1 Lg7 5.Td1 Td7 6.g5 fxg5 
  +/- (1.53)  Depth: 11/20   00:00:00
1.h5 Tfc7 2.Ka1 Lg7 3.hxg6 hxg6 4.g5 Tc2 5.Dg4 fxg5 6.Lxg5 
  +/- (1.34)  Depth: 12/20   00:00:00
...
1.h5 g5 2.h6 Db5 3.Th1 Tfc7 4.Ka1 Tf7 5.Kb1 
  = (0.00)  Depth: 16/25   00:00:01  946kN
1.d5 Db5 2.Tf1 Ld6 3.h5 Tcc7 4.hxg6 hxg6 5.Th1 Th7 6.Tf1 Le5 7.d6 Lxd6 8.Lc1 
  =/+ (-0.64)  Depth: 17/27   00:00:03  1742kN
...
1.d5 Ld6 2.h5 Dc7 3.Tc1 Dd7 4.Tc4 a5 5.Le3 Txc4 6.bxc4 a4 7.c5 b3 8.cxd6 Db5 9.d7 bxa2+ 10.Kxa2 Dxd7 11.hxg6 hxg6 
  -/+ (-1.21 --)  Depth: 19/33   00:00:04  2617kN
1.d5 Ld6 2.h5 Dc7 3.Tc1 Dd7 4.Tc4 a5 5.Le3 Txc4 6.bxc4 a4 7.c5 b3 8.cxd6 Db5 9.d7 bxa2+ 10.Kxa2 Dxd7 11.hxg6 hxg6 
  -/+ (-1.07 ++)  Depth: 19/33   00:00:04  2899kN
1.Dd3 Tc6 2.Ka1 Ta6 3.Db1 Kh8 4.Tc1 h5 5.gxh5 Dxh5 6.Tc8 Kg7 7.Le3 De2 8.Dc1 Ld6 9.Tc4 
  -/+ (-1.08)  Depth: 19/33   00:00:05  3237kN
...
1.Tc1 Txc1+ 2.Lxc1 Db5 3.Dd1 Kg7 4.h5 g5 5.Df3 Kg8 6.Ka1 h6 7.Ld2 Dd7 8.Le3 De6 9.Lb1 Dxb3 10.La2 Da4 11.Ld2 Kg7 
  -/+ (-1.31 ++)  Depth: 22/31   00:00:10  6641kN
1.Dd3 Tc6 2.Ka1 Ta6 3.Db1 Kh8 4.Le3 Te7 5.Ld2 Tc7 6.h5 gxh5 7.gxh5 Td7 8.h6 Txd4 9.Lf4 Dd8 10.Dc1 Ld6 11.e5 fxe5 12.Txe5 
  -/+ (-1.07)  Depth: 22/32   00:00:11  6978kN
...
1.Dd3 Tc6 2.Ka1 Ta6 3.Db1 Kh8 4.Le3 Te7 5.Lf4 Te8 6.g5 Lg7 7.gxf6 Txf6 8.Le3 Tf3 9.e5 Dd5 10.Dc2 Tef8 11.Dc4 Dxc4 12.bxc4 Tf1 13.Tb1 T8f3 14.Ld2 Lf8 15.c5 b3 16.e6 bxa2 17.Txf1 Txf1+ 18.Kxa2 
  -+ (-1.98)  Depth: 26/42   00:00:37  26315kN
...
1.Dd3 Tc6 2.Ka1 Ta6 3.Db1 Kh8 4.Lf4 Ld6 5.Lxd6 Txd6 6.d5 f5 7.exf5 Dxd5 8.fxg6 Txg6 9.Td1 Da5 10.g5 Ta6 11.Th1 Dd5 12.Dc1 Tc6 13.Dd1 Dxd1+ 14.Txd1 Kg7 
  -+ (-1.70 --)  Depth: 28/49   00:00:44  30993kN
1.Dd3 Tc6 2.Ka1 Ta6 3.Db1 Kh8 4.Lf4 Ld6 5.Lxd6 Txd6 6.d5 f5 7.gxf5 gxf5 8.h5 Dc5 9.Tc1 De3 10.exf5 Tf8 11.Td1 Df3 12.f6 Tfxf6 13.Dc1 Tf5 14.Lb1 Tf8 15.Dd2 Tf4 16.La2 Tg4 17.Te1 Dxd5 18.Te8+ Kg7 19.h6+ Kg6 
  -+ (-1.62)  Depth: 28/54   00:00:49  35095kN

And pure NNUE:

2r2bk1/p4r1p/5pp1/q7/1p1PP1PP/1P3Q2/BP1B4/1K2R3 w - - 0 1

Analysis by Stockfish-MOJO 020521:

1.h5 Db6 2.hxg6 hxg6 3.Td1 Th7 4.Lf4 
  +- (1.98)  Depth: 10/15   00:00:00
...
1.h5 g5 2.Dd3 Kh8 3.Df3 
  = (0.00)  Depth: 17/25   00:00:03  2018kN
1.Lf4 
  = (0.08 ++)  Depth: 18/21   00:00:03  2215kN
...
1.Lf4 Tc6 
  -/+ (-0.99 --)  Depth: 19/28   00:00:06  4185kN
1.h5 
  =/+ (-0.70 ++)  Depth: 19/28   00:00:06  4359kN
...
1.h5 g5 2.d5 Dc7 3.Tc1 Dd7 4.Td1 Ld6 5.Le3 Kg7 6.Ka1 a5 7.Ld4 Lf4 8.Lb6 a4 9.h6+ Kxh6 
  =/+ (-0.69 ++)  Depth: 20/30   00:00:07  4994kN
1.h5 g5 2.Ka1 Tc6 3.Dd1 h6 4.Le3 Kh8 5.Tf1 Ta6 6.Db1 Db5 7.Dd1 Ta5 8.Tf5 Da6 9.Txa5 Dxa5 10.Dd3 
  =/+ (-0.67)  Depth: 20/30   00:00:07  5089kN
1.h5 g5 2.Ka1 Tc6 3.Dd1 h6 4.Le3 Kh8 5.Tf1 Ta6 6.Db1 Db5 7.Dd1 Ta5 8.Tf5 Da6 9.Txa5 Dxa5 10.Dd3 
  -/+ (-0.75 --)  Depth: 21/27   00:00:08  5203kN
1.h5 g5 2.Ka1 Tc6 3.Dd1 h6 4.Le3 Kh8 5.Tf1 Ta6 6.Db1 Db5 7.Dd1 Ta5 8.Tf5 Da6 9.Txa5 Dxa5 10.Dd3 
  -/+ (-0.84 --)  Depth: 21/27   00:00:08  5318kN
1.h5 g5 2.Ka1 Tc6 3.Dd1 h6 4.Le3 Kh8 5.Tf1 Ta6 6.Db1 Db5 7.Dd1 Ta5 8.Tf5 Da6 9.Txa5 Dxa5 10.Dd3 
  -/+ (-1.00 --)  Depth: 21/27   00:00:08  5765kN
1.Tc1 
  -/+ (-0.88 ++)  Depth: 21/32   00:00:09  6407kN
1.Tc1 Txc1+ 2.Lxc1 Db6 3.Le3 Db5 4.Lc1 Dd7 5.Le3 g5 6.hxg5 fxg5 7.De2 Dd6 8.e5 Dc6 9.Ka1 Df3 10.De1 Df1 11.Db1 Tf3 12.Lxg5 Dxb1+ 13.Lxb1 
  -/+ (-1.07)  Depth: 21/35   00:00:10  6867kN
1.d5 
  -/+ (-0.99 ++)  Depth: 22/31   00:00:11  7654kN
1.d5 Ld6 
  -/+ (-1.15 --)  Depth: 22/31   00:00:11  7883kN
1.h5 
  -/+ (-1.02 ++)  Depth: 22/33   00:00:13  9108kN
1.h5 g5 
  -/+ (-1.33 --)  Depth: 22/33   00:00:14  9679kN
1.Dd3 Tc6 2.Ka1 Kh8 3.Lf4 Ta6 4.Db1 Td7 5.Le3 Te7 6.Ld2 Db5 7.h5 Td7 8.d5 Ld6 9.Tc1 Tc7 10.hxg6 Txc1 11.Dxc1 hxg6 12.e5 Lxe5 13.Dc8+ Kh7 
  -/+ (-1.08)  Depth: 22/36   00:00:16  10927kN
...
1.Dd3 Tc6 2.Ka1 Ta6 3.Db1 Kh8 4.Lf4 Td7 5.Le3 Te7 6.Lf4 Te8 7.h5 gxh5 8.gxh5 Dxh5 9.Dd3 Da5 10.Db1 Ld6 11.Ld2 Td8 12.Le3 Tg8 13.Th1 Tg7 14.Ld2 Tg2 15.Lh6 Tg4 16.Tc1 Tg8 17.Th1 Le7 
  -/+ (-1.49 ++)  Depth: 28/42   00:00:33  22858kN
...
1.Dd3 Tc6 2.Ka1 Ta6 3.Db1 Kh8 4.Lf4 Td7 5.d5 Ld6 6.Lxd6 Taxd6 7.h5 Kg7 8.Tc1 Ta6 9.Th1 g5 10.h6+ Kf8 11.Td1 Ke8 12.Tf1 Tad6 13.De1 Kd8 14.e5 Ta6 15.Db1 Dxd5 16.exf6 Dxb3 
  -+ (-1.70 --)  Depth: 30/46   00:00:57  39781kN
Problem with the position is, that 1.h5(?!) just loses as well as some other moves, so time to "solution" can be judged by longer output- line and eval only regards...

Edit: As expected latest SF dev. doesn't have any problems neither, again single core and without usage of NNUE:

Code: Select all

2r2bk1/p4r1p/5pp1/q7/1p1PP1PP/1P3Q2/BP1B4/1K2R3 w - - 0 1

Analysis by Stockfish 211021:

1.Tc1 Txc1+ 2.Lxc1 Da6 3.Le3 Te7 4.Ka1 
  -+ (-1.73)  Depth: 10/15   00:00:00
...
1.Tc1 Txc1+ 2.Lxc1 Da6 3.h5 g5 4.De3 h6 5.Df3 Td7 6.e5 Txd4 7.exf6 Dd3+ 8.Dxd3 Txd3 9.Kc2 
  -+ (-2.31)  Depth: 13/21   00:00:00  203kN
1.d5 Ld6 2.h5 Le5 3.hxg6 hxg6 4.Lf4 Dc5 5.Lxe5 fxe5 6.Dd3 a5 7.d6 Td7 
  -+ (-2.46)  Depth: 14/22   00:00:00  299kN
...
1.d5 Ld6 2.h5 Le5 3.Lf4 Dc7 4.Lxe5 Dxe5 5.De3 Tfc7 6.Dd2 a5 7.hxg6 hxg6 8.d6 Th7 9.Ka1 Td8 10.Td1 Th2 11.Dd5+ Dxd5 12.Txd5 Kf7 13.g5 fxg5 
  -+ (-3.15)  Depth: 23/40   00:00:13  14245kN
1.d5 Ld6 2.h5 Le5 3.Lf4 Dc7 4.Lxe5 Dxe5 5.De3 Tfc7 6.Dd2 a5 7.hxg6 hxg6 8.d6 Th7 9.Ka1 Td8 10.Td1 Th2 11.Dd5+ Dxd5 12.Txd5 Kf7 13.g5 fxg5 
  -+ (-3.07 ++)  Depth: 24/34   00:00:13  14310kN
1.d5 Ld6 2.h5 Le5 3.Lf4 Dc7 4.Lxe5 Dxe5 5.De3 Tfc7 6.Dd2 a5 7.hxg6 hxg6 8.d6 Th7 9.Ka1 Td8 10.Td1 Th2 11.Dd5+ Dxd5 12.Txd5 Kf7 13.g5 fxg5 
  -+ (-2.99 ++)  Depth: 24/34   00:00:13  15035kN
Peter.
Top

Return to “Computer Chess Club: General Topics”