Minic 3.18 NN endgame problem ... (14 games, *.zip)

xr_a_y · Post by **xr_a_y** » Sun Mar 13, 2022 11:00 am

The story continues...

Disabling TT reduces the issue but still nearly 5% of failed win

Score of minic_dev_dev vs stockfish-dev: 1101 - 1222 - 55 [0.475]
...      minic_dev_dev playing White: 540 - 625 - 25  [0.464] 1190
...      minic_dev_dev playing Black: 561 - 597 - 30  [0.485] 1188
...      White vs Black: 1137 - 1186 - 55  [0.490] 2378
Elo difference: -17.7 +/- 13.8, LOS: 0.6 %, DrawRatio: 2.3 %

So it's more complicated than just a TT issue

xr_a_y · Post by **xr_a_y** » Sun Mar 13, 2022 11:21 am

Disabling pruning is not helping...
Disabling pruning and TT is not helping...
I thought it may be an unfortunate interaction between PST and end-game tables, but no...

Interesting problem !

Frank Quisinsky · Post by **Frank Quisinsky** » Sun Mar 13, 2022 11:29 am

Hm ...
An own goal.

Now Minic will be soon the number 1 and the different to Wasp will be higher.

No, no ... all is OK!

Vivien on PC (weather is good).
Sorry for all that.

Best
Frank

RubiChess · Post by **RubiChess** » Sun Mar 13, 2022 4:52 pm

xr_a_y wrote: ↑Sun Mar 13, 2022 8:14 am I have generated using a python script around 20000 KNBvK endgame positions and run a match from here SF versus Minic at STC (1s+0.1) to evaluate this probability of failure (here to draw). On this specific end-game it is quite a lot (around 11% of the games that shall be won here)
Code: Select all
Score of minic_dev_dev vs stockfish-dev: 21450 - 24425 - 2686 [0.469]
...      minic_dev_dev playing White: 10692 - 12216 - 1373  [0.469] 24281
...      minic_dev_dev playing Black: 10758 - 12209 - 1313  [0.470] 24280
...      White vs Black: 22901 - 22974 - 2686  [0.499] 48561
Elo difference: -21.3 +/- 3.0, LOS: 0.0 %, DrawRatio: 5.5 %
and looses some 20Elo.

I'll analyze those games today if I have time

Do you have special eval code for this endgame? Without this code and with quite little time left it is no big surprise (at least for me) to miss the mate in quite a lot of these games.

Regards, Andreas

xr_a_y · Post by **xr_a_y** » Sun Mar 13, 2022 4:57 pm

RubiChess wrote: ↑Sun Mar 13, 2022 4:52 pm
xr_a_y wrote: ↑Sun Mar 13, 2022 8:14 am I have generated using a python script around 20000 KNBvK endgame positions and run a match from here SF versus Minic at STC (1s+0.1) to evaluate this probability of failure (here to draw). On this specific end-game it is quite a lot (around 11% of the games that shall be won here)
Code: Select all
Score of minic_dev_dev vs stockfish-dev: 21450 - 24425 - 2686 [0.469]
...      minic_dev_dev playing White: 10692 - 12216 - 1373  [0.469] 24281
...      minic_dev_dev playing Black: 10758 - 12209 - 1313  [0.470] 24280
...      White vs Black: 22901 - 22974 - 2686  [0.499] 48561
Elo difference: -21.3 +/- 3.0, LOS: 0.0 %, DrawRatio: 5.5 %
and looses some 20Elo.

I'll analyze those games today if I have time
Do you have special eval code for this endgame? Without this code and with quite little time left it is no big surprise (at least for me) to miss the mate in quite a lot of these games.

Regards, Andreas

Yes the "classic" pushToCorner + nearKings stuff.

The issue is more complex it seems. Not directly related to TT or prunings.
It seems not even reproductible even while clearing tables (TT, counter, ...) between each moves, even at fixed nodes TC... a bit worrying !

I'm spent some hours on that today but still can't find it. Interesting thing ...

xr_a_y · Post by **xr_a_y** » Sun Mar 13, 2022 5:07 pm

When wrong draw occurs, the mating path is not correct.

Sometimes the mate is too slow and 50moves rule is seen too late

Code: Select all

[White "stockfish-dev"]
[Black "minic_dev_dev"]
[Result "1/2-1/2"]

1. Ke3 {-1.99/13 0.055s} Kf7 {+37.69/17 0.056s} 2. Ke4 {-2.17/19 0.14s}
Ke6 {+37.72/20 0.079s} 3. Kd4 {-2.18/18 0.070s} Bb7 {+37.73/20 0.057s}
4. Kc4 {-2.24/16 0.026s} Ke5 {+37.73/21 0.063s} 5. Kd3 {-2.31/17 0.044s}
Kd5 {+37.79/20 0.12s} 6. Ke3 {-2.22/18 0.052s} Ng6 {+37.89/19 0.10s}
7. Kd3 {-2.26/15 0.056s} Ke5 {+37.89/20 0.11s} 8. Kc3 {-2.16/17 0.17s}
Nf4 {+37.90/20 0.097s} 9. Kc4 {-2.00/18 0.086s} Bd5+ {+37.90/19 0.095s}
10. Kc3 {-2.08/16 0.048s} Ke4 {+37.90/20 0.12s} 11. Kd2 {-1.87/17 0.078s}
Bf7 {+37.90/19 0.079s} 12. Kc2 {-1.79/17 0.10s} Kd4 {+37.90/19 0.084s}
13. Kb2 {-1.77/16 0.078s} Ng6 {+37.90/18 0.080s} 14. Kc2 {-1.59/18 0.093s}
Ne5 {+37.91/18 0.12s} 15. Kb2 {-1.61/17 0.13s} Kd3 {+37.93/19 0.096s}
16. Kb1 {-1.50/15 0.038s} Kc3 {+37.91/20 0.11s} 17. Ka1 {-1.48/15 0.069s}
Be6 {+37.90/20 0.14s} 18. Kb1 {-1.41/14 0.059s} Nc6 {+37.90/20 0.096s}
19. Ka1 {-1.35/14 0.059s} Bd5 {+37.90/19 0.088s} 20. Kb1 {-1.36/14 0.072s}
Ne5 {+37.90/20 0.15s} 21. Ka1 {-1.28/16 0.065s} Nd3 {+37.90/21 0.12s}
22. Kb1 {-1.20/15 0.067s} Nc5 {+37.90/21 0.17s} 23. Ka1 {-1.02/17 0.17s}
Nb3+ {+37.90/19 0.097s} 24. Ka2 {-1.37/19 0.16s} Kc2 {+37.94/21 0.11s}
25. Ka3 {-0.95/17 0.077s} Nd4 {+38.03/22 0.13s} 26. Kb4 {-1.22/21 0.13s}
Ne6 {+38.07/22 0.14s} 27. Ka4 {-M36/29 0.27s} Kc3 {+38.15/21 0.14s}
28. Ka3 {-M34/29 0.10s} Nc7 {+38.16/20 0.10s} 29. Ka4 {-M32/26 0.069s}
Bf7 {+38.30/20 0.10s} 30. Ka5 {-M30/27 0.084s} Ne6 {+38.30/21 0.13s}
31. Ka4 {-M32/25 0.092s} Nd4 {+38.17/21 0.12s} 32. Ka3 {-M34/32 0.19s}
Nb5+ {+M21/22 0.14s} 33. Ka4 {-M32/29 0.061s} Be8 {+M23/23 0.15s}
34. Ka5 {-M30/31 0.076s} Nc7 {+M19/21 0.11s} 35. Kb6 {-M28/33 0.079s}
Ne6 {+M23/20 0.18s} 36. Ka5 {-M26/32 0.073s} Bd7 {+M17/19 0.17s}
37. Kb6 {-M24/30 0.10s} Kc4 {+M17/18 0.12s} 38. Ka5 {-M22/31 0.091s}
Be8 {+M17/18 0.11s} 39. Kb6 {-M24/32 0.075s} Kb4 {+M17/18 0.15s}
40. Ka6 {-M18/32 0.10s} Kc5 {+M15/18 0.19s} 41. Ka5 {-M16/32 0.098s}
Nd4 {+M13/14 0.014s} 42. Ka6 {-0.36/14 0.15s} Bb5+ {+M13/15 0.17s}
43. Kb7 {-M14/34 0.28s} Ne6 {+M9/11 0.020s} 44. Ka7 {-M12/31 0.079s}
Bd7 {+38.69/14 0.27s} 45. Ka6 {0.00/15 0.078s} Nf8 {+37.93/11 0.12s}
46. Kb7 {0.00/20 0.077s} Bb5 {+37.45/11 0.10s} 47. Kc8 {0.00/23 0.078s}
Kb4 {+0.01/16 0.11s} 48. Kb7 {0.00/27 0.087s} Ne6 {+0.01/23 0.11s}
49. Kb6 {0.00/30 0.086s} Nf4 {+0.01/25 0.14s} 50. Kc7 {0.00/245 0.014s}
Bf1 {-0.01/121 0.006s, Draw by fifty moves rule} 1/2-1/2

Sometimes mate is never found

Code: Select all

[White "stockfish-dev"]
[Black "minic_dev_dev"]
[Result "1/2-1/2"]

1... Kf6 {-37.71/17 0.065s} 2. Kf4 {+2.37/16 0.15s} Ke6 {-37.72/19 0.057s}
3. Ke4 {+2.38/15 0.024s} Kd6 {-37.84/20 0.11s} 4. Nf5+ {+2.49/15 0.045s}
Kc6 {-37.90/17 0.079s} 5. Kd4 {+2.53/17 0.12s} Kb6 {-37.90/20 0.088s}
6. Nd6 {+2.75/16 0.064s} Ka6 {-37.90/21 0.080s} 7. Kc5 {+2.88/16 0.073s}
Ka7 {-37.90/21 0.084s} 8. Bg1 {+2.88/19 0.33s} Kb8 {-37.90/20 0.071s}
9. Bh2 {+1.96/19 0.22s} Kc7 {-37.90/21 0.080s} 10. Bf4 {+2.76/18 0.039s}
Kb8 {-37.90/20 0.086s} 11. Kb6 {+1.67/18 0.14s} Ka8 {-37.90/18 0.021s}
12. Bg3 {+2.13/18 0.31s} Kb8 {-37.90/15 0.015s} 13. Nb5+ {+1.72/19 0.18s}
Ka8 {-37.90/21 0.14s} 14. Nd6 {+1.43/18 0.085s} Kb8 {-37.90/17 0.019s}
15. Bh4 {+1.61/17 0.17s} Ka8 {-37.89/15 0.023s} 16. Bg5 {+1.53/17 0.18s}
Kb8 {-37.87/15 0.022s} 17. Be3 {+1.37/18 0.18s} Ka8 {-37.87/15 0.021s}
18. Kc6 {+1.20/19 0.21s} Kb8 {-37.87/16 0.028s} 19. Bd4 {+1.26/16 0.040s}
Ka8 {-37.88/17 0.014s} 20. Kb6 {+1.31/17 0.10s} Kb8 {-37.88/17 0.015s}
21. Bh8 {+1.21/17 0.14s} Ka8 {-37.87/16 0.024s} 22. Nf5 {+1.08/17 0.074s}
Kb8 {-37.87/15 0.013s} 23. Be5+ {+1.11/14 0.059s} Ka8 {-37.87/22 0.20s}
24. Bg7 {+1.00/16 0.080s} Kb8 {-37.87/16 0.013s} 25. Ne7 {+1.08/17 0.14s}
Ka8 {-37.89/15 0.014s} 26. Nc8 {+0.97/14 0.022s} Kb8 {-37.89/17 0.015s}
27. Nd6 {+1.00/17 0.12s} Ka8 {-37.89/18 0.027s} 28. Bf6 {+0.80/16 0.10s}
Kb8 {-37.85/16 0.028s} 29. Be5 {+0.82/14 0.091s} Ka8 {-37.87/17 0.016s}
30. Ba1 {+0.63/19 0.29s} Kb8 {-37.87/17 0.028s} 31. Be5 {+0.62/14 0.054s}
Ka8 {-37.87/17 0.025s} 32. Kc6 {+0.61/15 0.13s} Ka7 {-38.01/24 0.54s}
33. Bd4+ {+0.62/16 0.10s} Kb8 {-38.01/25 0.43s} 34. Kb6 {+0.55/14 0.075s}
Ka8 {-38.01/18 0.017s} 35. Bf6 {+0.52/15 0.059s} Kb8 {-38.01/21 0.038s}
36. Bh4 {+0.37/16 0.20s} Ka8 {-38.01/20 0.036s} 37. Be7 {+0.35/14 0.062s}
Kb8 {-38.01/21 0.033s} 38. Bg5 {+0.30/15 0.14s} Ka8 {-38.01/18 0.035s}
39. Be7 {+0.26/13 0.034s} Kb8 {-37.95/18 0.043s} 40. Bf6 {+0.20/15 0.12s}
Ka8 {-37.95/18 0.053s} 41. Bd4 {+0.16/16 0.10s} Kb8 {-37.91/16 0.042s}
42. Bf6 {+0.11/15 0.053s} Ka8 {-37.85/17 0.073s} 43. Nc8 {+0.11/16 0.13s}
Kb8 {-37.72/16 0.072s} 44. Ne7 {+0.11/12 0.027s} Ka8 {-37.66/15 0.074s}
45. Bg5 {0.00/15 0.13s} Kb8 {-37.63/14 0.076s} 46. Bc1 {0.00/15 0.034s}
Ka8 {-0.49/13 0.078s} 47. Ka6 {0.00/21 0.093s} Kb8 {-0.01/18 0.045s}
48. Nd5 {0.00/20 0.022s} Kc8 {-0.01/33 0.59s} 49. Nc3 {0.00/29 0.039s}
Kb8 {+0.01/61 0.72s} 50. Bf4+ {0.00/69 0.041s} Kc8 {-0.01/121 0.003s}
51. Nb1 {0.00/245 0.009s, Draw by fifty moves rule} 1/2-1/2

Minic 3.18 NN endgame problem ... (14 games, *.zip)

Re: Minic 3.18 NN endgame problem ... (14 games, *.zip)

Re: Minic 3.18 NN endgame problem ... (14 games, *.zip)

Re: Minic 3.18 NN endgame problem ... (14 games, *.zip)

Re: Minic 3.18 NN endgame problem ... (14 games, *.zip)

Re: Minic 3.18 NN endgame problem ... (14 games, *.zip)

Re: Minic 3.18 NN endgame problem ... (14 games, *.zip)