Natural TB

[mcostalba]

Natural is performing above Syzygy.

Thanks for testing it!

We know result is not definitive because TC is very short and result well within noise level given the not so many number of games. More tests are needed and if people want to test it it would be great!

Of course ELO is measured on real games, not on synthetic endgame positions (and anyhow, as already stated many times, I am interested in ELO equivalence with syzygy, no more no less than this).

[Vinvin]

Code: Select all

(Avg game length = 8.915 sec)

...

Way to short to be serious. Not shorter than 5 sec/move, please !

The only difference (between Syzygy and natural) should be when there's positions with 5 pieces or less, isn't it ? Because with more than 5 pieces, they play same moves (from the WDL files).
So, natural draw instead of winning games with 5 pieces. What are these positions (if you saved games) ?

[Laskos]

Code: Select all

(Avg game length = 8.915 sec)

...

Way to short to be serious. Not shorter than 5 sec/move, please !

The only difference (between Syzygy and natural) should be when there's positions with 5 pieces or less, isn't it ? Because with more than 5 pieces, they play same moves (from the WDL files).
So, natural draw instead of winning games with 5 pieces. What are these positions (if you saved games) ?

I think TBs can be tested at ultra-fast, but it's safer to test at a bit longer tc. Also, as Marco observed, the result for 6-10 men endgames is within error margins, too few games. Yes, I don't know how Natural can be in any way better than Syzygy from more than 5 pieces.

Here are examples of 5-men hard wins, I have several hundred of them, please check these. Natural fails to win copiously on many of them.

8/8/8/8/3n4/6K1/k7/2B2B2 w - -
2B5/6B1/3K4/8/8/8/2kn4/8 w - -
4B3/6n1/8/8/1K5k/8/8/6B1 w - -
8/8/8/8/8/1n6/1k3K1B/1B6 w - -
6n1/8/7k/1K6/7B/5B2/8/8 w - -
8/8/8/6Bk/8/8/K7/5Bn1 w - -
8/3BB3/8/8/4k3/1K6/5n2/8 w - -
6K1/k7/8/8/8/3n1B2/3B4/8 w - -
4k3/8/3n4/8/8/8/4B1K1/B7 w - -
8/4n3/4B3/8/8/4B3/1K6/5k2 w - -
1B3k2/8/8/8/6n1/2K5/8/7B w - -
2k2B2/8/8/8/8/8/6B1/K2n4 w - -
6B1/B3n3/8/8/8/5K2/8/1k6 w - -
3BK3/8/8/5B2/8/5k2/8/n7 w - -
8/1B6/3B4/2K5/8/4n3/2k5/8 w - -
6K1/k7/8/2n1B3/6B1/8/8/8 w - -
1K3B1k/8/8/8/8/5n2/8/5B2 w - -

[bob]

Code: Select all

(Avg game length = 8.915 sec)

...

Way to short to be serious. Not shorter than 5 sec/move, please !

The only difference (between Syzygy and natural) should be when there's positions with 5 pieces or less, isn't it ? Because with more than 5 pieces, they play same moves (from the WDL files).
So, natural draw instead of winning games with 5 pieces. What are these positions (if you saved games) ?

I think TBs can be tested at ultra-fast, but it's safer to test at a bit longer tc. Also, as Marco observed, the result for 6-10 men endgames is within error margins, too few games. Yes, I don't know how Natural can be in any way better than Syzygy from more than 5 pieces.

Here are examples of 5-men hard wins, I have several hundred of them, please check these. Natural fails to win copiously on many of them.

8/8/8/8/3n4/6K1/k7/2B2B2 w - -
2B5/6B1/3K4/8/8/8/2kn4/8 w - -
4B3/6n1/8/8/1K5k/8/8/6B1 w - -
8/8/8/8/8/1n6/1k3K1B/1B6 w - -
6n1/8/7k/1K6/7B/5B2/8/8 w - -
8/8/8/6Bk/8/8/K7/5Bn1 w - -
8/3BB3/8/8/4k3/1K6/5n2/8 w - -
6K1/k7/8/8/8/3n1B2/3B4/8 w - -
4k3/8/3n4/8/8/8/4B1K1/B7 w - -
8/4n3/4B3/8/8/4B3/1K6/5k2 w - -
1B3k2/8/8/8/6n1/2K5/8/7B w - -
2k2B2/8/8/8/8/8/6B1/K2n4 w - -
6B1/B3n3/8/8/8/5K2/8/1k6 w - -
3BK3/8/8/5B2/8/5k2/8/n7 w - -
8/1B6/3B4/2K5/8/4n3/2k5/8 w - -
6K1/k7/8/2n1B3/6B1/8/8/8 w - -
1K3B1k/8/8/8/8/5n2/8/5B2 w - -

Are you using a somewhat unusual definition of "copiously"??? IE does this mean easily, or quickly, or with least loss of material? I assume it does not mean that "natural fails to win when it should in many of them?"

[Joerg Oster]

Today I played some games between natural and syzygy, 2 min + 1 sec.
Looking into the games, I stumbled over this one:

[pgn][Event "?"]
[Site "?"]
[Date "2016.06.01"]
[Round "13"]
[White "SF-NaturalTB"]
[Black "SF-Master"]
[Result "1/2-1/2"]
[ECO "A05"]
[Opening "Reti Opening"]
[PlyCount "180"]
[TimeControl "120+1"]

1. Nf3 {book} Nf6 {book} 2. c4 {book} e6 {book} 3. g3 {book} d5 {book}
4. Bg2 {book} Be7 {book} 5. O-O {book} O-O {book} 6. d4 {book} Nbd7 {book}
7. Qc2 {book} c6 {book} 8. Bf4 {book} b6 {book} 9. b3 {0.00/19 6.0s}
Ba6 {+0.02/20 6.4s} 10. Rd1 {0.00/20 3.6s} Rc8 {0.00/21 11s}
11. Nc3 {+0.03/20 2.4s} dxc4 {+0.05/22 4.4s} 12. bxc4 {-0.07/22 4.7s}
Bxc4 {0.00/21 0.82s} 13. Nd2 {+0.01/21 0.86s} Bb5 {-0.05/25 7.5s}
14. Qb3 {+0.14/23 5.1s} Nd5 {-0.12/24 4.7s} 15. Bxd5 {+0.07/22 0.76s}
exd5 {-0.05/25 3.0s} 16. Nxb5 {+0.12/24 2.3s} cxb5 {0.00/23 1.0s}
17. Qxb5 {+0.14/21 0.81s} Nf6 {-0.22/25 9.5s} 18. Rac1 {+0.18/22 4.6s}
Ba3 {-0.15/23 0.76s} 19. Rxc8 {+0.24/19 0.58s} Qxc8 {-0.09/24 2.1s}
20. Qa4 {+0.18/24 5.7s} Bc1 {-0.13/21 0.89s} 21. Kg2 {+0.29/25 19s}
h5 {-0.30/22 12s} 22. Be5 {+0.27/22 4.9s} Bxd2 {-0.39/23 13s}
23. Rxd2 {+0.32/21 0.82s} Ng4 {-0.30/23 2.8s} 24. h3 {+0.20/23 5.7s}
Nxe5 {-0.33/21 0.68s} 25. dxe5 {+0.24/22 1.0s} Qc4 {-0.21/22 1.9s}
26. Qxa7 {+0.23/23 3.7s} Qb4 {-0.20/22 0.70s} 27. Rd3 {+0.13/24 1.9s}
Qe4+ {-0.14/22 3.2s} 28. Rf3 {+0.35/26 13s} Qxe5 {-0.25/21 1.1s}
29. Qa6 {+0.23/26 8.3s} Qe6 {-0.22/22 5.5s} 30. Qb5 {+0.30/22 2.2s}
Ra8 {-0.34/22 4.9s} 31. a3 {+0.32/22 3.3s} Qe4 {-0.25/23 4.6s}
32. Kh2 {+0.21/18 0.69s} h4 {-0.27/24 5.0s} 33. gxh4 {+0.38/22 1.8s}
Re8 {-0.26/24 2.3s} 34. Qd7 {+0.30/22 1.4s} Re7 {-0.26/24 0.73s}
35. Qd8+ {+0.43/21 2.4s} Kh7 {-0.26/22 2.2s} 36. Qxb6 {+0.24/25 5.9s}
Qxe2 {-0.25/22 0.52s} 37. Rf5 {+0.25/23 0.68s} f6 {-0.22/22 2.9s}
38. Qd4 {+0.43/25 16s} Re5 {-0.17/23 1.7s} 39. Qg4 {+0.17/25 8.5s}
d4 {-0.23/23 0.64s} 40. Qxe2 {+0.17/26 4.6s} Rxe2 {-0.22/27 4.4s}
41. Kg2 {+0.17/26 0.43s} Rc2 {-0.18/25 0.48s} 42. Ra5 {+0.16/25 3.0s}
d3 {-0.15/27 1.9s} 43. Rd5 {+0.15/27 1.5s} d2 {-0.15/32 2.7s}
44. a4 {+0.15/27 0.40s} Ra2 {-0.15/31 0.68s} 45. a5 {+0.15/31 1.7s}
g5 {-0.15/32 2.5s} 46. hxg5 {+0.18/28 1.2s} fxg5 {-0.15/31 0.52s}
47. Kf3 {+0.18/29 0.31s} Rxa5 {-0.15/36 2.1s} 48. Rxd2 {+0.18/36 1.5s}
Kg6 {-0.15/37 0.57s} 49. Rd4 {+0.18/33 1.3s} Ra3+ {-0.15/40 2.2s}
50. Kg2 {+0.18/35 1.5s} Ra2 {-0.15/45 2.1s} 51. Rd7 {+0.18/40 1.5s}
Ra3 {-0.15/45 2.8s} 52. Rd6+ {+0.18/40 1.6s} Kg7 {-0.15/45 0.84s}
53. Rd5 {+0.18/40 0.29s} Kg6 {-0.15/45 2.1s} 54. f3 {+0.18/41 1.5s}
Kf6 {-0.15/45 2.1s} 55. Kg3 {+0.18/38 1.4s} Kg6 {-0.15/46 1.8s}
56. Rd6+ {+0.18/38 1.3s} Kf5 {-0.15/46 0.55s} 57. Rb6 {+0.18/39 0.32s}
Ra2 {-0.15/48 2.1s} 58. Rb7 {+0.18/41 1.4s} Kg6 {-0.15/46 2.1s}
59. Rb8 {+0.18/40 0.34s} Kg7 {-0.15/47 2.9s} 60. Re8 {+0.18/41 2.6s}
Kg6 {-0.15/50 1.7s} 61. Re4 {+0.18/39 1.3s} Kg7 {-0.15/47 1.6s}
62. Re7+ {+0.18/42 1.2s} Kg6 {-0.15/50 1.9s} 63. Re6+ {+0.18/43 0.31s}
Kf5 {-0.15/50 0.44s} 64. Rc6 {+0.18/42 1.2s} Ra3 {-0.15/47 1.9s}
65. Rc5+ {+0.18/45 1.4s} Kg6 {-0.15/49 1.7s} 66. Rd5 {+0.18/44 1.6s}
Ra2 {-0.15/48 2.1s} 67. Rd4 {+0.18/43 1.5s} Kg7 {-0.15/49 1.7s}
68. Rd7+ {+0.18/46 1.2s} Kf6 {-0.15/49 2.3s} 69. Rd6+ {+0.18/47 1.4s}
Kg7 {-0.15/49 0.79s} 70. Rd4 {+0.18/47 0.25s} Kg6 {-0.15/50 1.9s}
71. Rd8 {+0.18/48 1.9s} Ra3 {-0.15/50 0.42s} 72. Rg8+ {+0.18/42 1.1s}
Kf6 {-0.15/48 1.4s} 73. Rf8+ {+0.18/41 1.2s} Kg7 {-0.15/49 1.5s}
74. Re8 {+0.18/42 0.41s} Kg6 {-0.15/48 0.69s} 75. Re6+ {+0.18/39 1.3s}
Kf5 {-0.15/53 1.5s} 76. Rh6 {+0.18/41 1.2s} Ra4 {-0.15/50 2.1s}
77. Rh8 {+0.18/44 1.6s} Ra3 {-0.15/50 3.2s} 78. Rf8+ {+0.18/43 0.22s}
Kg6 {-0.15/48 0.30s} 79. Rd8 {+0.18/46 1.4s} Ra1 {-0.15/48 2.2s}
80. Rg8+ {+0.18/40 1.1s} Kf5 {-0.15/48 0.24s} 81. Kg2 {+0.18/40 1.3s}
Ra2+ {0.00/40 0.80s} 82. Kf1 {0.00/41 2.1s} Kf4 {0.00/40 1.2s}
83. Rf8+ {0.00/41 0.22s} Kg3 {0.00/42 0.27s} 84. Rf5 {0.00/44 1.2s}
Rf2+ {0.00/53 1.5s} 85. Ke1 {0.00/48 0.28s} Rxf3 {0.00/59 0.28s}
86. Rxf3+ {0.00/67 1.1s} Kxf3 {0.00/1 0.003s} 87. h4 {+M1/1 0.001s}
gxh4 {0.00/35 1.9s} 88. Kf1 {+M1/1 0s} h3 {0.00/36 0.75s}
89. Kg1 {0.00/1 0.001s} Kg3 {0.00/39 1.6s} 90. Kh1 {0.00/1 0s}
h2 {0.00/40 0.59s, Draw by stalemate} 1/2-1/2
[/pgn]

At move 87 and at move 88, White (SF-Natural) announces mate.
This seems rather odd.

[hgm]

It also only searches one ply deep. Which probably explains it, because Stockifish usually doesn't search any moves before the depth is 8, and if you have no moves mate is a natural conclusion. But of course that it only searches 1 ply is odd in itself.

[Laskos]

I assume it does not mean that "natural fails to win when it should in many of them?"

I meant that, in some positions often (not always, it depends on time used too) and in many of them. Sorry for using the word wrong way.

[Laskos]

Natural is performing above Syzygy.

Thanks for testing it!

We know result is not definitive because TC is very short and result well within noise level given the not so many number of games. More tests are needed and if people want to test it it would be great!

Of course ELO is measured on real games, not on synthetic endgame positions (and anyhow, as already stated many times, I am interested in ELO equivalence with syzygy, no more no less than this).

In real games from regular opening such as 2moves or 8moves, the difference might be too small to measure in casual testing like mine, serious testing is needed. Even in these endgame positions which act as a magnifying glass, the difference between No TB at all and different kinds of TBs is only of the order of 15-20 ELO points.

[bob]

(1) Unnatural scores. The case I am aware of where this was reported was a draw position which SF thought was lost. At that time Ronald's TB implementation returned the SF search score unfiltered (despite of course being aware of the draw). I guess it would be simple matter to return a draw score at root if the position is a TB draw.

That is what current SF does.

I suppose what Marco considers "unnatural" is reporting a TB win score (+123 or so) where the engine in fact has found a TB win. Rationality has nothing to do with this thread.

(2) Sacrifices to reach a TB position This appears to be unfixable for positions where the search cannot yet detect a mate.

The current implementation I think would normally already postpone the sacrifice in a proven win position if the postponement does not invalidate the proof (Ronald?).

The current implementation treats all TB wins equal and goes for the quickest one. This is by far the simplest approach. It would be possible to differentiate between TB wins (e.g. on the basis of material balance), but you will also need to take into account the distance to the TB win or the search might never get to any.

So it's not that nothing can be done (although even DTM will sometimes cause the engine to sack its queen), but doing something right needs careful thought (so forget about that in the context of Stockfish) and careful implementation. And whether it is worth to add a lot of complexity just to be able to win in a slightly nicer way (where the current engine already is guaranteed to win anyway)...

There are non-TB situations that draw complaints. Everyone likes the daring sacrifice, even if there is an actual shorter win possible. TBs have highlighted this because in general, it makes sense to take a proven win over something that might win quicker (i.e. by avoiding tossing a queen). I presume computers will NEVER play chess in a way that humans will accept without any complaints at all.

[jhellis3]

When you post a solution on page 1 and 7 pages later... /feelsbadman