Stockfish is AWESOME as endgame player

[Jouni]

SF 8.9.2013 played 200 games from EG_MSb starting positions vs Houdini3.
Result: 121,5 - 78,5 (+75,=93,-32) = +78 ELO . Time control 60 moves in minute.
Houdini has 5 piece Nalimov access. Note, that in complete games SF scores 50% or less with same conditions. So who needs tablebases?

[carldaman]

SF 8.9.2013 played 200 games from EG_MSb starting positions vs Houdini3.
Result: 121,5 - 78,5 (+75,=93,-32) = +78 ELO . Time control 60 moves in minute.
Houdini has 5 piece Nalimov access. Note, that in complete games SF scores 50% or less with same conditions. So who needs tablebases?

Here I have to agree wholeheartedly. For serious endgame analysis, SF has been indispensable for some time.

[Vinvin]

SF 8.9.2013 played 200 games from EG_MSb starting positions vs Houdini3.
Result: 121,5 - 78,5 (+75,=93,-32) = +78 ELO . Time control 60 moves in minute.
Houdini has 5 piece Nalimov access. Note, that in complete games SF scores 50% or less with same conditions. So who needs tablebases?

Nice news, thanks !
May be the greatly improved search is very useful in endgame.

[Jouni]

I repeated match with 100 different endgame position. Again SF was impressive (even in my outdated hardware):

SF - Houdini : 119.5/200 +72-33=95 (+70 ELO)

SF is specially good in winning RPP vs R fast. But there are limitations. This position seems to be unsolvable without tablebases

[d]1k6/8/1B5p/1R6/6K1/8/8/7r w - - 0 1 Bg1! only winning move

[syzygy]

This position seems to be unsolvable without tablebases

[d]1k6/8/1B5p/1R6/6K1/8/8/7r w - - 0 1 Bg1! only winning move

Without tablebases, my engine (using 6 cores) finds Bg1+ in 8.8 seconds, the score goes to +6.4 after 11.5 seconds, to mate in 25 after 26.5 seconds and to mate in 23 after 37.2 seconds.

Of course tablebases are still very useful to be able to correctly evaluate such positions deep in the tree.

Stockfish (without tablebases) seems to prune away the winning line here. No luck even at depth 57.

[syzygy]

Stockfish (without tablebases) seems to prune away the winning line here. No luck even at depth 57.

The problem here is of course nullmove / zugzwang. I'm not sure why my engine finds it. The 8.8s run was a bit lucky, but it does find it consistenly within 30 seconds or so.

In terms of Elo, being able to find the solution in this position is probably a weakness

[mcostalba]

SF 8.9.2013 played 200 games from EG_MSb starting positions vs Houdini3.
Result: 121,5 - 78,5 (+75,=93,-32) = +78 ELO . Time control 60 moves in minute.
Houdini has 5 piece Nalimov access. Note, that in complete games SF scores 50% or less with same conditions. So who needs tablebases?

Here I have to agree wholeheartedly. For serious endgame analysis, SF has been indispensable for some time.

60 moves in a minute has nothing to do with analysis, it is more a time management achievement than anything else.

Any test is always welcome but tests at super fast TC are not very telling.

[Jouni]

Or maybe not . There seems to be weird bug in Arena GUI. There SF scores always about 65% vs Houdini, but only 52% in Fritz GUI! Actually I am not sure which GUI to trust, but 65% is not reasonable.

[M ANSARI]

This makes me think that maybe it is time to create an endgame "module" that work independently of the engine using a PCI slot. The module would use all 6 piece bitbases and have some sort of MC implementation. It doesn't really need a strong chess engine, just an engine that will check for blocked positions or positions where no progress is made. Since this will work independent of the mainboard chess engine, it will not hurt ELO as there would be no performance hit, but it would have a trigger that overrides the main engine if a good hit is found. I wonder if a CUDA card or some other similar card can be coded for such a thing.

[pohl4711]

Hi all,

I did a test of Stockfish with Syzygy-Bases (3-5 pieces, on a fast USB-Stick, default-settings except the Syzygy-path) with LS-conditions http://ls-ratinglist.beepworld.de (10000 games against the opponents of the LS top10 tournament (all opponents without any endgame-bases)). That made it possible to compare the result with the LS-results of Stockfish (without Syzygy-Bases) against the same opponents.
LS-tested version of Stockfish was 131022 (latest version of that day). The tested Syzygy-Stockfish was 131023 (version from 00:43, wich is the "Merge remote-tracking branch 'upstream/master'" of Stockfish 131022). So both versions were identical, except the Syzygy-support.

Here the result (first value is Stockfish without Syzygy, second value is Stockfish with Syzygy-Bases)

Code: Select all

Stockfish - Houdini 3             48.25% | 50.00%
Stockfish - Komodo 6              52.80% | 52.50%
Stockfish - Amitis 130912         56.75% | 57.25%
Stockfish - Critter 1.6a          57.10% | 56.85%
Stockfish - Gull 2.3              56.10% | 56.60%
Stockfish - Strelka 5.5           56.30% | 57.30%
Stockfish - Mars 1                57.40% | 62.10%
Stockfish - Panchess 00.537       59.90% | 62.85%
Stockfish - Bouquet 1.8           59.80% | 61.95%
Stockfish - Robbolito 0.085g3     64.00% | 66.20%

Total: 56.84% | 58.36% = +11 Elo for Stockfish with Syzygy-Bases. (95%-Errorbar of both testruns (10000 games each)= +/-5 Elo)

This test shows, that there is a Elo gain for Stockfish with 3-5 Syzygy-bases, if all opponents dont use any endgame-bases. But the Elo gain is small...

Stefan