8man Endgame Tablebases ELO gain?

[dangi12012]

https://digitalcommons.calpoly.edu/cgi/ ... text=cpesp
https://github.com/glinscott/fishtest/wiki/UsefulData Elo gain using syzygy

The summary of these documents hints at:
syzygy5: 939.0 MiB - 2 ELO
syzygy6: 149.2 GiB - 15 ELO
syzygy7: 16.7 TiB - ? ELO
syzygy7: 1700 Tib? - ? ELO

Multiple questions:
a) Could it be that the storage size increase of around 110x per man added does not justify endgame tablebases at all?
b) Is 1.7PB an accurate estimate?
c) Does someone have a fishtest estimate for 7 man egtb?
d) Could it be that this looks bad only on first look and that the ELO increase could explode with more added pieces?

Please answer including the letter.

[yeni_sekme]

The values you post is outdated . Current stockfish doesn't gain elo from using 6-men syzygy.

[dangi12012]

The values you post is outdated . Current stockfish doesn't gain elo from using 6-men syzygy.

URL to Source? Why would official fishtest publish 15-20ELO then?

[yeni_sekme]

from vondele:

hmm, I'm running a test with current master using cutechess:

Code: Select all

./cutechess-cli -repeat -rounds 100000 -games 2 -tournament round-robin -srand 1942127704 \
                -resign movecount=3 score=400 -draw movenumber=34 movecount=8 score=20 \
                -concurrency 30 -openings file=noob_3moves.epd format=epd order=random plies=16 \
                -engine name=TB cmd=stockfish.master option.SyzygyPath=/home/vondele/chess/syzygy/3-4-5-6/WDL:/home/vondele/chess/syzygy/3-4-5-6/DTZ \
                -engine name=noTB cmd=stockfish.master \
                -each proto=uci tc=10.00+0.1 -ratinginterval 1

WDL on M.2 SSD
right now, no positive effect of TB. Will post numbers once I have a few more games.

My results for 4000 (10+0.1s) games:

Code: Select all

Score of TB vs noTB: 235 - 250 - 3515  [0.498] 4000
Elo difference: -1.3 +/- 3.7, LOS: 24.8 %, DrawRatio: 87.9 %

[dangi12012]

from vondele:

hmm, I'm running a test with current master using cutechess:

Code: Select all

./cutechess-cli -repeat -rounds 100000 -games 2 -tournament round-robin -srand 1942127704 \
                -resign movecount=3 score=400 -draw movenumber=34 movecount=8 score=20 \
                -concurrency 30 -openings file=noob_3moves.epd format=epd order=random plies=16 \
                -engine name=TB cmd=stockfish.master option.SyzygyPath=/home/vondele/chess/syzygy/3-4-5-6/WDL:/home/vondele/chess/syzygy/3-4-5-6/DTZ \
                -engine name=noTB cmd=stockfish.master \
                -each proto=uci tc=10.00+0.1 -ratinginterval 1

WDL on M.2 SSD
right now, no positive effect of TB. Will post numbers once I have a few more games.

My results for 4000 (10+0.1s) games:

Code: Select all

Score of TB vs noTB: 235 - 250 - 3515  [0.498] 4000
Elo difference: -1.3 +/- 3.7, LOS: 24.8 %, DrawRatio: 87.9 %

And stockfish prints "Found 510 Tablebases" on startup? NVME is good that you have it. How big is your cache size?
Because a tablebase hit will be stored in cache and thats important to not tank performance.

I think maybe it looks useless but in reality Tablebases maybe allow hyperexponentional ELO gains - but only after 7+ man!

[yeni_sekme]

Test is done by vondele, I don't know his hardware.

Yeah If you have a fast Nvme SSD, syzygy6 may gain a few elo. I have a 1tb 3500mb/s ssd but I didn't test the effect of syzygy.

Also If I remember correctly syzygy7 was like ~5elo according to a test in talkchess.I may be wrong , you can search in talkchess to find the test.

[Sesse]

Here is a relevant question: Do you care about the evaluation being more accurate?

If you do, I frequently see tablebases being relevant (although I have to admit it is anecdotal). If you only care about winning computer-computer matches, Elo is all that matters (more accurate evaluation often helps winning, but not always).

[chrisw]

Stockfish already has substantial end game knowledge both in HCE and NNUE. You’re not comparing “no end game knowledge” with EGTB knowledge and all that the minimal or even negative Elo differences show is that SF endgame knowledge is of the power of a 6 man EGTB already.

[Sesse]

I don't think you can make that conclusion.

The typical case is a drawn endgame with one or two pawns up. Stockfish, even with NNUE, will show a clear positive score; with tablebases, you'll get the appropriate message of a TB draw. Will no-TB Stockfish still be able to hold the draw? Sure! Would TB have been able to get into a non-drawn endgame? Probably not. (So there's no Elo to be had from TBs here, even though the evaluation is clearly wrong. The second simply does not follow from the first, because all that matters for Elo is whether there is a winning move and it has a higher score, not the absolute values of the evaluations.) Does this prove that Stockfish already has the required knowledge? Again, it depends on what your goal is.

[yurikvelo]

10 + 0.1 games (Fishtest STC) typically are adjudicated before 6-men hit numbers arrive.

At time controls where you can run enough games - TBHits do not come at stage yet.
Decisive blunders which lead to non-draw game (or non draw game-pair in case of busted GROB-opening) are made before TBHits