Page 1 of 1

Stockfish github has nice syzygy test

Posted: Thu Jan 16, 2020 8:32 am
by Jouni
2-6 pieces and up to 80 000 games. But I don't find it anymore. Where is it?

Re: Stockfish github has nice syzygy test

Posted: Thu Jan 16, 2020 11:01 am
by Jouni
Of course I mean 3-6 piece. And 3-4 gave nothing as SF has basic knowledge.

Re: Stockfish github has nice syzygy test

Posted: Thu Jan 23, 2020 8:15 am
by Jouni
It was cleverly hidden https://github.com/glinscott/fishtest/wiki/UsefulData.

Elo gain using syzygy

Tested at 10+0.1, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0), 4, 5, and 6 man TB in a round-robin tournament (SF10dev).
Rank Name Elo +/- Games Score Draws
1 syzygy6 13 2 82591 51.8% 59.5%
2 syzygy5 2 2 82590 50.3% 59.4%
3 syzygy4 -7 2 82591 49.0% 59.3%
4 syzygy0 -7 2 82592 48.9% 59.4%

Tested at 60+0.6, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0) against 6 man TB:

Score of syzygy6 vs syzygy0: 4084 - 3298 - 18510 [0.515] 25892 Elo difference: 10.55 +/- 2.25

So 6 piece = 149 GB data gives STC 20 ELO and LTC 10 ELO. Maybe at tournament level and SMP 5 ELO only :o ! Waste of storage?

Re: Stockfish github has nice syzygy test

Posted: Thu Jan 23, 2020 11:07 am
by Laskos
Jouni wrote:
Thu Jan 23, 2020 8:15 am
It was cleverly hidden https://github.com/glinscott/fishtest/wiki/UsefulData.

Elo gain using syzygy

Tested at 10+0.1, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0), 4, 5, and 6 man TB in a round-robin tournament (SF10dev).
Rank Name Elo +/- Games Score Draws
1 syzygy6 13 2 82591 51.8% 59.5%
2 syzygy5 2 2 82590 50.3% 59.4%
3 syzygy4 -7 2 82591 49.0% 59.3%
4 syzygy0 -7 2 82592 48.9% 59.4%

Tested at 60+0.6, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0) against 6 man TB:

Score of syzygy6 vs syzygy0: 4084 - 3298 - 18510 [0.515] 25892 Elo difference: 10.55 +/- 2.25

So 6 piece = 149 GB data gives STC 20 ELO and LTC 10 ELO. Maybe at tournament level and SMP 5 ELO only :o ! Waste of storage?
Hmmm, curious scaling. I would expect stronger play would be decided more often in endgames. Thanks for pointing, needs explanation, the increased draw rate here from 59% to 71% cannot stand alone for explanation.

Re: Stockfish github has nice syzygy test

Posted: Thu Jan 23, 2020 4:54 pm
by mbabigian
149GB is a fly spitting into the wind. Even if the occasional extra win was not a whole ELO point it would be worth it. Now if full 7 piece or 8, gave less than 5 ELO a debate about wasted space could be had.

At current storage costs 149GB is imperceptible bit dust. I have a thumb drive with more storage plugged into my TV.

Re: Stockfish github has nice syzygy test

Posted: Thu Jan 23, 2020 6:07 pm
by mwyoung
Jouni wrote:
Thu Jan 23, 2020 8:15 am
It was cleverly hidden https://github.com/glinscott/fishtest/wiki/UsefulData.

Elo gain using syzygy

Tested at 10+0.1, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0), 4, 5, and 6 man TB in a round-robin tournament (SF10dev).
Rank Name Elo +/- Games Score Draws
1 syzygy6 13 2 82591 51.8% 59.5%
2 syzygy5 2 2 82590 50.3% 59.4%
3 syzygy4 -7 2 82591 49.0% 59.3%
4 syzygy0 -7 2 82592 48.9% 59.4%

Tested at 60+0.6, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0) against 6 man TB:

Score of syzygy6 vs syzygy0: 4084 - 3298 - 18510 [0.515] 25892 Elo difference: 10.55 +/- 2.25

So 6 piece = 149 GB data gives STC 20 ELO and LTC 10 ELO. Maybe at tournament level and SMP 5 ELO only :o ! Waste of storage?
I agree with the numbers 5 to 20 Elo depending on the time controls. And this is what I see in my testing. But this is not the whole story. What does this Elo gain mean. It can mean 1 to 3 extra wins or saved losses in every 100 games. And Stockfish uses TB's very well because of the SF search, NN on the other had do not IMO. If I throw out 149 GB of Table Bases to save storage, on a 2 TB drive. Is this a wise move? No!

Accumulation of small advantages is how you win at chess at this Level!

Re: Stockfish github has nice syzygy test

Posted: Fri Jan 24, 2020 12:09 am
by mbabigian
If I throw out 149 GB of Table Bases to save storage, on a 2 TB drive. Is this a wise move? No!

Accumulation of small advantages is how you win at chess at this Level!
Exactly. I also agree that LCO's tablebase usage is a cruel joke.