Stockfish github has nice syzygy test

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Jouni
Posts: 2430
Joined: Wed Mar 08, 2006 7:15 pm

Stockfish github has nice syzygy test

Post by Jouni » Thu Jan 16, 2020 8:32 am

2-6 pieces and up to 80 000 games. But I don't find it anymore. Where is it?
Jouni

Jouni
Posts: 2430
Joined: Wed Mar 08, 2006 7:15 pm

Re: Stockfish github has nice syzygy test

Post by Jouni » Thu Jan 16, 2020 11:01 am

Of course I mean 3-6 piece. And 3-4 gave nothing as SF has basic knowledge.
Jouni

Jouni
Posts: 2430
Joined: Wed Mar 08, 2006 7:15 pm

Re: Stockfish github has nice syzygy test

Post by Jouni » Thu Jan 23, 2020 8:15 am

It was cleverly hidden https://github.com/glinscott/fishtest/wiki/UsefulData.

Elo gain using syzygy

Tested at 10+0.1, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0), 4, 5, and 6 man TB in a round-robin tournament (SF10dev).
Rank Name Elo +/- Games Score Draws
1 syzygy6 13 2 82591 51.8% 59.5%
2 syzygy5 2 2 82590 50.3% 59.4%
3 syzygy4 -7 2 82591 49.0% 59.3%
4 syzygy0 -7 2 82592 48.9% 59.4%

Tested at 60+0.6, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0) against 6 man TB:

Score of syzygy6 vs syzygy0: 4084 - 3298 - 18510 [0.515] 25892 Elo difference: 10.55 +/- 2.25

So 6 piece = 149 GB data gives STC 20 ELO and LTC 10 ELO. Maybe at tournament level and SMP 5 ELO only :o ! Waste of storage?
Jouni

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Stockfish github has nice syzygy test

Post by Laskos » Thu Jan 23, 2020 11:07 am

Jouni wrote:
Thu Jan 23, 2020 8:15 am
It was cleverly hidden https://github.com/glinscott/fishtest/wiki/UsefulData.

Elo gain using syzygy

Tested at 10+0.1, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0), 4, 5, and 6 man TB in a round-robin tournament (SF10dev).
Rank Name Elo +/- Games Score Draws
1 syzygy6 13 2 82591 51.8% 59.5%
2 syzygy5 2 2 82590 50.3% 59.4%
3 syzygy4 -7 2 82591 49.0% 59.3%
4 syzygy0 -7 2 82592 48.9% 59.4%

Tested at 60+0.6, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0) against 6 man TB:

Score of syzygy6 vs syzygy0: 4084 - 3298 - 18510 [0.515] 25892 Elo difference: 10.55 +/- 2.25

So 6 piece = 149 GB data gives STC 20 ELO and LTC 10 ELO. Maybe at tournament level and SMP 5 ELO only :o ! Waste of storage?
Hmmm, curious scaling. I would expect stronger play would be decided more often in endgames. Thanks for pointing, needs explanation, the increased draw rate here from 59% to 71% cannot stand alone for explanation.

mbabigian
Posts: 191
Joined: Tue Oct 15, 2013 12:34 am
Location: US
Full name: Mike Babigian

Re: Stockfish github has nice syzygy test

Post by mbabigian » Thu Jan 23, 2020 4:54 pm

149GB is a fly spitting into the wind. Even if the occasional extra win was not a whole ELO point it would be worth it. Now if full 7 piece or 8, gave less than 5 ELO a debate about wasted space could be had.

At current storage costs 149GB is imperceptible bit dust. I have a thumb drive with more storage plugged into my TV.
“Censorship is telling a man he can't have a steak just because a baby can't chew it.” ― Mark Twain

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: Stockfish github has nice syzygy test

Post by mwyoung » Thu Jan 23, 2020 6:07 pm

Jouni wrote:
Thu Jan 23, 2020 8:15 am
It was cleverly hidden https://github.com/glinscott/fishtest/wiki/UsefulData.

Elo gain using syzygy

Tested at 10+0.1, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0), 4, 5, and 6 man TB in a round-robin tournament (SF10dev).
Rank Name Elo +/- Games Score Draws
1 syzygy6 13 2 82591 51.8% 59.5%
2 syzygy5 2 2 82590 50.3% 59.4%
3 syzygy4 -7 2 82591 49.0% 59.3%
4 syzygy0 -7 2 82592 48.9% 59.4%

Tested at 60+0.6, with all syzygy WDL files on tmpfs (i.e. RAM), testing using none(0) against 6 man TB:

Score of syzygy6 vs syzygy0: 4084 - 3298 - 18510 [0.515] 25892 Elo difference: 10.55 +/- 2.25

So 6 piece = 149 GB data gives STC 20 ELO and LTC 10 ELO. Maybe at tournament level and SMP 5 ELO only :o ! Waste of storage?
I agree with the numbers 5 to 20 Elo depending on the time controls. And this is what I see in my testing. But this is not the whole story. What does this Elo gain mean. It can mean 1 to 3 extra wins or saved losses in every 100 games. And Stockfish uses TB's very well because of the SF search, NN on the other had do not IMO. If I throw out 149 GB of Table Bases to save storage, on a 2 TB drive. Is this a wise move? No!

Accumulation of small advantages is how you win at chess at this Level!
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

mbabigian
Posts: 191
Joined: Tue Oct 15, 2013 12:34 am
Location: US
Full name: Mike Babigian

Re: Stockfish github has nice syzygy test

Post by mbabigian » Fri Jan 24, 2020 12:09 am

If I throw out 149 GB of Table Bases to save storage, on a 2 TB drive. Is this a wise move? No!

Accumulation of small advantages is how you win at chess at this Level!
Exactly. I also agree that LCO's tablebase usage is a cruel joke.
“Censorship is telling a man he can't have a steak just because a baby can't chew it.” ― Mark Twain

Post Reply