My generator (with a few minor modifications) would need a machine with 1TB of shared memory to generate 7-piece tables. Such machines exist, but I don't have one Smile. I suppose they will become affordable eventually.
Maybe if you make the necessary modifications someone will pick up the generation?
In TB generation, problem is not the generation or the amount of RAM needed. Problem is that practically no one has enough disk capacity to store them... Or are you prepared to host 200TBs of data?
It will be more like 20 TB (for WDL + DTZ), so for someone with a 1 TB system willing to generate the tables, storing the results should not be a problem. But I agree this is still too much for any practical distribution scheme.
Some months ago someone expressed interest in generating the 7-piece set, but he quickly became demanding and annoying, so I decided I really had much better things to do with my time. But maybe I'll update the source to support 7-men generation.when I find some time. (The biggest problem are the tables with very deep mates. In principle my format supports very large DTZ values, but my compressor currently does not. The easiest solution is to simply divide the values by 2 and leave it to the root probing code to deal with it. It only affects positions that are drawn by the 50-move rule anyway.)
IGarcia wrote:
How you conclude about 11 number? "10 would still miss it but also that 12 will not be necesary" Its based on same mathematics you use to determine earth age (5774 years) ?
Do you realize the amount of bytes to hold a 10 or 11 TB?
very basically if every extra piece takes up to 60 times as much space then to get to 10 pieces would need 216,000 times as much space as 7 pieces. 7 piece takes 100 tb so 10 pieces would be about 22 million terrabaytes.
currently the size of internet is 5 million terabytes. by 2017 it should be 20 million.
in theory then in a couple of years it may be technically feasible to store 10 pieces. in practice, assuming 7 years a piece it should be 2035
So IF your numbers are correct, we need a 10 men TB with a size bigger than 4 times our current internet.
So take all hard drives in this earth holding that data, buy 3 or 4 more times that, sum them up, interconnect them, POWER them, FILL them with the data and keep them all up at the same time available for some chess players /engines consult them.
Who will do the work? Who will pay the energy? How much time will take to compute the TB? How much time to verify all data is correct?
How you will handle disk failures? Or you expect to have a backup?
This is like like taking a man to Pluto o Neptune...at some time will be possible, but it worth?
I think Ken Thompson did all 5-piece tables in 1986 (but maybe these were not complete?).
He also did 6-piece tables somewhere in the 90s, but it seems this set is not complete and in particular does not include pawn endgames. Lewis Stiller was the first to do pawnless 6-piece endgames on a 65536-processor Connection Machine, but he could not save the results.
I'm pretty sure Steven Edwards generated some 5-piece tables before 2000, but maybe he did not generate all of them.
This paper suggests that the 5-piece Nalimov set was finished in 2000 or earlier. In fact it mentions the tables were used in WCCC 1999, so they were probably available already in 1998 or 1999. This is also more in line with my memory.
My generator (with a few minor modifications) would need a machine with 1TB of shared memory to generate 7-piece tables. Such machines exist, but I don't have one Smile. I suppose they will become affordable eventually.
Maybe if you make the necessary modifications someone will pick up the generation?
In TB generation, problem is not the generation or the amount of RAM needed. Problem is that practically no one has enough disk capacity to store them... Or are you prepared to host 200TBs of data?
Storing of the once generated results should be not too hard, since it could be done distributed. I would be willing to store 1 TB of the generated data and I'm sure there are many who would do the same. Storge during generation is more of a problem since during creation you need more space than afterwards. But surely there is someone out there who can provide 200 TB for a few months (assuming generation will be done in a few months).
P.S.: I predict we will have at least some 8pcs-TBs by 2020.
syzygy wrote:
It will be more like 20 TB (for WDL + DTZ), so for someone with a 1 TB system willing to generate the tables, storing the results should not be a problem.
20 TB is not much. With a bit of community effort those tables could be made available I think.
syzygy wrote:
It will be more like 20 TB (for WDL + DTZ), so for someone with a 1 TB system willing to generate the tables, storing the results should not be a problem.
20 TB is not much. With a bit of community effort those tables could be made available I think.
arjuntemurnikar wrote:Are there any plans to have a DTZ version of Lomonosov Endgame Tablebases by yourself, or someone else? Is this even feasible?
My generator (with a few minor modifications) would need a machine with 1TB of shared memory to generate 7-piece tables. Such machines exist, but I don't have one . I suppose they will become affordable eventually.
My generator (with a few minor modifications) would need a machine with 1TB of shared memory to generate 7-piece tables. Such machines exist, but I don't have one Smile. I suppose they will become affordable eventually.
Maybe if you make the necessary modifications someone will pick up the generation?
In TB generation, problem is not the generation or the amount of RAM needed. Problem is that practically no one has enough disk capacity to store them... Or are you prepared to host 200TBs of data?
At current prices, 200TB of HDD costs roughly $8000, 1TB of memory about $9000. So these are not outrageous amounts of money for someone who has the resources and will. Add another $10-20K (guesstimate) for the rest of the setup, and 7-piece Syzygy could be made by an obsessive person who is into this kind of thing.
Price of a decent car! But I would rather have the car, speaking for myself.
PS: Too bad automobiles don't follow something like Moore's Law!
Edit: I just saw that Ronald says 20 TB of storage, I have almost that among all my drives in my home (counting DVR expansion drives)
arjuntemurnikar wrote:Are there any plans to have a DTZ version of Lomonosov Endgame Tablebases by yourself, or someone else? Is this even feasible?
My generator (with a few minor modifications) would need a machine with 1TB of shared memory to generate 7-piece tables. Such machines exist, but I don't have one . I suppose they will become affordable eventually.
. I suppose they will become affordable eventually.
