Tablebase access using a Solid State Disk
Today I got my first SSD (Solid State Disk). It is an external unit with 240 GiB capacity and it connects using a USB3 interface. The data transfer rate ceiling is well over 400 MiB/second, limited mostly by the USB3 ceiling. The cost was about US$1/GiB.
Important to the topic of tablebase access, the SSD I/O operation rate is about 100,000 operations per second, meaning that my program can probe tablebase files far more rapidly when they're on the SSD vs on a typical SATA hard drive. The difference? The SSD operation rate is faster by a factor of about 60, although this will vary much according to particulars.
An SSD connected via a PCI-e interface would be slightly more than twice as fast as with a USB3 connection, but this would limit which machines I could use.
I should probably change Symbolic to adapt its tablebase probe strategy based on the I/O operation rate of the storage used by the tablebase files. No rush here, as Symbolic won't need it until it returns to server play sometime next year.
Tablebase access using a Solid State Disk
Moderators: hgm, Rebel, chrisw
-
wgarvin
- Posts: 838
- Joined: Thu Jul 05, 2007 5:03 pm
- Location: British Columbia, Canada
Re: Tablebase access using a Solid State Disk
What's really nice about SSDs is that their "seek time" is pretty much zero. For some use cases that makes a really huge difference. If you get a quality one, the raw throughput will also be excellent.
-
sje
- Posts: 4675
- Joined: Mon Mar 13, 2006 7:43 pm
Re: Tablebase access using a Solid State Disk
Yes, the reduction of overall latency is the Big Win when using an SSD for tablebase storage.
My current tablebase format needs only a single byte to convey the exact mate/lose distance. The future format will need to read a maximum of three bytes which could possibly span a block boundary.
In the first case, exactly one block is read; for the second case one or two blocks is read. Each logical block on an HFS+ (Mac OS/X default) filesystem is 4 KiB long, so most of time spent for a tablebase element transfer is wasted regardless of the storage medium. That's what limits the probe rate to about 100 KHz for my USB3/SSD setup.
An SSD connected to a 6 Mbps SATA3 interface will run about a third faster. An SSD connected via a PCI-e interface might be twice as fast with a non RAID configuration. A RAID could run even faster, but would have the same latency.
Beyond all that would be the use of a RAM disk, and that's actually feasible for the three and four man tablebases.
--------
I had also considered moving the opening book to an SSD. But there's little to be gained here because Symbolic hits the book file only once; this is done at program boot time to load the book into RAM.
My current tablebase format needs only a single byte to convey the exact mate/lose distance. The future format will need to read a maximum of three bytes which could possibly span a block boundary.
In the first case, exactly one block is read; for the second case one or two blocks is read. Each logical block on an HFS+ (Mac OS/X default) filesystem is 4 KiB long, so most of time spent for a tablebase element transfer is wasted regardless of the storage medium. That's what limits the probe rate to about 100 KHz for my USB3/SSD setup.
An SSD connected to a 6 Mbps SATA3 interface will run about a third faster. An SSD connected via a PCI-e interface might be twice as fast with a non RAID configuration. A RAID could run even faster, but would have the same latency.
Beyond all that would be the use of a RAM disk, and that's actually feasible for the three and four man tablebases.
--------
I had also considered moving the opening book to an SSD. But there's little to be gained here because Symbolic hits the book file only once; this is done at program boot time to load the book into RAM.