What is faster for endgame tables

[bob]

Hi !

Here's the thing. I did the original testing for compressed tablebases, and tried all sorts of blocksizes to see which offered the best tradeoff with respect to CPU operations vs disk drive transfers. We are not using 4K blocksizes. I honestly don't remember whether we settled on 8K or 16K for the blocksize, Eugene could chime in there. But we definitely read more than 4K of data.
I will add that all my testing was done exclusively on Linux boxes, using the fastest SCSI drives available at the time (10K U320). I now use 15K U320 drives.
I have not tried every sort of flash drive, SSD, and so forth, but so far I have not found a thing that is better than the raid-0 array I currently use...

Thanks for the explanation.
Perhabs it would be interesting for some users if you compare
your scsi-system with some fast usb flash if possible ?

Regards,
G.S.

It isn't easy to test. Ideally the best test would be 7gb of 3-4-5 piece tables, and I don't have any 7gb USB drives available for testing. Testing smaller sub-sets doesn't sound so interesting or useful...

[ThatsIt]

Hi !

Random access read: average 0.67, min 0.49, max 0.88 [ms]
Random access read (<504 MByte): average 0.51, min 0.44, max 0.71 [ms]

Many thx for testing !! The stick is really fast.

Best to you !
G.S.

[ThatsIt]

Hi !

End in end from all reading what is conclusion? Raid 0 or flash drives. keep in mind I have 300 gigs of 6 men tables but what about for 1-5 man tables?

A mixture of Raid-0 (very fast SCSI of course) and fast 8 GB USB stick
would be very good. The 1-5 men on the stick, the 6 men on the harddrive.

Best to you !
G.S.

[gigabyte137]

Now a little off topic but what do people use for hash size inside of Gui?

[smirobth]

I re-ran the four postions I tested earlier, this time using a 15,000 RPM SCSI hard drive. The flash solution was still fastest, although the SCSI is often not too far behind. This was a U160 drive on a 32 bit PCI bus, not one of the slightly faster U320 drives or a 64 bit PCI. Also the drive was not in a RAID array, just a single drive. OTOH it seems that access time is more important for tablebases than data transfer rates, and the access time of this drive should be just as good as a U320 and/or RAID 0. Here are the results. Again this is for Fritz 10, tablebase depth = 0, 2 MB tablebase cache, 128 MB hashtables. The first time in each row is the 200x flash, next is the 15000 RPM SCSI drive, last is the 7200 RPM SATA drive:

Morozevich,A-Mamedyarov,S, 2006
8/4K3/3P4/1N5p/6kp/6P1/8/8 b - - 0 1
Depth: 7/23 170kN, tb=540 0:00:00 0:00:01 0:00:01
Depth: 8/29 657kN, tb=3632 0:00:00 0:00:04 0:00:06
Depth: 9/29 1264kN, tb=7381 0:00:01 0:00:05 0:00:09
Depth: 10/26 1489kN, tb=8914 0:00:01 0:00:07 0:00:12
Depth: 11/25 1546kN, tb=9292 0:00:01 0:00:07 0:00:12
Depth: 12/30 2252kN, tb=15402 0:00:02 0:00:11 0:00:18
Depth: 13/31 2915kN, tb=21609 0:00:03 0:00:14 0:00:23
Depth: 14/32 5284kN, tb=43141 0:00:06 0:00:21 0:00:33
Depth: 15/35 18346kN, tb=182711 0:00:20 0:00:44 0:01:05
Depth: 16/36 49398kN, tb=530337 0:00:53 0:01:27 0:02:03
Depth: 17/42 105mN, tb=1224023 0:01:53 0:02:33 0:03:20

Kasparov,G-Kramnik,V, 1994
8/6k1/8/2K2p1P/1P3n2/5B2/8/8 b - - 0 1
Depth: 10/17 34kN, tb=456 0:00:00 0:00:00 0:00:01
Depth: 11/18 59kN, tb=827 0:00:00 0:00:01 0:00:02
Depth: 12/20 111kN, tb=1452 0:00:00 0:00:01 0:00:03
Depth: 13/30 367kN, tb=5240 0:00:02 0:00:03 0:00:07
Depth: 14/25 536kN, tb=7581 0:00:02 0:00:05 0:00:09
Depth: 15/32 1108kN, tb=17667 0:00:05 0:00:08 0:00:16
Depth: 16/33 2118kN, tb=32993 0:00:07 0:00:13 0:00:24
Depth: 17/28 3359kN, tb=48823 0:00:10 0:00:17 0:00:31
Depth: 18/33 5346kN, tb=74601 0:00:14 0:00:22 0:00:40
Depth: 19/37 12110kN, tb=183224 0:00:26 0:00:39 0:01:07
Depth: 20/35 17523kN, tb=258396 0:00:34 0:00:49 0:01:22
Depth: 21/36 26255kN, tb=373442 0:00:46 0:01:04 0:01:43
Depth: 22/42 48702kN, tb=719882 0:01:18 0:01:42 0:02:38
Depth: 23/40 78722kN, tb=1110124 0:01:54 0:02:23 0:03:31

Adams,M-Gelfand,B, 2006
4r3/2k5/2P3N1/3K1P2/8/8/8/8 w - - 0 1
Depth: 9/21 113kN, tb=2509 0:00:00 0:00:01 0:00:02
Depth: 10/24 155kN, tb=3540 0:00:00 0:00:01 0:00:02
Depth: 11/20 179kN, tb=4235 0:00:01 0:00:01 0:00:03
Depth: 12/22 255kN, tb=6247 0:00:01 0:00:02 0:00:03
Depth: 13/22 369kN, tb=9405 0:00:01 0:00:02 0:00:04
Depth: 14/31 605kN, tb=16008 0:00:02 0:00:03 0:00:06
Depth: 15/32 1157kN, tb=31059 0:00:03 0:00:05 0:00:09
Depth: 15/41 2291kN, tb=60107 0:00:05 0:00:08 0:00:13
Depth: 16/68 4102kN, tb=109563 0:00:08 0:00:12 0:00:20
Depth: 17/45 6322kN, tb=172556 0:00:11 0:00:16 0:00:25
Depth: 18/45 8600kN, tb=241286 0:00:15 0:00:20 0:00:32
Depth: 19/45 10577kN, tb=303013 0:00:18 0:00:24 0:00:36
Depth: 20/46 23562kN, tb=696976 0:00:36 0:00:45 0:01:04
Depth: 21/58 37338kN, tb=1112556 0:00:52 0:01:03 0:01:25
Depth: 21/58 46999kN, tb=1380196 0:01:00 0:01:11 0:01:34
Depth: 21/58 57288kN, tb=1673621 0:01:08 0:01:19 0:01:42
Depth: 21/97 137mN, tb=3238198 0:02:22 0:02:42 0:03:23

Karpov,A-Anand,V, 1998
7q/8/1P2R3/4R3/7K/7P/5k2/8 w - - 0 1
Depth: 9/29 361kN, tb=1143 0:00:01 0:00:01 0:00:03
Depth: 9/30 529kN, tb=1749 0:00:01 0:00:02 0:00:04
Depth: 9/32 897kN, tb=2664 0:00:01 0:00:03 0:00:05
Depth: 9/32 1002kN, tb=2938 0:00:02 0:00:03 0:00:05
Depth: 9/32 1045kN, tb=3102 0:00:02 0:00:03 0:00:05
Depth: 9/32 1071kN, tb=3261 0:00:02 0:00:03 0:00:05
Depth: 10/31 1227kN, tb=3959 0:00:02 0:00:04 0:00:06
Depth: 11/32 1527kN, tb=5081 0:00:02 0:00:04 0:00:07
Depth: 12/33 3167kN, tb=11774 0:00:05 0:00:08 0:00:13
Depth: 13/36 6045kN, tb=26443 0:00:08 0:00:13 0:00:21
Depth: 14/44 16776kN, tb=79116 0:00:19 0:00:27 0:00:40
Depth: 15/53 54457kN, tb=334503 0:00:56 0:01:10 0:01:34
Depth: 16/54 76074kN, tb=488671 0:01:17 0:01:34 0:02:02
Depth: 16/76 377mN, tb=2832102 0:05:46 0:06:10 0:06:58

[M ANSARI]

The fastest USB drive is made by Corsair ... if a fast USB drive can be faster than a SCSI 15K rpm drive, then imagine what this Samsung SSD can do. It has about 5X faster access time than the fastest USB drive. Here is an interesting read

http://www.pcworld.com/article/id,13326 ... ticle.html

Again I believe that the best deal would be a combination of both types of drives. A super fast SSD for 5 EGTB's and a SATA RAID 0 array for the 6 EGTB. U320 SCSI 15K rpm drives are really great but very expensive. The new SATA 10K drives in RAID O (while not as fast as SCSI) will do a nice job at a reasonable price.