What is faster for endgame tables

[bob]

RAID 0 array will not perform as fast as an solid state drivequote]

That is in theory. In reality raid0 murders solid state drives, because they are all using things like FLASH memory which is ridiculously slow. Yes, if you had a large DRAM disk (Cray Research builds them but they are not cheap) then they would be fast. But if you re-read the original post, he mentioned things like "thumbdrives" and the like and they are horribly slow compared to an array of SCSI drives.

Have you tested this for EGTBs? Flash drives have faster access times and slower read times. For small enough blocks, the access times should outweigh the read times.

I have, but only the USB variety. They are way slow. With EGTBs we need quick access time (low latency) but we also need terrific bandwidth because we have to read in large blocks of data to decompress and probe...

[Werner]

Hi Pax,
I am using a 8GB usb Memory stick (flash, no minidrive). Acess time very fast (0,7) from transcend - jetflash 160.
All tbs 1-5 men fit on it. The acess is very good. I can see no more low nps with Shredder 9 e.g.

[ThatsIt]

Hi Pax,
I am using a 8GB usb Memory stick (flash, no minidrive). Acess time very fast (0,7) from transcend - jetflash 160.
All tbs 1-5 men fit on it. The acess is very good. I can see no more low nps with Shredder 9 e.g.

I agree !
Until now there is no HDD and/or technology which is better (faster)
than flashmemory for TBs use, except the Shredderbases in the
main RAM of course.
The only thing which is important is the access time.

Best,
G.S.

[M ANSARI]

http://www.gigabyte.com.tw/Products/Sto ... uctID=2180

Here is something that is interesting and might be perfect for EGTB's ... but it would be expensive. I guess you could get away with it by using cheap second hard DDR 1 memory modules, but if you want to max memory out then DDR 2 is probably the only option. I guess depending on your PCI X slots that you have you could have several of these ... and memory access should be very very fast. I think 8 GB DIMMS might be out soon so 32GB or even 64GB is possible ... but that is still way short of the 1.2 TB that is needed.

So again the best solution would be a combination of fast ram and raid array.

[ThatsIt]

Hi !

{...snip...]

So again the best solution would be a combination of fast ram and raid array.

I think that Raid does not matter about the
access time at all. But only the access time is
the important thing to speed up the EGTB
access. The transfer rate is secondary.

I use 2x4GB sticks (swissbits) and the speed
up compared to fast SATA-HDDs is enormous.

Best to you !
G.S.

[smirobth]

http://www.gigabyte.com.tw/Products/Sto ... uctID=2180

Here is something that is interesting and might be perfect for EGTB's ... but it would be expensive. I guess you could get away with it by using cheap second hard DDR 1 memory modules, but if you want to max memory out then DDR 2 is probably the only option. I guess depending on your PCI X slots that you have you could have several of these ... and memory access should be very very fast. I think 8 GB DIMMS might be out soon so 32GB or even 64GB is possible ... but that is still way short of the 1.2 TB that is needed.

So again the best solution would be a combination of fast ram and raid array.

I think that this Gigabyte card can only handle a maximum of 4GB TOTAL (4x 1GB), so even for just all the 3-4-5 man it would not be big enough. But I guess you could get two of them, and it would be fast.

[Bryan Hofmann]

http://www.gigabyte.com.tw/Products/Sto ... uctID=2180

Here is something that is interesting and might be perfect for EGTB's ... but it would be expensive. I guess you could get away with it by using cheap second hard DDR 1 memory modules, but if you want to max memory out then DDR 2 is probably the only option. I guess depending on your PCI X slots that you have you could have several of these ... and memory access should be very very fast. I think 8 GB DIMMS might be out soon so 32GB or even 64GB is possible ... but that is still way short of the 1.2 TB that is needed.

So again the best solution would be a combination of fast ram and raid array.

There is also the hyperdrive-4 SSD which can be put into a RAID 0 configs, but again it is on the expensive side.

http://www.hyperdrive4.com/

Bryan

[bob]

Hi Pax,
I am using a 8GB usb Memory stick (flash, no minidrive). Acess time very fast (0,7) from transcend - jetflash 160.
All tbs 1-5 men fit on it. The acess is very good. I can see no more low nps with Shredder 9 e.g.

I agree !
Until now there is no HDD and/or technology which is better (faster)
than flashmemory for TBs use, except the Shredderbases in the
main RAM of course.
The only thing which is important is the access time.

Best,
G.S.

that's simply wrong. we read large blocks of data, as we have to decompress a complete block. We need significant transfer rate to pull this off. With a good raid-0 array, your NPS won't drop much at all in almost all cases...

[ThatsIt]

Hi Pax,
I am using a 8GB usb Memory stick (flash, no minidrive). Acess time very fast (0,7) from transcend - jetflash 160.
All tbs 1-5 men fit on it. The acess is very good. I can see no more low nps with Shredder 9 e.g.

I agree !
Until now there is no HDD and/or technology which is better (faster)
than flashmemory for TBs use, except the Shredderbases in the
main RAM of course.
The only thing which is important is the access time.

Best,
G.S.

that's simply wrong. we read large blocks of data, as we have to decompress a complete block. We need significant transfer rate to pull this off. With a good raid-0 array, your NPS won't drop much at all in almost all cases...

Lars Bremer has done a lot of measurements concerning EGTB on harddisks and
USB flashdrives for our article published in a german computerchess-magazine
in 2006/2007
---> http://www.computerschach.de/index.php? ... Itemid=272

Lars made additional tests with raid-0 in comparison to single drives, please
have a look at:
---> http://www.mustrum.de/artikel/TB-Karussell.pdf

We have done all the measurements with the 5men EGTB.

The blocksize which was requested by the harddisks was mostly 4KB, very seldom
8KB but not even higher ! Lars noticed that with tools that reports the demand
of datas directly on the driver.

Of course one must consider that there are different kinds of flashmemory concerning
the access time. We have used very fast USB sticks, but there are very slow ones on
the market too.

Regards,
G.S.

[Spock]

Slightly off-topic, but the best solution, assuming that you can put enough RAM in your machine, is surely a RAMDISK like this:

http://www.superspeed.com/desktop/ramdisk.php

I guess 7GB for the 5-men so for practical purposes given most multi-socket machine configurations means 16GB of system RAM