Ridiculously Fast SSD's.. How good for chess?

[BBauer]

Hi Bob,

you wrote:
it is interesting, but it is not a critical issue to most games. Very few
reach the point where endgame table probes begin to be a problem,
speed-wise. I have played lots of games recently on ICC with no
tables at all, and have not noticed much difference at all, and might
eventually just turn them completely off most of the time...

While table bases on disk may not give any difference in playing strength
the situation is different in analysis. Here table bases are of great help.
Table bases on disk often result in a *big* slow down in nps, at least for crafty. So it is helpfull to copy at least the 3-4-men to memory and if you have enough mem the 5-men too.
Again a test position
Study by Awerbach
[d]2k2K2/8/pp6/2p5/2P5/PP6/8/8 w - - 0 1

kind regards
Bernhard

[M ANSARI]

EGTB's on SSD are a huge boost to an engine's performance. The read and right speeds are not what makes SSD's superior to HDD in EGTB use but rather the huge improvement in accessing the data. This is why even a "slow" USB drive will perform very well. I am not sure how much a RAID 0 array of SSD's would improve in access time, but I am very interested to find out.

Transcend came out with reasonably priced SSD's (32GB for around $350) but the new Samsungs are really what you want to get. The 32GB SSD's from Samsung are around $450 if you can find one. Obviously 32GB is not enough to put all 6 piece EGTB's, but you can have the most relevant EGTB's on there and the remainder on HDD.

I think once improvements in ELO performance hit a brick wall with engine development, you will see the hunt for addition ELO points come in improvements in EGTB performance. Bitbases use dramatically less space and thus a combination of bitbases and SSD's could be a workable option for the 6 piece EGTB's in this or the next generation SSD's.

[bob]

Hello

So, a chess engine running on something like this, how much do you think it might help? for TB's access as well. Wonder if it could prove to be any benefit for an engine.. Maybe the 0.1ms Random Access Times, or anything else..? (it's impressive on all ends).

If you compare SSD and HD AND there is any benefit at all out of the TBs my guess is that the difference is somewhere around 3 Elo.

Of course it is quite difficult to prove a three Elo difference, but assuming you did, calculate the price per Elo?

As long as a SSD is more expensive than a HD the simple question is if one ELO is worth one € or even one $?

Bye
Ingo

PS: Thinking about it for me personal it seems that the price per ELO is not linear. For 5 Elo I am not willing to pay anything, for 50 Elo that looks different!

For the CCT event, the machine I used had 12 gigs of RAM. Before starting Crafty, I simply executed this automatically:

cat TB/* >/dev/null

and then there was _zero_ disk I/O during endgames.

[tano-urayoan]

EGTB's on SSD are a huge boost to an engine's performance. relevant EGTB's on there and the remainder on HDD.

Was this statement demonstrated?

[Dann Corbit]

EGTB's on SSD are a huge boost to an engine's performance. relevant EGTB's on there and the remainder on HDD.

Was this statement demonstrated?

Bitbases in RAM have demonstrated about +50 Elo advantage.
EGTB has superior information so we should expect *at least* this much advantage. Probably not a lot more, because you have to get to the endgame to use them and games are often won before the endgame phase when the board is fairly empty.

[bob]

EGTB's on SSD are a huge boost to an engine's performance. relevant EGTB's on there and the remainder on HDD.

Was this statement demonstrated?

Bitbases in RAM have demonstrated about +50 Elo advantage.
EGTB has superior information so we should expect *at least* this much advantage. Probably not a lot more, because you have to get to the endgame to use them and games are often won before the endgame phase when the board is fairly empty.

I've not even seen that, assuming you start off with a reasonably intelligent program. If the program has zero endgame knowledge, if it can't win simple endings (to a computer) like KQ vs KR or KBN vs K or KBB vs K, then egtbs/bitbases might offer a pretty significant Elo boost. But to a program with a modicum of endgame knowledge, I don't think the gain is anywhere near 50+ Elo...

[Dann Corbit]

EGTB's on SSD are a huge boost to an engine's performance. relevant EGTB's on there and the remainder on HDD.

Was this statement demonstrated?

Bitbases in RAM have demonstrated about +50 Elo advantage.
EGTB has superior information so we should expect *at least* this much advantage. Probably not a lot more, because you have to get to the endgame to use them and games are often won before the endgame phase when the board is fairly empty.

I've not even seen that, assuming you start off with a reasonably intelligent program. If the program has zero endgame knowledge, if it can't win simple endings (to a computer) like KQ vs KR or KBN vs K or KBB vs K, then egtbs/bitbases might offer a pretty significant Elo boost. But to a program with a modicum of endgame knowledge, I don't think the gain is anywhere near 50+ Elo...

You will easily hit 5 man bitbase tables with 10 men on the board.

I saw two different experiments that showed about +50 Elo for bitbase files, but I cannot find the experiments now. I seem to recall the tests used very strong engines (like Scorpio or Glaurung) but I can't remember for sure.

I agree that there will not be a dramatic strength increase from either tablebase or bitbase files.

[bob]

EGTB's on SSD are a huge boost to an engine's performance. relevant EGTB's on there and the remainder on HDD.

Was this statement demonstrated?

Bitbases in RAM have demonstrated about +50 Elo advantage.
EGTB has superior information so we should expect *at least* this much advantage. Probably not a lot more, because you have to get to the endgame to use them and games are often won before the endgame phase when the board is fairly empty.

I've not even seen that, assuming you start off with a reasonably intelligent program. If the program has zero endgame knowledge, if it can't win simple endings (to a computer) like KQ vs KR or KBN vs K or KBB vs K, then egtbs/bitbases might offer a pretty significant Elo boost. But to a program with a modicum of endgame knowledge, I don't think the gain is anywhere near 50+ Elo...

You will easily hit 5 man bitbase tables with 10 men on the board.

I saw two different experiments that showed about +50 Elo for bitbase files, but I cannot find the experiments now. I seem to recall the tests used very strong engines (like Scorpio or Glaurung) but I can't remember for sure.

I agree that there will not be a dramatic strength increase from either tablebase or bitbase files.

I'm going to definitively test this before long on our cluster. I can play 25K games against 5 opponents, once using no tables, once with them, to see exactly what the difference (if any) actually is. I'm not sure it is anywhere near +50 however. EGTBs are not free, they slow the search down somewhat, and add exact knowledge in return. In some cases they actually hurt, in others they help.

I will report the results of this after I have a chance to run it... With the infiniband switch for the nodes, I will probably just copy the 5-piece files to a node before running a game, and with the nodes having 12 gigs of ram each, it will all fit in ram to give an optimal upper bound on how much it helps... disk-based will be slower and not help quite as much...

[BBauer]

Hi!

Bitbases in RAM should *never* hurt.
Tablebases in RAM should *never* hurt.

Only if tablebases are in RAM we could expect a significant elo improvement.

kind regards
Bernhard

[M ANSARI]

I think with EGTB's on mechanical HDD the ELO advantage of having EGTB's is negated due to the nps performance hit the engine suffers when EGTB's start to get hits. The idea is to remove the performance hit and thus the engine will be able to use the EGTB's to its full potential. There is absolutely no doubt that if you play with EGTB's without the performance hit the engine will play at least 50 ELO stronger ... probably more. Not only that, the engine will feel like it is more "mature" and thus much more useful in analysis. A good example of that would be Shredder ... with bitbases loaded (which have a dramatically smaller footprint and thus can be loaded in fast RAM) Shredder plays much stronger.

Unfortunately not as much work is invested in EGTB as it is in engine development. We had Thompson (I think that is what it was called) EGTB's and then Nalimov ... now bitbases are here but apparently very few engines use them. I would be tremendously intersted in an engine in the top groups that would be able to use bitbases ... they just make much more sense. With bitbases ... 6 piece EGTB's would be manageable in a SSD. I guess you could put the equivalent of around 1TB of Nalimov EGTB's into an SSD but am not sure if this has been tried yet. Who knows maybe 7 piece bitbases on SSD will be the norm very soon.