Question for GCP about the Sjeng Cluster

[mhalstern]

What is the current hardware of the Sjeng Cluster. How much elo is it stronger than Deep Sjeng Running on a 4 core I7-920 at 4.0 GHZ, and the same pc at 2.67 GHZ.

Will you consider making a cluster version available as part of the next release, so A user can utilize the processing power of multiple pc's on the same network?

Thanks

[Gian-Carlo Pascutto]

What is the current hardware of the Sjeng Cluster.

Suj has to answer this one. IIRC, 2 times a 24 core AMD Istanbul and then filling up with 4 8-core Nehalems. Peak speed was a bit over 135Mnps.

How much elo is it stronger than Deep Sjeng Running on a 4 core I7-920 at 4.0 GHZ, and the same pc at 2.67 GHZ.

No idea. Very very roughly 200 ELO or something like that?

Will you consider making a cluster version available as part of the next release, so A user can utilize the processing power of multiple pc's on the same network?

That was the idea of making the cluster version...

[Highendman]

Will you consider making a cluster version available as part of the next release, so A user can utilize the processing power of multiple pc's on the same network?

That was the idea of making the cluster version...

That's very exciting news. Do you have an estimate you can share about when that release might be made available?

[mhalstern]

I'l excited too. I'll have a stronger engine but be poorer from hardware costs.

GCP - In a pre-release announcement, please report estimated elo gain from adding each additional core, for different chips.

Thanks

[M ANSARI]

A cluster version would really be interesting as it would be the ultimate equalizer against the strongest non cluster engines. I would be interested to know what cluster algo's are used, and if it uses a similar system like Rybka where all computers have to be equally endowed, or if this can work well even with different strength computers. Also if the algo is only DS specific or if other engines can be used.

[Gian-Carlo Pascutto]

I'l excited too. I'll have a stronger engine but be poorer from hardware costs.

It's cheaper to link up 2 older machines than to buy new hardware so I think it will be the opposite.

I would be interested to know what cluster algo's are used, and if it uses a similar system like Rybka where all computers have to be equally endowed, or if this can work well even with different strength computers.

From the descriptions I have seen about Rybka's algorithm, what I am doing is totally different. Deep Sjeng splits inside the search tree instead of just sending different root moves. This can cope with different speed clients, but of course adding slower clients to a very fast master will not help a lot due to diminishing returns.

Also if the algo is only DS specific or if other engines can be used.

I'm not sure what you mean due to how the question is phrased. The cluster algorithm is just an algorithm so it can be implemented in any engine. But you won't be able to use another engine as Deep Sjeng is the only one to support it.

[Highendman]

Also if the algo is only DS specific or if other engines can be used.

I'm not sure what you mean due to how the question is phrased. The cluster algorithm is just an algorithm so it can be implemented in any engine. But you won't be able to use another engine as Deep Sjeng is the only one to support it.

GCP - can you share some details on the resiliency implementation - what happens if a server dies mid-search? and how dynamic have you implemented it - what would be the process of adding servers (probably not mid-run but some setup in advance with IP address and a client/server running on each slave machine)?

As this is going to be the first commercial available Cluster-Deep engine, there's quite a lot of excitement / curiosity on this as you can imagine.

I guess there's a lot of curiousness on how this scales - when do you start getting diminishing returns.

I believe Vas said he is not planning to release his Cluster version commercially.

[Gian-Carlo Pascutto]

GCP - can you share some details on the resiliency implementation - what happens if a server dies mid-search? and how dynamic have you implemented it - what would be the process of adding servers (probably not mid-run but some setup in advance with IP address and a client/server running on each slave machine)?

One computer (should be the fastest one) is the master and acts as a TCP/IP server. This is also the one that connects to the GUI (it behaves as a normal UCI engine as far as the GUI can tell). If this one dies, it's game over until you can restart it. (Clients will reconnect automatically once it's back)

The clients are entirely resilient. They can die, disconnect, lag, connect during a search, whatever. The master always recovers from that automatically.

As this is going to be the first commercial available Cluster-Deep engine, there's quite a lot of excitement / curiosity on this as you can imagine.

I guess there's a lot of curiousness on how this scales - when do you start getting diminishing returns.

Currently we see no meaningful difference going above 80 cores (and less at fast timecontrols), which really means only 6 nodes.

I'm not going to release it until I am entirely happy with it so don't expect anything anytime soon If the first thing that gets publicly released sucks people might be scared away from the idea.

[Suj]

Current small cluster we use for tourney-24x3+8x1

Sometimes I alternate to a 24x2+8x4 depending on network bandwidth and latency.

Bigger version of the this cluster is usually 24x5+8x1 and all 24 cores are identical namely 24 core amd 2.4ghz though I have recently upgraded to 2 bigger clusters.

As far as sjeng cluster base tourney cluster has always been 80-128 and depending on opponents I have played a tourney game with 96 cores too(one game in Leiden and one game in CCT).

GCP 200 elo gain is pretty modest but I will keep the figures to myself as it ever keeps changing.

Biggest thing with the cluster is determing the right settings which has taken me 5 months to find a half decent one and this does change if the cores change.

As far as running cluster on overclocked cpu, I have tried them on various pc and its actually detrimental for the cluster.

Max peak kns i have seen is 853MNps on a big work cluster but at home 304 Mnps.At tourney level we get anywhere between 80-140mbps depending on position/endgame/line etc.

Hope this helps.

[Zach Wegner]

One computer (should be the fastest one) is the master and acts as a TCP/IP server.

Does this mean that all the cluster split points are from the master's tree? Or can there be slave-to-slave splits?