Off-the-shelf PC spec, engine development

[smatovic]

Money now vs time later.
The 3990x will let you test twice as fast.

Seriously so?

What is happening faster in the 3990X to double speed? Is this an across the board doubling, or only in some specific type of test?

if a general chess engine NPS doubler, then that would be interesting, I guess

Doubling of cores - testing is twice as fast ( twice as many games are completed in a given time period )- with 64 cores (3990x ), I strongly recommend 256 GB RAM - nothing less than 128 GB for sure. $2000 upgrade to 3990x is a good deal - some builders in the US will charge you $2500 more for the 3990x. Don't expect a huge Elo Jump, going from 32 to 64 core is at most 30 Elo , most likely less. It's really for those that want to cut their testing time in half.

Ah! Mike, thanks for that, I didn't spot the small print from 32x to 64x
okay, now I understand the 2000 euros extra

and whoosh, now with 256 Gb RAM it's at 7650 euros

With the 3970x , which I have , the 128 GB is fine - a large hash , say 65536M hash is generally full in under 10 minutes, so a 64 core unit would definitely benefit with a larger hash. I if you get 128 GB, get the bigger sticks so your builder will leave half the RAMS slots opens so you can always upgrade RAM later without incurring additional costs- i'e. that is replacing 8x16GB sticks with 8x32GB sticks . Just get 4x32 GB sticks up front, then you only need 4x32 Gb sticks to complete the upgrade to 256 GB.

Probably going CoronaVirus crazy, I figured the 5000 euros was “expensive”, and even considering paying the price of a small car seems nuts. Will restore sanity shortly.

If it is only about benchmarking, NPS scaling and alike, you can restore your sanity via cloud like AWS or GCP....

--
Srdja

[Alayan]

For chess engine testing, the 3990X will be maybe 1.9X as fast as the 3970X or so, as running a lot of concurrent single-threaded games is embarrassingly parallel and you only lose a small amount of clockspeed.

If the 3970X config cost 5000€ but the 3990X config cost 7000€, the test throughput/€ will be significantly superior with the 3990X in addition to the raw throughput.

When you really have a lot of hw power available, it can be difficult to feed the test system, but you can always increase TC and/or confidence margins for tests to not complete too fast while still reaping benefits from additional hardware power. In many situations, however, there is less motivation to write a patch if you already have one in testing and it's not going to complete before many hours. Getting a quicker test result makes the development experience more enjoyable imo.

[MikeB]

Money now vs time later.
The 3990x will let you test twice as fast.

Seriously so?

What is happening faster in the 3990X to double speed? Is this an across the board doubling, or only in some specific type of test?

if a general chess engine NPS doubler, then that would be interesting, I guess

Doubling of cores - testing is twice as fast ( twice as many games are completed in a given time period )- with 64 cores (3990x ), I strongly recommend 256 GB RAM - nothing less than 128 GB for sure. $2000 upgrade to 3990x is a good deal - some builders in the US will charge you $2500 more for the 3990x. Don't expect a huge Elo Jump, going from 32 to 64 core is at most 30 Elo , most likely less. It's really for those that want to cut their testing time in half.

Ah! Mike, thanks for that, I didn't spot the small print from 32x to 64x
okay, now I understand the 2000 euros extra

and whoosh, now with 256 Gb RAM it's at 7650 euros

With the 3970x , which I have , the 128 GB is fine - a large hash , say 65536M hash is generally full in under 10 minutes, so a 64 core unit would definitely benefit with a larger hash. I if you get 128 GB, get the bigger sticks so your builder will leave half the RAMS slots opens so you can always upgrade RAM later without incurring additional costs- i'e. that is replacing 8x16GB sticks with 8x32GB sticks . Just get 4x32 GB sticks up front, then you only need 4x32 Gb sticks to complete the upgrade to 256 GB.

Probably going CoronaVirus crazy, I figured the 5000 euros was “expensive”, and even considering paying the price of a small car seems nuts. Will restore sanity shortly.

—
Agreed - I went through some serious twisted rationales to justify $5000 — somehow my wife was ok with it. I guess she loves me. ;>) In another twisted rationale, I feel good that I did not go for the 3990x — that would have created buyer's remorse ...

[Dann Corbit]

When buying a computer, it is always a question of money and/or time.
For instance, if you buy a computer that costs 1/3 less but is 1/3 slower, then you can get the same data by waiting 1/3 longer.

Now, looking at it the other way, if I can buy a computer than costs twice as much but is twice as fast, then I can calculate twice as much data in the same unit of time.

So which is more valuable? Time or money?

It will always be a judgement call.

I also have inexpensive computers that are quite powerful. One of them has 64 cores and was around 7 or 8 hundred dollars. To use it in a house, a sound proof box is necessary (if you even want to be able to carry on a converstion in the room where the computer is when it is going full tilt). So another 8 hundred for a sound proof box made for that purpose. So $1600 gave me 47M NPS. But then there is the power consumed. With that machine, which has 4x1200 watt power supplies, it will heat a room in a hurry and pulls electricity like several steam iron's on the highest setting. So you pay a lot here too, but amortized as power consumed.

I guess i would put it this way, for some of us, computer chess is a hobby, and for others of us, it is an illness.

[chrisw]

When buying a computer, it is always a question of money and/or time.
For instance, if you buy a computer that costs 1/3 less but is 1/3 slower, then you can get the same data by waiting 1/3 longer.

Now, looking at it the other way, if I can buy a computer than costs twice as much but is twice as fast, then I can calculate twice as much data in the same unit of time.

So which is more valuable? Time or money?

Fine, but misses the target. The actual data you have twice as much of in the same time has no monetary value. Well, if it’s chess game and position data. You can’t exchange the data for money (nowadays anyway).
As Alayan says, the faster data creation you get is translated into faster feedback and thus more enjoyment of process.

It will always be a judgement call.

I also have inexpensive computers that are quite powerful. One of them has 64 cores and was around 7 or 8 hundred dollars. To use it in a house, a sound proof box is necessary (if you even want to be able to carry on a converstion in the room where the computer is when it is going full tilt). So another 8 hundred for a sound proof box made for that purpose. So $1600 gave me 47M NPS. But then there is the power consumed. With that machine, which has 4x1200 watt power supplies, it will heat a room in a hurry and pulls electricity like several steam iron's on the highest setting. So you pay a lot here too, but amortized as power consumed.

I guess i would put it this way, for some of us, computer chess is a hobby, and for others of us, it is an illness.

[Dann Corbit]

Fine, but misses the target. The actual data you have twice as much of in the same time has no monetary value. Well, if it’s chess game and position data. You can’t exchange the data for money (nowadays anyway).
As Alayan says, the faster data creation you get is translated into faster feedback and thus more enjoyment of process.

It's worth money to me. But your point is well taken, because I get a lot of enjoyment watching analysis, designing experiments, checking the results, etc.

[chrisw]

Workstations from Delta Computer in DE:

https://www.deltacomputer.com/workstation.html

From to, with and wo GP/GPU and so on.

***edit***

Ah, sorry, just realized shop is in German only, asusme then they operate only in DE.

--
Srdja

I've been offered this, it's re-conditioned workstation

HP Workstation Z840 met:

2 x Intel Xeon E5-2673V4 2.3-3.3GHz 20 Core with HT 80 Threads
256 GB RAM DDR4
4 TB WD SAS Harddisk
Enterprise SSD 500GB + bracket
nVidia Quadro K2200 4 GB met 1 x dvi en 2 x displayport
Optische Drive Type DVD+/-RW
Verbinding (USB/FW) 2x USB 2.0, 8x USB 3.0
Audio uitgangen Analoog
Verbinding (Ethernet) 1Gbps
Verbinding (overige) 2x PS/2, Serieel (9-pins)
Desktoptype Tower/desktop
Besturingssysteem Windows 10 Professional NL 64 bit
Kleur Zwart
Licht Gebruikt met 3 jaar Garantie
Including Shipment to France € 2979,00 ex. VAT.

What to make of this? I've never before had a PC with two processors, so no idea how that works, does threads.h know how to divide work up between 2x20 cores on two processors, or do I have explicitly code for two processors, or??

Windows NL might be a problem, I had a French PC once that was Windows F, and that config was extremely difficult to get round, all reboots (and a lot else) sent you back to French (which is ok, except when you want to use Help, or find some system menu item, technical words translated are not easy to search).

So, anybody know some kind of relative performance of those two processors compared to AMD 64x ?

[smatovic]

I've been offered this, it's re-conditioned workstation

HP Workstation Z840 met:

2 x Intel Xeon E5-2673V4 2.3-3.3GHz 20 Core with HT 80 Threads
256 GB RAM DDR4
4 TB WD SAS Harddisk
Enterprise SSD 500GB + bracket
nVidia Quadro K2200 4 GB met 1 x dvi en 2 x displayport
Optische Drive Type DVD+/-RW
Verbinding (USB/FW) 2x USB 2.0, 8x USB 3.0
Audio uitgangen Analoog
Verbinding (Ethernet) 1Gbps
Verbinding (overige) 2x PS/2, Serieel (9-pins)
Desktoptype Tower/desktop
Besturingssysteem Windows 10 Professional NL 64 bit
Kleur Zwart
Licht Gebruikt met 3 jaar Garantie
Including Shipment to France € 2979,00 ex. VAT.

What to make of this? I've never before had a PC with two processors, so no idea how that works, does threads.h know how to divide work up between 2x20 cores on two processors, or do I have explicitly code for two processors, or??

Windows NL might be a problem, I had a French PC once that was Windows F, and that config was extremely difficult to get round, all reboots (and a lot else) sent you back to French (which is ok, except when you want to use Help, or find some system menu item, technical words translated are not easy to search).

So, anybody know some kind of relative performance of those two processors compared to AMD 64x ?

If you run single thread engine-engine matches for testing I guess it pretty much does not matter, if you wish to scale nps with those CPUs above 20 cores I guess you have to consider NUMA (never been into this by myself):

https://www.chessprogramming.org/NUMA

Like I said, if it is only for benchmarking nps scaling I suggest to consider CPU/server renting in a cloud, develop local on a say 4 or 8 core machine, then bench in the cloud, some cloud providers offer per hour or even per minute rental.

Regarding HP/Dell/Lenovo brands, I personally prefer systems from system builders with known standard components, so I can upgrade/replace/repair cost-efficient and timely by myself.

***edit***
And you could consider to switch to a Linux based dev systems with mingw-cross-compiler for Windows and Wine emulator for running Windows executables, worked fine for me in Debian based Ubuntu.

--
Srdja

[Joost Buijs]

I've been offered this, it's re-conditioned workstation

HP Workstation Z840 met:

2 x Intel Xeon E5-2673V4 2.3-3.3GHz 20 Core with HT 80 Threads
256 GB RAM DDR4
4 TB WD SAS Harddisk
Enterprise SSD 500GB + bracket
nVidia Quadro K2200 4 GB met 1 x dvi en 2 x displayport
Optische Drive Type DVD+/-RW
Verbinding (USB/FW) 2x USB 2.0, 8x USB 3.0
Audio uitgangen Analoog
Verbinding (Ethernet) 1Gbps
Verbinding (overige) 2x PS/2, Serieel (9-pins)
Desktoptype Tower/desktop
Besturingssysteem Windows 10 Professional NL 64 bit
Kleur Zwart
Licht Gebruikt met 3 jaar Garantie
Including Shipment to France € 2979,00 ex. VAT.

What to make of this? I've never before had a PC with two processors, so no idea how that works, does threads.h know how to divide work up between 2x20 cores on two processors, or do I have explicitly code for two processors, or??

Windows NL might be a problem, I had a French PC once that was Windows F, and that config was extremely difficult to get round, all reboots (and a lot else) sent you back to French (which is ok, except when you want to use Help, or find some system menu item, technical words translated are not easy to search).

So, anybody know some kind of relative performance of those two processors compared to AMD 64x ?

I see it is a Dutch offer, they sell these refurbished systems ovverhere:

https://www.creoserver.com/hp-workstati ... gKv_vD_BwE

Two processors should not be a problem, even when you threat it like an SMP system, but for optimal performance you have to make your program NUMA aware.

These systems are probabely very noisy, and don't expect it to have the same performance as an AMD 3970X.

Windows 10 NL Pro should not be a problem because you can install the english language pack.

I have build a 3970X system myself, it's always cheaper when you build a system yourself and you have better control over the hardware you put into it.

IMHO it is nonsense to have more than 64 or 128 GB memory, my 3970X has 128 GB memory, and up until now I never needed more than 40 GB, ofcourse it solely depends upon what you want to do with it.

[chrisw]

I've been offered this, it's re-conditioned workstation

HP Workstation Z840 met:

2 x Intel Xeon E5-2673V4 2.3-3.3GHz 20 Core with HT 80 Threads
256 GB RAM DDR4
4 TB WD SAS Harddisk
Enterprise SSD 500GB + bracket
nVidia Quadro K2200 4 GB met 1 x dvi en 2 x displayport
Optische Drive Type DVD+/-RW
Verbinding (USB/FW) 2x USB 2.0, 8x USB 3.0
Audio uitgangen Analoog
Verbinding (Ethernet) 1Gbps
Verbinding (overige) 2x PS/2, Serieel (9-pins)
Desktoptype Tower/desktop
Besturingssysteem Windows 10 Professional NL 64 bit
Kleur Zwart
Licht Gebruikt met 3 jaar Garantie
Including Shipment to France € 2979,00 ex. VAT.

What to make of this? I've never before had a PC with two processors, so no idea how that works, does threads.h know how to divide work up between 2x20 cores on two processors, or do I have explicitly code for two processors, or??

Windows NL might be a problem, I had a French PC once that was Windows F, and that config was extremely difficult to get round, all reboots (and a lot else) sent you back to French (which is ok, except when you want to use Help, or find some system menu item, technical words translated are not easy to search).

So, anybody know some kind of relative performance of those two processors compared to AMD 64x ?

I see it is a Dutch offer, they sell these refurbished systems ovverhere:

https://www.creoserver.com/hp-workstati ... gKv_vD_BwE

Two processors should not be a problem, even when you threat it like an SMP system, but for optimal performance you have to make your program NUMA aware.

These systems are probabely very noisy, and don't expect it to have the same performance as an AMD 3970X.

Windows 10 NL Pro should not be a problem because you can install the english language pack.

I have build a 3970X system myself, it's always cheaper when you build a system yourself and you have better control over the hardware you put into it.

IMHO it is nonsense to have more than 64 or 128 GB memory, my 3970X has 128 GB memory, and up until now I never needed more than 40 GB, ofcourse it solely depends upon what you want to do with it.

Ok, thanks. I should consider also self-build, but like all complex things done for first time, there’s the anxiety effect to factor in, especially with high end new hardware, knowing what spec of components, forgetting to order some vital bits (or making sense of multiple cabling), I just tend to recoil in horror.

Well, engine1-engine2 matches, either if I write the code myself (Using Python chess for the handling), or Ferdy’s cutechess with concurrency handlers), is, I think, using one core/single thread/no ponder between e1 and e2 single match, scaled up by processes to N game pairs and N cores.
For 64 concurrent games requires scaling the RAM requirement for the two engines likewise. No doubt there are more efficient ways to do it, but if RAM for one game pair = 4Gb, there’s your 256Gb all used up