How to test a 1-core engines in paralell on a multicore PC?

[bob]

Can you solve this problem by just setting the Hash memory to some low value, e.g., 32M?

That works too, but most of the time you have to watch for CPU usage,... so one program doesn't get more time than the other...

I like to recompile with small values for fast games...

For Jacob's case, there would be no problems...

There could be problems since hash is just a small part of total memory bandwidth usage.

[bob]

Clearly wrong assessments. Bob is right.

Clearly, you must think we are still in the 1990's

No. The PC has gotten _worse_. CPU speeds have grown dramatically since the 90's. Memory is not any better now than it was back then in terms of latency and bandwidth, relatively speaking, when compared to the CPU speed improvements. So the memory problem has gotten worse, not better.

[bob]

You can use winboard, with Crafty and such... Of course, you'll have 4 GUI's open, but you can make them tiny... pretty simply too.

I've ran 30 Winboards, with 60 Crafty executables running, on a cheap i7 processor, and played 450k games in a couple days, with no memory or chess engine congestion problems...

Save all the games to the same pgn file. After the end of the (/mg x) games it will display the results in a dialog pop-up (comp1 wins - comp2 wins - draws)

If you run 60 crafty's in a single machine you have _huge_ memory issues and don't realize it. One way to measure is to run 1 engine for 60 secs and measure NPS. Then run two at once. Measure NPS of both. Probably no big change. Then run 4. NPS will drop. Ditto for 8, 16, 32.

Threads would be a bit better since 32 threads would have one copy of the instructions in memory and they could sit in cache. But not with 32 different processes.

[Don]

Hi, I'm wondering if anyone knows if there is a software where I can play 4 games in paralell on a 4-CPU PC where the engines only use 1 CPU each, and no pondering. At the moment I'm starting multiple instances of Arena, and adding the scores afterwards. No big deal, but it would be nice if there's a software administrating the total score in these simultanious matches.

Regards
Jacob

I will soon be releasing an open source tester that will do this. I already have my own tester which does this but it only works with Linux. The one I am going to release is Java, but with no GUI as I want it to be really fast. It may have a separate configuration GUI tool for configuring it more easily.

It does all the scheduling and multiplexing for you and you will like it.

[jhaglund]

If you run 60 crafty's in a single machine you have _huge_ memory issues and don't realize it. One way to measure is to run 1 engine for 60 secs and measure NPS. Then run two at once. Measure NPS of both. Probably no big change. Then run 4. NPS will drop. Ditto for 8, 16, 32.

Threads would be a bit better since 32 threads would have one copy of the instructions in memory and they could sit in cache. But not with 32 different processes.

They all run pretty equally the same. The i7 is great for multitasking... The 8 threads are all used.

If a PC would run out of RAM, then I could see a problem, but otherwise everything seems to be shared pretty equal.

No. The PC has gotten _worse_. CPU speeds have grown dramatically since the 90's. Memory is not any better now than it was back then in terms of latency and bandwidth, relatively speaking, when compared to the CPU speed improvements. So the memory problem has gotten worse, not better.

You're kidding? haha.

Multiple cores/threads is where that is headed for now. i7 - QPI.

While the CL has indeed increased, they solved problems with interleaving. Meaning that each module is accessed sequentially. Data is spread amongst the modules in an alternating pattern, like Triple-Channel. 50x the bandwidth... than 90's...

Very good links to compare bandwidth for CPU, Memory, etc. :
http://en.wikipedia.org/wiki/List_of_device_bandwidths
http://en.wikipedia.org/wiki/Memory_latency

Though, not everyone has the newest/latest computer hardware to use...
I'm sure this is what you have taken into consideration...

4CPUs without ponder = walk in the park.

[bob]

If you run 60 crafty's in a single machine you have _huge_ memory issues and don't realize it. One way to measure is to run 1 engine for 60 secs and measure NPS. Then run two at once. Measure NPS of both. Probably no big change. Then run 4. NPS will drop. Ditto for 8, 16, 32.

Threads would be a bit better since 32 threads would have one copy of the instructions in memory and they could sit in cache. But not with 32 different processes.

They all run pretty equally the same. The i7 is great for multitasking... The 8 threads are all used.

If a PC would run out of RAM, then I could see a problem, but otherwise everything seems to be shared pretty equal.

No. The PC has gotten _worse_. CPU speeds have grown dramatically since the 90's. Memory is not any better now than it was back then in terms of latency and bandwidth, relatively speaking, when compared to the CPU speed improvements. So the memory problem has gotten worse, not better.

You're kidding? haha.

Multiple cores/threads is where that is headed for now. i7 - QPI.

I realized that before anyone else it seemed. Check out which PC program was the first to do a parallel search. But memory latency today is no better than 20 years ago, within a factor of 2 or so. CPU demand for latency is a million times worse.

While the CL has indeed increased, they solved problems with interleaving. Meaning that each module is accessed sequentially. Data is spread amongst the modules in an alternating pattern, like Triple-Channel. 50x the bandwidth... than 90's...

Interleaving only addresses bandwidth specifically. Does zero for latency. We are not pumping huge blocks of memory thru a chess engine. We are doing random accesses all over the place. And more cores just pile up behind a very slow memory system. Cache helps. But it is not a cure-all.

Very good links to compare bandwidth for CPU, Memory, etc. :
http://en.wikipedia.org/wiki/List_of_device_bandwidths
http://en.wikipedia.org/wiki/Memory_latency

Though, not everyone has the newest/latest computer hardware to use...
I'm sure this is what you have taken into consideration...

4CPUs without ponder = walk in the park.

[/quote]

Again, it depends on the programs involved. Just write a program that loops over (say) 128 megs of ram. Run that with your favorite chess engine and see what happens to the NPS. run two of 'em with your favorite engine. Bandwidth is not infinite and latency is not zero. It adds up quite quickly.

I ran and tested extensively on an 8-core nehalem box. I see NPS drops, although Crafty is hardly the most memory intensive engine around. I've spent years trying to minimize those things that are inefficient, and memory accesses is one of those.

The main issue is that a one core box has a given bandwidth. Add 3 more cores and the bandwidth stays exactly the same, yet it is shared between 4 cores... If you run memory intensive apps, the 4 cores will bog down badly, if you use better algorithms, then the bandwidth limit is not so formidable.

[nthom]

http://www.kimiensoftware.com/littletho ... litzer.php

You beat me to it I wrote this for exactly this purpose, I use it for my LittleThought testing and it was the main reason my last version jumped so high so quickly - because I was no longer making random changes

The version up there now is a bit limited, I will upload my latest code tonight which fixes a lot of issues and is a bit more flexible.

[jacobbl]

Thanks for the advice, this is exactly what I'm looking for except that it doesn't support winboard engines
My engine Sjakk doesn't support the UCI protocoll...

Regards Jacob

[jacobbl]

This sounds really interesting I'm looking forard to try it out. I hope it will support the winboard protocoll, cause my engine doesn't support UCI.

Regards Jacob

[nthom]

I have uploaded v2.2 of LittleBlitzer to my site http://kimiensoftware.com/littlethought ... litzer.php.