1 core can be better than 6 cores (Stockfish)

[Joerg Oster]

Arasan 200 position testsuite. 15s per position

64/64 cores 180/200
1/64 cores 183/200

Yep...something is wrong.

No, nothing wrong.

[Werewolf]

Arasan 200 position testsuite. 15s per position

64/64 cores 180/200
1/64 cores 183/200

Yep...something is wrong.

No, nothing wrong.

Well justify that then. If the testsuite is sound (this needs investigating) and SF is optimised well for large thread counts (which it appears to be), it should have more search luck as threads go up.

That's what happens on every other testsuite I have tested.

But not this one. Most people are recording the same as well.

[Joerg Oster]

Arasan 200 position testsuite. 15s per position

64/64 cores 180/200
1/64 cores 183/200

Yep...something is wrong.

No, nothing wrong.

Well justify that then. If the testsuite is sound (this needs investigating) and SF is optimised well for large thread counts (which it appears to be), it should have more search luck as threads go up.

That's what happens on every other testsuite I have tested.

But not this one. Most people are recording the same as well.

With more than 1 thread Stockfish may choose a best move from one of the other threads after the search has finished.
This may lead to a better or worse result in a testsuite.
You may also experience different results for different testruns ...

[Werewolf]

Arasan 200 position testsuite. 15s per position

64/64 cores 180/200
1/64 cores 183/200

Yep...something is wrong.

No, nothing wrong.

Well justify that then. If the testsuite is sound (this needs investigating) and SF is optimised well for large thread counts (which it appears to be), it should have more search luck as threads go up.

That's what happens on every other testsuite I have tested.

But not this one. Most people are recording the same as well.

With more than 1 thread Stockfish may choose a best move from one of the other threads after the search has finished.
This may lead to a better or worse result in a testsuite.
You may also experience different results for different testruns ...

Thanks for your answer.
I accept that more than one thread is non-deterministic.

However, do you accept that more threads generally allows for a bigger search of the game tree and, as a result, typically results in what could be described as more "search luck", at least over plenty of time and a high number of positions?

I'm wondering that the issue here is that splitting the search over many threads at a short time control does not compensate for the search overheads introduced with many threads. Therefore, I'm wondering that if the test is repeated with a longer solve time (say one minute) and repeated several times and averaged - we should see the 64 cores as the clear winner?

[gordonr]

I tried the Arasan 200 tests. I used a recent Stockfish dev and a 15 sec analysis limit on my 5950X. With 1 thread, 178 solved. With 2 threads, 181 solved. I realise that this result with 2 threads could vary with repeated runs and I've not tried with more threads/time.

[gordonr]

I've now tried with 4 threads too but only a single run.

1 thread -> 178
2 threads -> 181
4 threads -> 183

So my very small sample with a small amount of threads isn't yet showing any surprises. Unfortunately I was using Arena and it failed to run all the tests when using Stockfish with 8 threads.

[Werewolf]

I think there is an issue with the Arasan Testsuite, unfortunately. Some positions are so easy they are found well under a second making it impossible to distingush between 1 core, 8 cores or 512 cores etc etc.

Some positions are cooked. For example, number 43 has an alternative move which is a fraction of a pawn less good than the main solution, but still very good.

I'm now moving on to Hai's suggestion.

[Vinvin]

Unfortunately I was using Arena and it failed to run all the tests when using Stockfish with 8 threads.

Why ? I used Arena with 16 and 32 threads without trouble.

[Werewolf]

Hai's testsuite is also cooked.

Seems like this discussion can't go anywhere until we have a testsuite with positions that one move clearly better than all the others.

[gordonr]

Unfortunately I was using Arena and it failed to run all the tests when using Stockfish with 8 threads.

Why ? I used Arena with 16 and 32 threads without trouble.

I'm using Arena 3.5. I've found this issue with Arena for a long time now.

If I manually step through an EPD set of positions using the "EPD+" button, it seems that Arena doesn't always successfully stop the current Stockfish analysis and start a new analysis for the new position. I sometimes have to stop the analysis myself and restart it. I see the same effect sometimes while doing the automated analysis for a test set.

I can use SF with 32 threads and step through a game with analysis without issues. I can play SF engine matches with no issues. It's not an SF issue but for some reason I see it mostly with SF and more threads. It's like when SF is using a lot of threads, Arena's code for sending UCI commands to stop/start SF has a timing issue or something?! And this may be totally coincidental, but if SF is accessing tablebases heavily, that increases the chances of it happening.