Norman, I feel like you have done this test knowing the results beforehand ...kranium wrote: ↑Wed Aug 22, 2018 12:34 pm
I've taken some time to run some tests and gather NPS for Ethereal (as you requested) and Stockfish.
I ran them each with 46 and 92 threads for 10 seconds:
(I'm presenting only the last PV output before 'bestmove')
Code: Select all
Ethereal 10.86 (bmi2) setoption name Threads value 46 info string set Threads to 46 go movetime 10000 info depth 25 seldepth 32 score cp 24 time 3125 nodes 206966264 nps 66208000 tbhits 0 hashfull 996 pv d2d4 g8f6 c2c4 e7e6 g1f3 d7d5 b1c3 f8b4 c4d5 e6d5 c1g5 e8g8 e2e3 b8d7 f1d3 c7c5 e1g1 b4c3 b2c3 c5c4 d3c2 d8a5 g5f4 f6e4 c2e4 bestmove d2d4 ponder g8f6 setoption name Threads value 92 info string set Threads to 92 go movetime 10000 info depth 22 seldepth 31 score cp 26 time 7984 nodes 533074064 nps 66759000 tbhits 0 hashfull 1000 pv d2d4 d7d5 g1f3 e7e6 c2c4 g8f6 b1c3 c7c5 c1g5 c5d4 f3d4 d5c4 e2e3 b8d7 f1c4 f8e7 e1g1 e8g8 c4e2 a7a6 d4f3 h7h6 g5f4 g7g5 bestmove d2d4 ponder d7d5 nps +1%
As you can see there's a big difference in how effectively they scale above 40-50 threads.Code: Select all
Stockfish 130818 (bmi2) setoption name Threads value 46 go movetime 10000 info depth 27 seldepth 26 multipv 1 score cp 46 upperbound nodes 720732280 nps 72066021 hashfull 999 tbhits 0 time 10001 pv d2d4 g8f6 bestmove d2d4 ponder g8f6 setoption name Threads value 92 go movetime 10000 info depth 26 seldepth 32 multipv 1 score cp 45 nodes 916696131 nps 91660447 hashfull 999 tbhits 0 time 10001 pv d2d4 g8f6 bestmove d2d4 ponder g8f6 nps +27%
Ethereal is scaling quite poorly and Stockfish doing much better...perhaps due to Lazy SMP as some are suggesting, but even with that 27% increase, we're talking about an relatively insignificant net Elo gain of approx +25 Elo. Not really an efficient use of the resources if you ask me, especially since some engine may not benefit at all.
I'm sure you know that on a Windows machine, without special Numa code to handle 64+ threads, the threads are left on the same processor group.
Furthermore, if I were to send the latest Ethereal, that Numa support does exist, and Ethereal would exhibit the same scaling as Stockfish.