Lazy SMP

[Werewolf]

hence time to depth is the wrong metric here, Elo is the right one

And my claim is that the shorter time to depth the more elo, as it leaves more time on the clock so on other moves this engine will outsearch the one with longer time to depth.

Would like to see an example of an engine reaching lower depth and beating the higher depth one, all things being equal, more depth = more elo.

Your understanding of modern chess engines is pretty inexistent if depth is the metric you use

Yep.

I wonder if nps is now better than depth as a rough guide.

Chessify's 1000 core cluster stockfish has similar depths to a regular 8 core, but it's much faster at finding tactical shots

[MonteCarlo]

Speaking of Chessify, they actually did publish results of a similar-ish test, albeit with very low sample size at https://chessify.me/blog/nps-vs-time-to ... -stockfish

Given the massive score of 50-core vs 1-core at a fixed "depth"/iteration 20, I highly doubt that the results will flip in favor of 1-core if 50-core is limited to fixed "depth"/iteration 19.

I can also confirm something similar as Werewolf says about solve-times for difficult positions for the high-core-count Chessify options. They consistently solve very quickly and at a lower nominal "depth" than my PC.

Solving isolated difficult positions is obviously different than game play, but the evidence seems pretty strong that a high-core-count SF will outperform low-core-count SF even with nominal "depth" being lower. Just how much lower might be an interesting question, but given that the nominal "depth" is already a bit complicated these days, and given that lazy SMP clearly increases strength without necessarily improving time to depth, I'm not really sure why time-to-depth for lazy SMP engines across machines with very different configurations should be of much interest these days.

Cheers!

[Ovyron]

hence time to depth is the wrong metric here, Elo is the right one

And my claim is that the shorter time to depth the more elo, as it leaves more time on the clock so on other moves this engine will outsearch the one with longer time to depth.

Would like to see an example of an engine reaching lower depth and beating the higher depth one, all things being equal, more depth = more elo.

Your understanding of modern chess engines is pretty inexistent if depth is the metric you use

Yep.

In theory. Looks like nobody knows and people only talk about how they expect things to be. But if people are happy to reach conclusions without any testing then I guess more questions like this can be answered easily.

[Ronald]

I did a test with a multi threaded versions against a single thread version with diferent fixed depths. In rofChade helper threads are not bound to search beyond a fixed depth, so had to change it to make sure that helper threads do not iterate deeper than the fixed depth.

Code: Select all

Score of 3ThreadsDepth14 vs 1ThreadDepth15: 276 - 200 - 1524  [0.519] 2000
...      3ThreadsDepth14 playing White: 177 - 62 - 761  [0.557] 1000
...      3ThreadsDepth14 playing Black: 99 - 138 - 763  [0.480] 1000
...      White vs Black: 315 - 161 - 1524  [0.538] 2000
Elo difference: 13.2 +/- 7.4, LOS: 100.0 %, DrawRatio: 76.2 %

Score of 32ThreadsDepth12 vs 1ThreadDepth14: 328 - 185 - 1487  [0.536] 2000
...      32ThreadsDepth12 playing White: 225 - 58 - 717  [0.584] 1000
...      32ThreadsDepth12 playing Black: 103 - 127 - 770  [0.488] 1000
...      White vs Black: 352 - 161 - 1487  [0.548] 2000
Elo difference: 24.9 +/- 7.7, LOS: 100.0 %, DrawRatio: 74.4 %

Score of 32ThreadsDepth13 vs 1ThreadDepth15: 395 - 105 - 1500  [0.573] 2000
...      32ThreadsDepth13 playing White: 260 - 28 - 712  [0.616] 1000
...      32ThreadsDepth13 playing Black: 135 - 77 - 788  [0.529] 1000
...      White vs Black: 337 - 163 - 1500  [0.543] 2000
Elo difference: 50.7 +/- 7.4, LOS: 100.0 %, DrawRatio: 75.0 %

Score of 32ThreadsDepth13 vs 1ThreadDepth16: 240 - 184 - 1576  [0.514] 2000
...      32ThreadsDepth13 playing White: 168 - 57 - 775  [0.555] 1000
...      32ThreadsDepth13 playing Black: 72 - 127 - 801  [0.472] 1000
...      White vs Black: 295 - 129 - 1576  [0.541] 2000
Elo difference: 9.7 +/- 7.0, LOS: 99.7 %, DrawRatio: 78.8 %

With higher depth the difference seems to increase.

[schack]

On Chessify - how much is it being handicapped by having just 8gb hash for the 25-100m nps tier, and 16gb for anything bigger?

[Ovyron]

Thanks Ronald, it seems that time to depth no longer matters for modern engines, and I was happy to see this tested.