Tuning Min Split Depth in Stockfish

[zullil]

Currently, Stockfish defaults to Min Split Depth = 7 whenever Threads ≥ 8. My suspicion is that the default setting of 7 is likely suboptimal for Threads = 16 (and probably even for Threads = 12).

Suppose I had a 16-core machine and I wanted to find the "best" value for Min Split Depth (for use in SF when doing deep searches, not blitz games). My inclination would be to modify the code in benchmark.cpp like this:

Code: Select all

LZsMacPro-OSX6: ~/Documents/Chess/Stockfish/src] diff benchmark.cpp_orig benchmark.cpp
83,85c83,85
<   string ttSize    = (is >> token) ? token : "32";
<   string threads   = (is >> token) ? token : "1";
<   string limit     = (is >> token) ? token : "13";
---
>   string ttSize    = (is >> token) ? token : "4096";
>   string threads   = (is >> token) ? token : "16";
>   string limit     = (is >> token) ? token : "35";
90a91,92
>   Options["Min Split Depth"] = 7; // Vary this to tune
>   

and build a Stockfish binary.

I'd then run Stockfish's bench, and get an output like this one (which came from my current 8-core machine):

Code: Select all

===========================
Total time (ms) : 1339757
Nodes searched  : 9722597395
Nodes/second    : 7256985

Here's the question: Should I aim to minimize the time-to-depth (Total time) or to maximize the nodes/sec? (Nodes searched varies a lot from run to run.)

My inclination is minimize time to depth. Of course, whatever I do, I'd take an average of several runs of the bench command, and then I'd increment the Min Split Depth value, rebuild and repeat.

Thanks for any advice.

[Daniel Shawul]

Currently, Stockfish defaults to Min Split Depth = 7 whenever Threads ≥ 8. My suspicion is that the default setting of 7 is likely suboptimal for Threads = 16 (and probably even for Threads = 12).

Note that the split depth does not necessarily have to increase with number of processors. If the time it takes to split (copying board & other stuff) is small enough, one can split at shallow depths for all number of processors. If the split_depth=7 was already a tuned value, then using that may be better. Infact increasing it may hurt because keeping many processors busy gets harder with processors, thus a smaller split depth will only help you by giving more chances to split. I am not saying that is the case here, but just wondering about your motive to increase it.

[Eelco de Groot]

I don't quite see, why you would have to change the sources Louis to measure optimum Min Split Depth? You can set the Min Split Depth in the engine options and you can vary the depth for the bench I'm sure of that, and also I think number of threads. Especially in Linux this is easier to do because you don't have to navigate with the Dos command window to the executable?

See also this thread http://www.talkchess.com/forum/viewtopic.php?t=49231

Don't know much about the bench code or multithreading code but nodes per second is not the thing you'd want to optimize I suspect. Total nodes to depth may be better and I would combine that with total time to depth. Total nodes to depth means you don't have to bother with processes in the background, and total time to depth measures the real speed but varies a bit if there are background tasks. But that averages out if you take many measurements. bench inside the comand window is nice for Windows but in Linux it is easier to add the extra parameters, although I could be mistaken about that? I only ever use bench, with the default parameters.

Regards, Eelco

[zullil]

Currently, Stockfish defaults to Min Split Depth = 7 whenever Threads ≥ 8. My suspicion is that the default setting of 7 is likely suboptimal for Threads = 16 (and probably even for Threads = 12).

Note that the split depth does not necessarily have to increase with number of processors. If the time it takes to split (copying board & other stuff) is small enough, one can split at shallow depths for all number of processors. If the split_depth=7 was already a tuned value, then using that may be better. Infact increasing it may hurt because keeping many processors busy gets harder with processors, thus a smaller split depth will only help you by giving more chances to split. I am not saying that is the case here, but just wondering about your motive to increase it.

There seems to be some evidence that increasing Min Split Depth leads to increased nps, at least for Threads = 16. See here: http://talkchess.com/forum/viewtopic.ph ... 96&t=50759.
Not sure if this will translate into better ELO!

I think the default value of 7 resulted from testing I did for Marco long ago on my then new 8-core machine. That machine is now old, and I'm not sure any testing has been done for new versions of SF on new hardware.

[zullil]

I don't quite see, why you would have to change the sources Louis to measure optimum Min Split Depth?

Regards, Eelco

Thanks. I didn't realize that I can now run the bench command from within stockfish, after setting the Min Split Depth value. Though editing the code and recompiling was simple too.

[bob]

Currently, Stockfish defaults to Min Split Depth = 7 whenever Threads ≥ 8. My suspicion is that the default setting of 7 is likely suboptimal for Threads = 16 (and probably even for Threads = 12).

Suppose I had a 16-core machine and I wanted to find the "best" value for Min Split Depth (for use in SF when doing deep searches, not blitz games). My inclination would be to modify the code in benchmark.cpp like this:

Code: Select all

LZsMacPro-OSX6: ~/Documents/Chess/Stockfish/src] diff benchmark.cpp_orig benchmark.cpp
83,85c83,85
<   string ttSize    = (is >> token) ? token : "32";
<   string threads   = (is >> token) ? token : "1";
<   string limit     = (is >> token) ? token : "13";
---
>   string ttSize    = (is >> token) ? token : "4096";
>   string threads   = (is >> token) ? token : "16";
>   string limit     = (is >> token) ? token : "35";
90a91,92
>   Options["Min Split Depth"] = 7; // Vary this to tune
>   

and build a Stockfish binary.

I'd then run Stockfish's bench, and get an output like this one (which came from my current 8-core machine):

Code: Select all

===========================
Total time (ms) : 1339757
Nodes searched  : 9722597395
Nodes/second    : 7256985

Here's the question: Should I aim to minimize the time-to-depth (Total time) or to maximize the nodes/sec? (Nodes searched varies a lot from run to run.)

My inclination is minimize time to depth. Of course, whatever I do, I'd take an average of several runs of the bench command, and then I'd increment the Min Split Depth value, rebuild and repeat.

Thanks for any advice.

Time to depth is all that matters. Just like in a car, the time to get from A to B is more important than the tachometer reading...

BTW in all my testing, "split depth" (which Crafty has not used in several years) is really something that varies according to the hardware used more than the number of cores. I have another term that is more useful as threads go up, which is "thread group". That limits the number of threads that can split at one split point. I typically use 4. Instead of split depth, I use "split nodes" where I won't split the tree until N nodes have been searched by the thread that wants to further split the search. "min split depth" needs to vary depending on whether you are in the middle game or endgame (being larger in endgame). Split nodes corrects for that as in the endgame, you can't split near the tips because the trees are too small to satisfy the node count...

[bob]

Currently, Stockfish defaults to Min Split Depth = 7 whenever Threads ≥ 8. My suspicion is that the default setting of 7 is likely suboptimal for Threads = 16 (and probably even for Threads = 12).

Note that the split depth does not necessarily have to increase with number of processors. If the time it takes to split (copying board & other stuff) is small enough, one can split at shallow depths for all number of processors. If the split_depth=7 was already a tuned value, then using that may be better. Infact increasing it may hurt because keeping many processors busy gets harder with processors, thus a smaller split depth will only help you by giving more chances to split. I am not saying that is the case here, but just wondering about your motive to increase it.

There seems to be some evidence that increasing Min Split Depth leads to increased nps, at least for Threads = 16. See here: http://talkchess.com/forum/viewtopic.ph ... 96&t=50759.
Not sure if this will translate into better ELO!

I think the default value of 7 resulted from testing I did for Marco long ago on my then new 8-core machine. That machine is now old, and I'm not sure any testing has been done for new versions of SF on new hardware.

That's really backward. By increasing min split depth, you make processors wait more until a thread can back up far enough to satisfy that depth limit. That should LOWER the NPS, but up until some point, should also improve time to depth.