Stockfish-1.6.2 Benchmarks for 1 to 8 Threads

[mcostalba]

It is tempting to wonder what might happen with Minimum Split Depth = 8 or 9 or 10.

Hi Louis,

it is easy to raise the allowed maximum. In file ucioption.cpp you will find how the options are defined.

Code: Select all

o["Minimum Split Depth"] = Option(4, 4, 7);
o["Maximum Number of Threads per Split Point"] = Option(5, 4, 8);

The first number (4 for Minimum Split Depth) is the default value, the second number (4) is the minimum allowed value and the third one (7) is the maximum allowed.

So you just need to change the 7 in say 50, recompile and then you should be able to set the UCI parameter high as you want.

Hope it is clear.

[mcostalba]

Hi Marco,

if you come up with new settings for 4 cores - I am happy to run a decent length gauntlet.

(I already have the gauntlet run with default 1.6 to compare with).

Would the same setting benefit both long and short time controls or would that need testing too?

Hi Shaun,

unfortunatly I don't have a QUAD to do some speed tests, so, if you are interested, I suggest to setup a table like the one Louis has done for the 8 cores case:

Code: Select all

Threads = 8, Hash = 1024
position fen = 8/4r1nR/6P1/3B1k2/1pK5/1P6/8/8 w - -
go depth 20


Table giving Nodes/Second as a function of Minimum Split Depth and Maximum Number of Threads per Split Point

                                      Max Num Threads per Split Point
			   
	          		            4          5          6          7          8 
			      
Minimum Split Depth = 4    4685588    5679449    5723323    5936101    6214629

Minimum Split Depth = 5    5786147    6950683    6725553    6923536    6735906

Minimum Split Depth = 6    6645197    7229128    7377308    7127838    7216431

Minimum Split Depth = 7    7337907    7785847    7538651    7814227    7694497

He made 20 runs with different combination of the two parameters, one minute or less per run, if you choose depth 12-13 instead of 20 then it should take about 30 seconds per run and results should be similar.

IMPORTANT NOTE: Because you have a QUAD then 4 threads are the maximum (you can set more but performance will be lower), so in your case the table will be half of that because you can set "Max Num Threads per Split Point" fixed to 4.

Then you pick up the setup that shows the highest nps, or if there are many setup with _similar_ nps then choose the more conservative one (that with the smaller parameters) and test that against standard 1.6.2 version.

Because it is just a speed test (the two versions are functional identical) I suggest to use a short time control, about 40 seconds per match (40''+0) and run about 1000 games, it will take one night more or less.

Thanks
Marco

[zullil]

There are two UCI parameters that could be tweaked to increase nps with 4 or 8 cores:

Code: Select all

"Minimum Split Depth"  (defaults to 4, maximum is 7)
"Maximum Number of Threads per Split Point" (defaults to 5, maximum is 8)

Changing those, possibly increasing, you should increase also nps at 4 or 8 cores.

Of course it doesn't mean that an increased nps yields to a stronger engine because the added nps could be used with less efficency (wasted) so at the end a test match with hundreds of games between the stock version 1.6.2 and the one with modified parameters is necessary once you find the possible best recipe for 4 and the possible best recipe for 8 cores.

Hi Marco

More meager data--with 8 threads, Stockfish-1.6.2(with Min. Split Depth=7 and Max. Num. of Threads per Split Point=5) defeated Stockfish-1.6.2(with Min. Split Depth=4 and Max. Num. of Threads per Split Point=5) by a score of 35.5-24.5 in 60 games at 3m+1s with no book and no TBs.

(I decided to leave Max. Num. of Threads per Split Point=5, since it isn't clear to me that raising it to 7 raises nps in a statistically meaningful way.)

Raising Min. Split Depth to 7 seems to have enhanced endgame play.

[zullil]

It is tempting to wonder what might happen with Minimum Split Depth = 8 or 9 or 10.

Hi Louis,

it is easy to raise the allowed maximum. In file ucioption.cpp you will find how the options are defined.

Code: Select all

o["Minimum Split Depth"] = Option(4, 4, 7);
o["Maximum Number of Threads per Split Point"] = Option(5, 4, 8);

The first number (4 for Minimum Split Depth) is the default value, the second number (4) is the minimum allowed value and the third one (7) is the maximum allowed.

So you just need to change the 7 in say 50, recompile and then you should be able to set the UCI parameter high as you want.

Hope it is clear.

Yes, I discovered this on my own too. May experiment a bit with raising that Min. Split Depth a bit more. Not sure that changing Maximum Number of Threads per Split Point is that significant.

Louis

[mcostalba]

Hi Marco

More meager data--with 8 threads, Stockfish-1.6.2(with Min. Split Depth=7 and Max. Num. of Threads per Split Point=5) defeated Stockfish-1.6.2(with Min. Split Depth=4 and Max. Num. of Threads per Split Point=5) by a score of 35.5-24.5 in 60 games at 3m+1s with no book and no TBs.

Hi Louis,

this is very promising !

In case you are interested it would be great if you could test with the same setup as above but at 40"+0 time control for about 1000 games and _with_ a balanced book, although a limited one, let's say to about 10 moves. You can use the ones used in public tests, at your choice.

This would be very valuable data, and if your setup will succeed I will teach SF about the optimal parameters in the next version of SF, so that on an 8 core SF will automatically set to the new best parameters.


Marco

[zullil]

Hi Marco

More meager data--with 8 threads, Stockfish-1.6.2(with Min. Split Depth=7 and Max. Num. of Threads per Split Point=5) defeated Stockfish-1.6.2(with Min. Split Depth=4 and Max. Num. of Threads per Split Point=5) by a score of 35.5-24.5 in 60 games at 3m+1s with no book and no TBs.

Hi Louis,

this is very promising !

In case you are interested it would be great if you could test with the same setup as above but at 40"+0 time control for about 1000 games and _with_ a balanced book, although a limited one, let's say to about 10 moves. You can use the ones used in public tests, at your choice.

This would be very valuable data, and if your setup will succeed I will teach SF about the optimal parameters in the next version of SF, so that on an 8 core SF will automatically set to the new best parameters.


Marco

I'm willing to help, but first I need to find a suitable GUI for use on my Mac. I have Sigma Chess (a fabulous program but now rather old), which doesn't allow time controls less than 1 minute. Any suggestions? I know Tord uses a Mac. Guess I'll do some Googling.

[zullil]

Perhaps I'll try to get Xboard + PolyGlot up and working for me.

[mcostalba]

I'm willing to help, but first I need to find a suitable GUI for use on my Mac. I have Sigma Chess (a fabulous program but now rather old), which doesn't allow time controls less than 1 minute. Any suggestions? I know Tord uses a Mac. Guess I'll do some Googling.

cutechess is very good I don't know if there is a version also for Mac.

[Joerg Oster]

Hi Marco,

if you come up with new settings for 4 cores - I am happy to run a decent length gauntlet.

(I already have the gauntlet run with default 1.6 to compare with).

Would the same setting benefit both long and short time controls or would that need testing too?

Hi Shaun,

unfortunatly I don't have a QUAD to do some speed tests, so, if you are interested, I suggest to setup a table like the one Louis has done for the 8 cores case:

He made 20 runs with different combination of the two parameters, one minute or less per run, if you choose depth 12-13 instead of 20 then it should take about 30 seconds per run and results should be similar.

IMPORTANT NOTE: Because you have a QUAD then 4 threads are the maximum (you can set more but performance will be lower), so in your case the table will be half of that because you can set "Max Num Threads per Split Point" fixed to 4.

Then you pick up the setup that shows the highest nps, or if there are many setup with _similar_ nps then choose the more conservative one (that with the smaller parameters) and test that against standard 1.6.2 version.

Because it is just a speed test (the two versions are functional identical) I suggest to use a short time control, about 40 seconds per match (40''+0) and run about 1000 games, it will take one night more or less.

Thanks
Marco

Hi Marco,

of course, you get different results on different positions!
So I doubt it is a good idea to test this on only 1 position.

Btw. my tests on my Quad Q6600 indicate, that the default setting is not the worst ...

Regards,
Joerg.

[zullil]

Hi Marco,

of course, you get different results on different positions!
So I doubt it is a good idea to test this on only 1 position.

Btw. my tests on my Quad Q6600 indicate, that the default setting is not the worst ...

Regards,
Joerg.

Probably not a good idea. Just a first effort. I'll do some speed testing using the default positions in the bench command.