Stockfish-1.6.2 Benchmarks for 1 to 8 Threads

[mcostalba]

I plan to run a match of 1000 games at game/40s between stockfish-1.6.2-MSD4 (the official version) and stockfish-1.6.2-MSD7.

Also MSD6 looks good, perhaps even better, constantly it takes less time to finish the search (that at the end is what it counts).

[zullil]

I plan to run a match of 1000 games at game/40s between stockfish-1.6.2-MSD4 (the official version) and stockfish-1.6.2-MSD7.

Also MSD6 looks good, perhaps even better, constantly it takes less time to finish the search (that at the end is what it counts).

OK, I'll use MSD4 and MSD6 for the first 1000 game match.

Louis

[zullil]

Decided to test initially using 100 games, first for MSD=7 and later for MSD=6. Here's the first result:

Code: Select all

./cutechess-cli.sh -fcp cmd=/Users/louis/Desktop/stockfish-162MSD/src/stockfish-1.6.2-MSD4 name=Stockfish-1.6.2-MSD4 -scp cmd=/Users/louis/Desktop/stockfish-162MSD/src/stockfish-1.6.2-MSD7 name=Stockfish-1.6.2-MSD7 -both dir=/Users/louis/Desktop/stockfish-162MSD/src proto=uci tc=40 book=performance.bin bookdepth=10 option.Hash=512 option.Ponder=false option.OwnBook=false -event MSD7_TEST_MATCH -games 100 -repeat -site EastonPA -pgnout ~/Desktop/MSD7_TEST_MATCH.pgn

Score of Stockfish-1.6.2-MSD4 vs Stockfish-1.6.2-MSD7: 12 - 33 - 55
Finished match 

Note: Source was modified to make Threads=8 by default.

[zullil]

Code: Select all

./cutechess-cli.sh -fcp cmd=/Users/louis/Desktop/stockfish-162MSD/src/stockfish-1.6.2-MSD4 name=Stockfish-1.6.2-MSD4 -scp cmd=/Users/louis/Desktop/stockfish-162MSD/src/stockfish-1.6.2-MSD6 name=Stockfish-1.6.2-MSD6 -both dir=/Users/louis/Desktop/stockfish-162MSD/src proto=uci tc=40 book=performance.bin bookdepth=10 option.Hash=512 option.Ponder=false option.OwnBook=false -event MSD6_TEST_MATCH -games 100 -repeat -site EastonPA -pgnout ~/Desktop/MSD6_TEST_MATCH.pgn

Score of Stockfish-1.6.2-MSD4 vs Stockfish-1.6.2-MSD6: 15 - 24 - 61
Finished match 

[mcostalba]

I see you mastered cutechess quickly

I am really curious about the final result....

I know it is not good luck to say this so early, but I suspect that we will end up with a much stronger setup for 8 cores then current one.

[zullil]

Code: Select all

./cutechess-cli.sh -fcp cmd=/Users/louis/Desktop/stockfish-162MSD/src/stockfish-1.6.2-MSD6 name=Stockfish-1.6.2-MSD6 -scp cmd=/Users/louis/Desktop/stockfish-162MSD/src/stockfish-1.6.2-MSD7 name=Stockfish-1.6.2-MSD7 -both dir=/Users/louis/Desktop/stockfish-162MSD/src proto=uci tc=40 book=performance.bin bookdepth=10 option.Hash=512 option.Ponder=false option.OwnBook=false -event MSD67_TEST_MATCH -games 100 -repeat -site EastonPA -pgnout ~/Desktop/MSD67_TEST_MATCH.pgn

Score of Stockfish-1.6.2-MSD6 vs Stockfish-1.6.2-MSD7: 16 - 19 - 65
Finished match 

[zullil]

I see you mastered cutechess quickly

I am really curious about the final result....

I know it is not good luck to say this so early, but I suspect that we will end up with a much stronger setup for 8 cores then current one.

Hi Marco,

Based on the three 100-game match results posted above, what match would you like me to run next? MSD4--MSD7 for 1000 games? MSD6--MSD7 for 1000 games?

Let me know.

Louis

[Joerg Oster]

Hi Louis
Hi Marco,

first a great thanks for your efforts Louis. Interesting results so far.

But I have a question.
You left Max. Num. of Threads per Split Point = 5 in your tests as you wrote above. Is that right?

Does this really make sense, Marco?
Shouldn't Max. Num. at least be set to 8 to ensure all the 8 cores get some work? Or am I getting something totally wrong?

Joerg.

[Tord Romstad]

Hi Louis
Hi Marco,

first a great thanks for your efforts Louis. Interesting results so far.

But I have a question.
You left Max. Num. of Threads per Split Point = 5 in your tests as you wrote above. Is that right?

Does this really make sense, Marco?
Shouldn't Max. Num. at least be set to 8 to ensure all the 8 cores get some work? Or am I getting something totally wrong?

This is difficult to explain without explaining all the complex details of the YBW algorithm, but all 8 cores will almost always be busy doing work even if the maximum number of threads per split point is less than 8. They will just be busy searching more distant parts of the search tree.

[zullil]

Hi Louis
Hi Marco,

first a great thanks for your efforts Louis. Interesting results so far.

But I have a question.
You left Max. Num. of Threads per Split Point = 5 in your tests as you wrote above. Is that right?

Does this really make sense, Marco?
Shouldn't Max. Num. at least be set to 8 to ensure all the 8 cores get some work? Or am I getting something totally wrong?

This is difficult to explain without explaining all the complex details of the YBW algorithm, but all 8 cores will almost always be busy doing work even if the maximum number of threads per split point is less than 8. They will just be busy searching more distant parts of the search tree.

I did indeed leave MNTpSP=5. I also monitored CPU during some initial testing, and indeed each program was at 800% (=8 cores in use) almost always.

If Tord is still around---should I do some benchmarking/testing with larger values of MNTpSP? Crude early testing seemed to indicate that changing this parameter had little effect on nps, compared to changes in MSD. I suppose it can't hurt to try.