Stockfish benchmark data

[Adam Hair]

I am trying to collect Stockfish nodes per second data from the computer chess community and I hope to get plenty of volunteers. The information will be useful for comparing computer chess results between computers.

I have created a package that can be downloaded at http://www.mediafire.com/file/3ivy28eqx ... _bench.zip . In that package you will find the various official compiles of Stockfish 8 (and the source code) and directions for running a script that will collect nps data for 30 positions searched to depth 24. I am hoping that people will submit data for various numbers of CPUs, especially 1 CPU and the max number of CPUs their computer has. The process took 8 1/2 minutes using 1 CPU of my i7 3840QM processor.

I will be posting people'e submissions at http://adamsccpages.blogspot.com/p/stoc ... hmark.html . I will greatly appreciate it if you are willing to devote some time to generate the data.

Thanks,
Adam

[Ferdy]

I am trying to collect Stockfish nodes per second data from the computer chess community and I hope to get plenty of volunteers. The information will be useful for comparing computer chess results between computers.

I have created a package that can be downloaded at http://www.mediafire.com/file/3ivy28eqx ... _bench.zip . In that package you will find the various official compiles of Stockfish 8 (and the source code) and directions for running a script that will collect nps data for 30 positions searched to depth 24. I am hoping that people will submit data for various numbers of CPUs, especially 1 CPU and the max number of CPUs their computer has. The process took 8 1/2 minutes using 1 CPU of my i7 3840QM processor.

I will be posting people'e submissions at http://adamsccpages.blogspot.com/p/stoc ... hmark.html . I will greatly appreciate it if you are willing to devote some time to generate the data.

Thanks,
Adam

Would also be nice if you display the average time in the table.

[matthewlai]

I am trying to collect Stockfish nodes per second data from the computer chess community and I hope to get plenty of volunteers. The information will be useful for comparing computer chess results between computers.

I have created a package that can be downloaded at http://www.mediafire.com/file/3ivy28eqx ... _bench.zip . In that package you will find the various official compiles of Stockfish 8 (and the source code) and directions for running a script that will collect nps data for 30 positions searched to depth 24. I am hoping that people will submit data for various numbers of CPUs, especially 1 CPU and the max number of CPUs their computer has. The process took 8 1/2 minutes using 1 CPU of my i7 3840QM processor.

I will be posting people'e submissions at http://adamsccpages.blogspot.com/p/stoc ... hmark.html . I will greatly appreciate it if you are willing to devote some time to generate the data.

Thanks,
Adam

Happy to run it if there's a command line version for Linux. The tclkit script doesn't work for me even after installing tclkit.

Code: Select all

matthew@bigfoot:~/Downloads/Linux$ ./Stockfish_8_bench.sh 
couldn't read file "/home/matthew/Downloads/Linux/sim03.kit/main.tcl": no such file or directory
    while executing
"source /home/matthew/Downloads/Linux/sim03.kit/main.tcl"
    ("uplevel" body line 1)
    invoked from within
"uplevel [list source [file join $self main.tcl]]"

Is there a reason why this needs a GUI?

[Ferdy]

Stockfish does not recognize the Clear Hash command.

Code: Select all

Engine->Adapter: Unknown command: Clear Hash

It was sent like this,

Code: Select all

Adapter->Engine: Clear Hash

Perhaps this should have been,

Code: Select all

Adapter->Engine: setoption name Clear Hash

But since ucinewgame was sent before every position startpos ... perhaps the Hash was cleared.

[MikeB]

I am trying to collect Stockfish nodes per second data from the computer chess community and I hope to get plenty of volunteers. The information will be useful for comparing computer chess results between computers.

I have created a package that can be downloaded at http://www.mediafire.com/file/3ivy28eqx ... _bench.zip . In that package you will find the various official compiles of Stockfish 8 (and the source code) and directions for running a script that will collect nps data for 30 positions searched to depth 24. I am hoping that people will submit data for various numbers of CPUs, especially 1 CPU and the max number of CPUs their computer has. The process took 8 1/2 minutes using 1 CPU of my i7 3840QM processor.

I will be posting people'e submissions at http://adamsccpages.blogspot.com/p/stoc ... hmark.html . I will greatly appreciate it if you are willing to devote some time to generate the data.

Thanks,
Adam

No macOS. You also should have guidelines for hash when using multiple cores. For example using Hash 256, is not appropriate when using 12 cores, it would be full in no time. I would suggest Hash = # of physical cores tested times 256. Also I downloaded the official macOS from https://stockfishchess.org/download/ and will be running the test.

[MikeB]

I am trying to collect Stockfish nodes per second data from the computer chess community and I hope to get plenty of volunteers. The information will be useful for comparing computer chess results between computers.

I have created a package that can be downloaded at http://www.mediafire.com/file/3ivy28eqx ... _bench.zip . In that package you will find the various official compiles of Stockfish 8 (and the source code) and directions for running a script that will collect nps data for 30 positions searched to depth 24. I am hoping that people will submit data for various numbers of CPUs, especially 1 CPU and the max number of CPUs their computer has. The process took 8 1/2 minutes using 1 CPU of my i7 3840QM processor.

I will be posting people'e submissions at http://adamsccpages.blogspot.com/p/stoc ... hmark.html . I will greatly appreciate it if you are willing to devote some time to generate the data.

Thanks,
Adam

No macOS. You also should have guidelines for hash when using multiple cores. For example using Hash 256, is not appropriate when using 12 cores, it would be full in no time. I would suggest Hash = # of physical cores tested times 256. Also I downloaded the official macOS from https://stockfishchess.org/download/ and will be running the test.

Here are the mac versions of tclkit and polyglot - script is working.

https://www.dropbox.com/s/e2bamxmtxar4m ... S.zip?dl=1

[tpoppins]

Wake up Sedat, you got competition! ;)

Sent my benchmarks in, done on Win 7 Pro x64. The provided w32 exec won't run on Win XP SP3 32-bit, though. I can provide a baseline SSE-less w32 compile that will if there's interest.

A couple suggestions for your results table, Adam. It would look nicer and easier to read if it were modeled on that by Sedat.

Specifically, the NPS column should be right-justified, the Processor and the Name columns left-justified.
I doubt the usefulness of two-places decimal precision for NPS but if you want to keep it please consider using digit grouping (i.e. 123,456,789.xx). Abbreviating to kN/s would probably be best.

[MikeB]

I am trying to collect Stockfish nodes per second data from the computer chess community and I hope to get plenty of volunteers. The information will be useful for comparing computer chess results between computers.

I have created a package that can be downloaded at http://www.mediafire.com/file/3ivy28eqx ... _bench.zip . In that package you will find the various official compiles of Stockfish 8 (and the source code) and directions for running a script that will collect nps data for 30 positions searched to depth 24. I am hoping that people will submit data for various numbers of CPUs, especially 1 CPU and the max number of CPUs their computer has. The process took 8 1/2 minutes using 1 CPU of my i7 3840QM processor.

I will be posting people'e submissions at http://adamsccpages.blogspot.com/p/stoc ... hmark.html . I will greatly appreciate it if you are willing to devote some time to generate the data.

Thanks,
Adam

One final comment - total time spent reaching depth 24 is probably more useful than average nps ,especially when dealing with multi core results.

[Ferdy]

I just sent 1cpu and 2cpu sf8 pop bench data, 3cpu and 4cpu will follow.

[Joerg Oster]

Stockfish does not recognize the Clear Hash command.

Code: Select all

Engine->Adapter: Unknown command: Clear Hash

It was sent like this,

Code: Select all

Adapter->Engine: Clear Hash

Perhaps this should have been,

Code: Select all

Adapter->Engine: setoption name Clear Hash

But since ucinewgame was sent before every position startpos ... perhaps the Hash was cleared.

Yes, Hash is being cleared after ucinewgame is sent, as well as the various History/CounterMove tables.