Perft helpers

[Don]

P.S.

When we get to the perft(14) calculation it doesn't matter if you have little CPU time to volunteer, it will numbers that count. If you have a quad you can run 4 instance, or just one instance.

One user is running one instance in Linux using nice by setting the binary in the config file to "perftbin = nice ./kperft"

Also possible (if we can get one) is to run 1 instance but with an MP binary - which I believe would be a big efficiency gain due to hash table sharing.

Don

[Ajedrecista]

Hello again:

Hello Don:

Perft(12) is moving along nicely with several people already contributing.

Before I start perft(14) I will provide a faster perft binary based on the one that Steven Edwards wrote. I don't know if or when his bus will come in but we can switch to his binary if he comes up with performance enhancements. It is designed to use any binary you want to plug in anyway so it's not an issue.

I have a web page with a project completion date based on the last hours worth of activity, the web page is updated every hour. Also, the windows binary seems to be working quite well.

http://komodochess.com/pub/perft12.html

Before perft(14) I will add a feature that allows us to track by user their contributions - several have already requested that feature.

The web page is rudimentary and will be improved for the perft(14) calculation.

Thank you very much for your effort and interest in this project.

I took a quick look to your web and it seems almost fine for me; maybe 'percentage completed' can be misunderstood in the sense of the total count instead of the supposed correct meaning of completed perft(n) drafts (n < 12) for calculate Perft(12).

Percentage complete is percentage of the 9+ million positions that have been searched so far. It's not to be mistaken for the "correct" percentage of the final number.

At the bottom I have "ratio to correct answer" and that is for debugging - it will go away for perft(14) (since we don't know the correct number.) I needed to see that it is actually converging due to transpositions.

I could be interested in taking part in this distributed perft project although my contribution would be very modest (only few hours of cycles of an Intel i5-760 (2.8 GHz), which is a quad, with Windows XP 32-bit). But I need more info, please:

a) Is it an .exe file that can be used whenever the user wants? I guess that the time of calculations of drafts will be small enough (let me say around an hour, for example) for making all the project more handling for the users.

b) Is a registration needed?

c) How can the programme be downloaded, if possible?

Thanks in advance and good luck with the project.

Regards from Spain.

Ajedrecista.

No registration necessary right now, but probably will for the perft(14) numbers because I want to track the users and display their contributions. It will not be sophisticated but something simple.

You have to email me to get further instructions as I consider this more of a controlled beta test and I need to be able to send out emails to all the volunteers should there be issues.

And yes, it's very simple to run. It's 2 exe files (for windows) or 2 binaries for Linux. One is the perft binary (it can be any "conforming" binary but I supply one) and the other is a GUI that displayed the actual board being searched along with a move list with sub-totals. It's cool and it's very easy to run - just unzip it and run the GUI and it's all set up as is.

Don

Thanks for your quick answer. I still have to think about it; furthermore, I do not have admin rights in that i5 quad (I must open a session with username and keyword), so I do not know if I will be allowed to run your programme, which will need access to Internet for sending the results (Internet should not be a problem); so far, I can run Komodo 3, SF, Critter... in console mode.

I stay tuned just in case. If I finally take part in this project (very unlikely), I am one of the candidates for being in the very last place of contributors!

Regards from Spain.

Ajedrecista.

[Peter Skinner]

I have 4 instances running on one of my quads. Will likely setup another quad later today and run another 4 instances.

[Don]

I have 4 instances running on one of my quads. Will likely setup another quad later today and run another 4 instances.

That is awesome! We are approaching 12% completion now.

When we move to perft(14) we will need volunteers in a big way - this is going to be a massive computation.

Maybe I can talk a few people with big setups to help some - it doesn't have to be 24/7, but if we could get 50 volunteers with quads or more and perhaps a few people with clusters to help some such as Bob Hyatt, etc...

Don

[Peter Skinner]

That is awesome! We are approaching 12% completion now.

When we move to perft(14) we will need volunteers in a big way - this is going to be a massive computation.

Maybe I can talk a few people with big setups to help some - it doesn't have to be 24/7, but if we could get 50 volunteers with quads or more and perhaps a few people with clusters to help some such as Bob Hyatt, etc...

Don

As of this morning we are at 18.421%. Pretty big jump overnight. I have 8 instances going here currently, but had 12 going overnight

Peter

[mathmoi]

I'm also running 6 instances since yesterday (really 7 instances on 6 cores). I could run 6 more if/when there is a command line version of the client.

[Don]

I'm also running 6 instances since yesterday (really 7 instances on 6 cores). I could run 6 more if/when there is a command line version of the client.

The estimated completion date has also moved up a bit!

Don

[Don]

Perft has passed the 1/3 mark! And we expect to shortly have a new much faster perft calcuation binary from Paul Byrne which will more than double the calculation speed even for perft(12)

Don

[rvida]

I wonder if you considered security aspects of this project. Imagine a malicious "helper" who deliberately sends back erratic results - even a simple off by one error (which is normally undetectable) can spoil the whole effort.

A few advices you might consider:
- assign more (but smaller) work units and include some random redundancy (2 results from different clients for the same work unit can detect deliberate miscalculations).
- with the client's response include a hash digest from the used executable (helps detect tinkering with the binary)
- Use asymmetric cryptography with public/private keys to be sure about the identity of clients
- security through obscurity - while it is a bad practice, it has some merits. Don't send the results in open-text format, instead invent a very obscure (shuffled/encoded/xored/whatewer) binary format. Don't publish source code of the client software.

[Don]

I wonder if you considered security aspects of this project. Imagine a malicious "helper" who deliberately sends back erratic results - even a simple off by one error (which is normally undetectable) can spoil the whole effort.

A few advices you might consider:
- assign more (but smaller) work units and include some random redundancy (2 results from different clients for the same work unit can detect deliberate miscalculations).
- with the client's response include a hash digest from the used executable (helps detect tinkering with the binary)
- Use asymmetric cryptography with public/private keys to be sure about the identity of clients
- security through obscurity - while it is a bad practice, it has some merits. Don't send the results in open-text format, instead invent a very obscure (shuffled/encoded/xored/whatewer) binary format. Don't publish source code of the client software.

I have already considered many of those points. The actually perft(14) calculation is going to be tracked by user. The user will register with me and be assigned a code (probably his user name with salt added then md5sum or other cryptographic hash. It will be included with his results. It's not perfectly secure but I'm not going to build a heavy authentication system into this as it is a skunks work thing for me. It's still fairly secure because I will know who the users are, only the server will know the code for each user and even if the network is sniffed I will at least be able to isolate the user having issues. I could also do this with a challenge system - when work is requested I send a random challenge, it is hashed by the client along with the users code and sent back with the results. That would be hard to fake without knowing the users code and would be easy to implement.

I am tracking the client performing the calculation (the binary being used) so if some client is discovered to have problems I can rerun those results, but I plan to run a verification pass anyway.

Don