Perft(15) - it will cost you only US$500,000

Discussion of chess software programming and technical issues.

Moderator: Ras

Dann Corbit
Posts: 12856
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Perft(15) - it will cost you only US$500,000.

Post by Dann Corbit »

Henk wrote:Buying computers is just a waste of money. For after three or five years it's worthless. So I never understood people that bought expensive computers. Biggest problem is these software updates that force us to buy better hardware for otherwise it won't run. They are all thieves.
This is true of literally every technology. Everything is quickly outdated. So why bother to even get it? You have the enjoyment of good technology while it is the best. And you can get a pretty good computer for less than $1000. If you want a real hot-rod computer, you need a lot of money, but it will only be incrementally better than a much cheaper one.

I saw on TV that if there is no oil anymore then no satellites won't be launched. That means end of internet and perhaps end of these annoying software updates.
There are always replacement technologies. Fusion? Solar? It will be something.
Better invest in Lithium or algae.
I guess that the future battery technology will be based on aluminum. Or as they say in Europe aluminium, once famously mentioned by Neils Bohr.

Of course, it is always possible that we will poison, blast or otherwise stupid ourselves into a corner. But I believe that there is even a solution for that.
Dann Corbit
Posts: 12856
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Perft(15) - it will cost you only US$500,000

Post by Dann Corbit »

elpapa wrote:
Dann Corbit wrote:Next year: $250,000.00
In 16 years: $3.81
I find that hard to believe. You forgot to adjust for inflation.
Right. It will only be a few cents. Unless inflation goes the other way (which is pretty unusual, especially with interest rates as low as they are right now, it would be hard to imagine plummeting interest rates).
Dann Corbit
Posts: 12856
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Optimizations

Post by Dann Corbit »

sje wrote:Symbolic already uses bulk counting at penultimate nodes. The observed speedup is a factor of at least about fourteen and with higher values for more complex positions as expected.

Oscar also has bulk counting although it's not a bitboard program.

----

At present I'm looking at writing a perft(6) server. This would run on a dedicated machine on the LAN and would store and serve perft(6) results to the worker machines. The idea is for the server to use a big fat drive holding a vector of binary trees of perft(6) results. Each of the N=2^Q trees would hold values for positions with signatures where the bottom Q bits mapped to the tree's index. With 2^16 trees, the first 16 key comparisons are done with a simple mask; disk access for location/insertion is thus greatly reduced.

Each tree entry is 32 bytes long: signature/key 16 bytes, count 8 bytes, left and right record indices 4 bytes each. For perft(14), I estimate the distinct perft(6) count to be about one billion which would need about 32 GiB storage. Which is not that much, but will increase a lot with projects doing perft(15), perft(16), etc; also, a big increase if the server handles perft(5) results as well.

Using trees instead of transposition tables means that no entries are ever lost. Also, the nice feature of having persistent data retained across server machine power cycles.

One attraction here is that the effect of network latency is nearly zero because each worker machine can have its copy of Symbolic run extra threads so that waiting threads do not make any cores go idle. Symbolic's semaphore/queue mechanism has shown to have less than 1% time overhead even when running active threads at a ratio of eight threads per core.
GPU projects like protein folding, GIMPS, gravitation wave detection, etc. are very attractive for the following reason:
They don't tax the computer for other tasks. For instance, I can test factor a large Mersenne prime candidate on the GPU and my CPU still runs full speed doing chess analysis.

I think that there is also a framework built for such cooperation type projects.
ibid
Posts: 89
Joined: Mon Jun 13, 2011 12:09 pm

Re: Perft(15) - it will cost you only US$500,000

Post by ibid »

hgm wrote:That makes a lot of assumptions. Smarter perft programs, e.g. incrementally calculating the number of moves in the last ply rather than generating them all, could easily speed up the calculation by a factor 30 compared to existing perft programs. The forelast move does not have that much effect on what the opponent can move. Many moves have no effect at all (if they do not happen to block or unblock enemy sliders, attack squares next to the king, or pin pieces), and their contribution would be equal to perft(1) after null move, which you could calculate once and then use for all such moves.
This is the theory behind the development version of gperft. In practice, I am not getting anywhere close to a factor of 30. Or even a factor of 2.

The problem is that last Makemove/Count/Unmakemove can be done quite quickly anyhow. I've had a specialized MCU() function just for this since the early versions; if you can quickly exclude certain moves then the M and U sections reduce to some quite simple code if the C section is modified to account for the fact that the M is cheating and not updating the entire board structure.

When you try to get smart and use a nullmove perft(1), or portions of it, the question becomes... how quickly can you figure out what needs recomputing? Yes, it is generally quicker than a MCU call, but it is not trivial. Is the move a checking move? Are you removing or adding pins? Are you blocking sliders, or removing blocks? Are you attacking or removing attacks on squares next to the king, changing king moves? etc.

There are a lot of different possibilities to try out though. Do you simply try to identify moves which have no effect at all so you can use the null move value directly? Or do you try to determine which pieces have a different move count and just update those pieces? And how accurate are those determinations? (i.e. if you find every case where a queen's moves don't change, it will probably take longer to decide than simply computing the queen moves. If you do a really fast check, you might only catch a small fraction of the times you can skip that queen.)

It depends a lot on the position. Some changes can speed one position up by 20% or more, and slow others by the same amount. My current version has a worst case speedup of about 5% for some pathalogical positions (things like the positions with 218 legal moves, for example), about 40% for the initial position, and about 25% for a typical opening position. This compared to the last release of gperft. I am still slowly testing out possibilities though, hopefully I can extract a little more speed.
elpapa
Posts: 211
Joined: Sun Jan 18, 2009 11:27 pm
Location: Sweden
Full name: Patrik Karlsson

Re: Perft(15) - it will cost you only US$500,000

Post by elpapa »

Dann Corbit wrote:
elpapa wrote:
Dann Corbit wrote:Next year: $250,000.00
In 16 years: $3.81
I find that hard to believe. You forgot to adjust for inflation.
Right. It will only be a few cents. Unless inflation goes the other way (which is pretty unusual, especially with interest rates as low as they are right now, it would be hard to imagine plummeting interest rates).
Yes, inflation has been close to 0 percent the last 5 or 10 years, at least here in Sweden. It's obviously totally irrelevant in this context, that was the joke I was trying to make.