Impressive Preliminary Results of Rybka 3 by Larry Kaufman!

[mclane]

exactly. engine and engine+1 is to similar to find out ANYTHING if you let them fight each other.

you can only find out by games versus many different engines, strong AND weak ones.

[bob]

Vas has been working on MP scaling for quite a bit more than 6 weeks ... 60 weeks seems more accurate. I don't think that he has switched to threads from processes ... but if he says he has cracked MP scaling then I would tend to believe him. I could never figure out why Vas used processes instead of threads, but he must have a reason. Anyway in a few days we will find out.

It is much easier to use processes instead of threads.
The processes do not have to worry about any sharing of objects except for those things deliberately placed in shared memory.

The only downside is that a process is a much heavier burden on the OS than a thread.

I guess that you won't see more than 5% speedup from changing from processes to threads, but you will see some benefit.

I don't follow _any_ of this. Crafty started out using threads. I had some compatibility issues with posix threads on various platforms and decided to switch to processes one day before a tournament a couple ofyears ago. It took about 2 hours to make it work. Processes are a bit harder to use, because now you have to work at sharing things. Where with threads sharing is implicitly done for you.

Cray Blitz used processes and threads, depending on what time-frame you looked at. We started with processes as that was all we had. Then we switched to threads when Cray added the "tfork()" facility to fork what were known as "lightweight processes" (aka threads).

So the simpler approach I use in Crafty works with either with no significant difference in effort, performance or readability, the significantly more complicated DTS as used in Cray Blitz works equally well with either approach as well. I'm not sure why this processes vs threads discussion keeps coming up...

[bob]

Perhaps, but let's see him run Rybka on 512 processors and see how well it really scales.

Unfortunately we don't have _any_ published data to show how any program scales on that kind of machine. I know what I would modify to use 256 nodes, and it would not be that big of a change although it would take some testing to get it tuned.

Better to wait until version 3 of Rybka comes out, then anyone can run the test from 1 to 8 or 1 to 16 to see how it does. I know what I do on those platforms at least...

[bob]

exactly. engine and engine+1 is to similar to find out ANYTHING if you let them fight each other.

you can only find out by games versus many different engines, strong AND weak ones.

about all you can learn from X vs X+1 is that X+1 is better or worse, but not by how much... Another problem is that a good piece of knowledge can actually hurt your performance against one program (usually a small amount) and help it against several others. You need to test against a variety of opponents to make sure your new change doesn't make you too aggressive against programs that can handle/refute the aggression, etc...

[S.Taylor]

exactly. engine and engine+1 is to similar to find out ANYTHING if you let them fight each other.

you can only find out by games versus many different engines, strong AND weak ones.

about all you can learn from X vs X+1 is that X+1 is better or worse, but not by how much... Another problem is that a good piece of knowledge can actually hurt your performance against one program (usually a small amount) and help it against several others. You need to test against a variety of opponents to make sure your new change doesn't make you too aggressive against programs that can handle/refute the aggression, etc...

I have felt the a wonderful thing about computer chess is that it cannot prepare for you, but you can prepare for it. Unlike human chess.
This means that when a computer program keeps its standard, that standard might be more genuine.
I'd like to see a program which EVERYONE (and thing) prepares for, and see how much it still wins. I mean, it would be interesting to see a program which no player human nor computer, below elo of 2600 will ever beat. (Maybe I can't say "never", but I mean in a match. Or maybe we can say never if below 2700 in a match, And no win ever if below 2500 and no draw ever if below 2350?)


[Oh! one exception and one only, would be if the player plays exact moves of a GM game and the computer happens to play those exact moves of that high level game in which the player of the known game was playing the moves of the computer, and you happened to know those moves which beat it....... by heart].

[bob]

I agree Tord. I started off with processes because of the performance benefit: you don't have to carry around a BOARD * everywhere you go, which reduces register pressure and requires slightly less calculation for reading parts of the board. I'm not sure how much this last part makes a difference, but theoretically for a static global you know the address of each element at compile time, whereas with a pointer you have to add the offset for each element to the pointer you get passed.

But I have had a much easier time dealing with threaded programs. I'm considering converting ZCT to threads, but I think other things would be more productive.

Remember that (a) not everyone is doing an iterative-type search, most are using recursion. (b) your iterative data is also requiring an extra register reference for the subscript. Bottom line is there is very little difference between the two approaches. When I added the pointer to crafty I found no appreciable slow-down at all. In Cray Blitz, which used iterative search, I used both threads and processes depending on the year in question, and didn't notice any speed difference at all. And, in fact, that should be the desired result.

Only issue with using processes rather than threads today is that egtb.cpp is written for threading so that it shares the LRU buffers (egtb cache). In processes, each process will end up with its own egtb cache which is not nearly as effective as a shared cache with threads...

[bob]

I am not convinced that iterative search is required for good speedups. The DTS approach is certainly going to be _more_ efficient, but the margin is not huge between the two based on my dissertation results.

In the last test I did on this, Crafty got 3.3 / 4.0 on the same test set that CB got 3.7 on. While that is certainly significant, it is not overwhelming. Also at 16, I think CB got around 11.7 or so, while Crafty was around 10.8, although without the DTS paper in front of me, I might be a little off on the 11.7 number. And the 16 core numbers need to be backed up with more runs to be as reliable as the 8-core numbers I have provided in the past.

I have some 32 and 64 node numbers but not enough runs to want to publish any numbers and start yet another pointless discussion. But at least thru 16 processors, current Crafty does well. I have (and can easily run more) tons of 8-core results sitting around. Quite a few are on my ftp box, although those are AMD quad dual-core boxes as opposed to dual-quads based on Intel which seems to perform better with crafty.

I already have approaches ready and in place when I get access to a 32-64 core system for any significant amount of time. The issues are certainly non-trivial, but are clearly solvable given enough test time.

[bob]

i read what cozzie said, what chrilly donninger said. and what many others said.

If the analysis on the actual Rybka NPS is correct, then it is just a well-written bean counter with some search tricks not yet discovered by others. That is the most logical explanation as one would not intentionally try to obfuscate node counts and search depths if the evaluation was the main thing providing Rybka's skill level.

[bob]

Today we rely on a hack/crutch. Programs can't come anywhere near seeing deep enough to understand what is needed to exploit a closed position. So we just tune them to hate closed positions. And in doing so, we introduce yet another weakness that can be exploited by top players. For example, a simpler idea some have used is to try to make their programs avoid trading queens because queens increase the tactical complexity of positions. But if your program tries too hard, it can get into awkward positions where it has avoided the trade, but is hopelessly cramped or weakened as a result. And then it succumbs to some squeeze-play because it held onto its queen when it should not have.

So we just have "hacks" to avoid anti-computer play. And that's why, at least in Crafty, I normally ran with this off and only turned it on for specific players. Because they were not good enough to exploit the weakness, and were fanatical on trying to block the position. And Crafty refused to have any of it, even if it had to make other concessions. Because the opponent was not good enough to exploit them. But I didn't use this against top-level GM players, because they didn't go for the blocked positions (for whatever reasons) in the first place...

But no matter what, this kind of stuff makes the engine weaker overall, although it might avoid a particular kind of loss to a particular kind of player...

[bob]

my only comment would be that zero losses and zero draws at the ratings you gave would mean the sample is statistically invalid. In human chess, a 2000 player will beat a 2600 here and there. Maybe once every 100 games, but it will happen...