Rybka 3: The dark truth behind the hype

[M ANSARI]

Sorry I mean 2x speedup ... somehow i assumed there are 100 seconds to the minute ... I guess I need some sleep . As for time to depth ... yes ofcourse ... but also how many Knps in an equivalent test position should be quite accurate ... remember here the position is identical ... only the hardware is changing ... so Knps and time to achieve that should give an accurate estimate. I really don't know how Vas calculates his nodes and I am sure a LOT of people will make a big fuss about this when Rybka 3 comes out since it is showing even less nodes than 2.3.2a. But really what matters most is how strong the engine is and how quickly it finds solutions to difficult chess problems. This I assure you R3 is very good at.

[Zach Wegner]

... but also how many Knps in an equivalent test position should be quite accurate ... remember here the position is identical ... only the hardware is changing ... so Knps and time to achieve that should give an accurate estimate.

No, it won't be, because the MP version will take a different number of nodes to get to the same depth. As I mentioned with Toga, it will double the NPS with double the number of cores, but it will have to search way more nodes, because it has a crappy parallel algorithm. The problem with Rybka is that you don't know how many nodes it's actually searching, as the MP versions are altered.

Of course you are right about what matters most is how well it plays, and that is where Rybka is the clear leader.

[Thomas Mayer]

Hi,

Sorry I mean 2x speedup ... somehow i assumed there are 100 seconds to the minute ... I guess I need some sleep .

hehe, during the french revolution this wouldn't have been too wrong. As far as I know they tried to introduced the 10 hours day with 100 minutes an hour...

Greets, Thomas

[bob]

I think much more accurate way of testing speed up of multiple cores is to check the output in KNs at certain depth between the Quad at 3.2 Ghz and the 8 core at 4.8 Ghz I had posted ... you will notice almost 3X speed up at equivalent depth ... which would make me think that it is actually scaling very well. If you try and find time to finding solution you will find that it will vary quite a lot due to the non determenistic characteristics of MP scaling.

OK, in 5th gear, my ford ranger pickup will run 5800 rpm. Is that faster or slower than your car???

RPM and NPS are just interesting numbers. What is important is how much faster will the program find reach the same depth when it uses more than one processor. Nothing else matters at all. And when a programmer intentionally obfuscates node counts anyway, that number is worthless.

[AdminX]

I think much more accurate way of testing speed up of multiple cores is to check the output in KNs at certain depth between the Quad at 3.2 Ghz and the 8 core at 4.8 Ghz I had posted ... you will notice almost 3X speed up at equivalent depth ... which would make me think that it is actually scaling very well. If you try and find time to finding solution you will find that it will vary quite a lot due to the non determenistic characteristics of MP scaling.

OK, in 5th gear, my ford ranger pickup will run 5800 rpm. Is that faster or slower than your car???

RPM and NPS are just interesting numbers. What is important is how much faster will the program find reach the same depth when it uses more than one processor. Nothing else matters at all. And when a programmer intentionally obfuscates node counts anyway, that number is worthless.

I agree, the numbers are worthless to us. Now to Vas, that is a different matter. Only he knows ...

[M ANSARI]

I hear this "obfuscate" thing quite often by some knowledgeable people .. but I still don't understant why Vas would want to do that. What advantage does that have? Is there information that can be gleaned from having the actual Knps (if it is getting obfuscated). Unless the obfuscation is to hide something that gives Rybka a competitve advantage I don't understand why he would do it? So my guess is either there is no obfuscation ... or there is obfuscation because it protects a certain secret which if exposed would remove Rybka's competitve advantage. Either way it really doesn't matter IMHO so long strength of the program is not affected.

[Tony Thomas]

I hear this "obfuscate" thing quite often by some knowledgeable people .. but I still don't understant why Vas would want to do that. What advantage does that have? Is there information that can be gleaned from having the actual Knps (if it is getting obfuscated). Unless the obfuscation is to hide something that gives Rybka a competitve advantage I don't understand why he would do it? So my guess is either there is no obfuscation ... or there is obfuscation because it protects a certain secret which if exposed would remove Rybka's competitve advantage. Either way it really doesn't matter IMHO so long strength of the program is not affected.

If my impression is correct, the obfuscation was introduced to give people the wrong idea that the initial Rybka is a knowledge based program with low node count.

[bob]

I hear this "obfuscate" thing quite often by some knowledgeable people .. but I still don't understant why Vas would want to do that. What advantage does that have? Is there information that can be gleaned from having the actual Knps (if it is getting obfuscated). Unless the obfuscation is to hide something that gives Rybka a competitve advantage I don't understand why he would do it? So my guess is either there is no obfuscation ... or there is obfuscation because it protects a certain secret which if exposed would remove Rybka's competitve advantage. Either way it really doesn't matter IMHO so long strength of the program is not affected.

Easy. If you find some new search idea... take null-move for example. And then you distribute an engine that uses that algorithm, it is possible for a user that knows the real node count, the real search depth, the real PV, the real score, to figure out what is going on in the search and "break the code" of the new idea.

So you obfuscate the node count and depth, you chop the PV off and hide part of it, etc, and use that to at least make it harder to figure out what is happening.

Ideally you would like to produce an engine that just gives a best move and nothing else, which would be a "black box" that would be nearly impossible to break.

[bob]

So 1 minute. I'm a bit puzzled by this since i thought an octal Skultraill QX9775 overclocked, as yours, would be around 14-15 times faster than Albert's 7.5 minutes solution that was on a 2.2GHz Athlon64.

An MP program generally has to look at more nodes to evaluate a position than an SP program. This is because cutoff values aren't known precisely. I have read that, in order to make comparing hardware strength easier, Rybka doesn't count the extra nodes the MP program searches.

There is also a lot of randomness in time it takes the MP program to find the critical move, so you can't project the speedup with any accuracy.

Yes when comparing time to find critical lines, but what about when comparing nodes per seconds?

According to Vasik he says about the Rybka 2.2n2:
"This updated version of Rybka will display node counts which represent the multi-processing efficiency. (They are adjusted to take into account the wasted work.) So, you'll be able to compare the strength of different hardware, ie. you can compare a 4-core machine with a 2-core machine. The one with the better nodes-per-second is better "


Also i remember him saying that for next versions of Rybka 2.2n2 that(the aforementioned) will be the standard way of measuring speedup efficiency on multi-processors.

So nodes per second should give the true efficiency for Rybka!

So why the nodes per second does show only a 7.5 factor on a Skulltrail with 8 cores 4.8GHz Versus an Athlon 1 CPU 2.2GHz?

If we compare them:
•Athlon is 1 single CPU while Skulltrail is 8 CPUs(cores). So we have a speedup factor of 8x from this.
•But also this Athlon is 2.2GHz while the octal is 4.8 GHz. So we have a speedup factor of about 2x from this.

The "nodes per second" speedup(that Vasik said it should give the efficiency of Rybka) factor is 7.5 as i've said earlier.

If we now remove the second factor of speedup(the different GHz values) we then have:
Theoretical speedup: 1 CPU versus 8 CPUs gives 8x
Actual speedup based on "nodes per second": 1 CPU versus 8 CPUs for Rybka 3 gives a 7.5/2 = 3.75x

So Rybka's speedup from 1 to 8 CPUs seems to be for that position 3.75.
I don't understand why it's so low. What is going on wrong?

Personally that statement about comparing NPS is a crock of crap. You can take the same program and run it miltiple times on the same position and get different node counts each time. Sometimes _significantly_ different node counts.

Trying to adjust the node counts to reflect the SMP overhead is just silly, because there is no way to _measure_ SMP overhead on a position unless you do the following:

search it with one processor to the key depth and remember the nodes searched. Then re-search with all the processors used, to the same depth, and remember the nodes search. Compare the two. Usually the SMP search will search extra nodes and these are overhead. But it is _impossible_ to recognize overhead nodes in an SMP search because there is no way to know you are searching something the serial search would not search, unless you do the above.

Lots of nosense, little reality...

[Uri Blass]

So 1 minute. I'm a bit puzzled by this since i thought an octal Skultraill QX9775 overclocked, as yours, would be around 14-15 times faster than Albert's 7.5 minutes solution that was on a 2.2GHz Athlon64.

An MP program generally has to look at more nodes to evaluate a position than an SP program. This is because cutoff values aren't known precisely. I have read that, in order to make comparing hardware strength easier, Rybka doesn't count the extra nodes the MP program searches.

There is also a lot of randomness in time it takes the MP program to find the critical move, so you can't project the speedup with any accuracy.

Yes when comparing time to find critical lines, but what about when comparing nodes per seconds?

According to Vasik he says about the Rybka 2.2n2:
"This updated version of Rybka will display node counts which represent the multi-processing efficiency. (They are adjusted to take into account the wasted work.) So, you'll be able to compare the strength of different hardware, ie. you can compare a 4-core machine with a 2-core machine. The one with the better nodes-per-second is better "


Also i remember him saying that for next versions of Rybka 2.2n2 that(the aforementioned) will be the standard way of measuring speedup efficiency on multi-processors.

So nodes per second should give the true efficiency for Rybka!

So why the nodes per second does show only a 7.5 factor on a Skulltrail with 8 cores 4.8GHz Versus an Athlon 1 CPU 2.2GHz?

If we compare them:
•Athlon is 1 single CPU while Skulltrail is 8 CPUs(cores). So we have a speedup factor of 8x from this.
•But also this Athlon is 2.2GHz while the octal is 4.8 GHz. So we have a speedup factor of about 2x from this.

The "nodes per second" speedup(that Vasik said it should give the efficiency of Rybka) factor is 7.5 as i've said earlier.

If we now remove the second factor of speedup(the different GHz values) we then have:
Theoretical speedup: 1 CPU versus 8 CPUs gives 8x
Actual speedup based on "nodes per second": 1 CPU versus 8 CPUs for Rybka 3 gives a 7.5/2 = 3.75x

So Rybka's speedup from 1 to 8 CPUs seems to be for that position 3.75.
I don't understand why it's so low. What is going on wrong?

Personally that statement about comparing NPS is a crock of crap. You can take the same program and run it miltiple times on the same position and get different node counts each time. Sometimes _significantly_ different node counts.

Trying to adjust the node counts to reflect the SMP overhead is just silly, because there is no way to _measure_ SMP overhead on a position unless you do the following:

search it with one processor to the key depth and remember the nodes searched. Then re-search with all the processors used, to the same depth, and remember the nodes search. Compare the two. Usually the SMP search will search extra nodes and these are overhead. But it is _impossible_ to recognize overhead nodes in an SMP search because there is no way to know you are searching something the serial search would not search, unless you do the above.

Lots of nosense, little reality...

The question here is not about nodes per solution but "nodes" per second(we know that it is not nodes but something else that the program counts)

The question is if "nodes" was designed in a way that "nodes" per second is proportional to effective speed of the machine for deep rybka3.

There are some questions that only people who have deep rybka3 can answer by comparing results on different machines.

1)Does Deep rybka report the same number of nodes per second if you run the same position twice(I do not care about difference that are not more than 5%)?

2)Is it correct that rybka plays better in machines with more nodes per second?

3)Is it correct that the number of nodes per second is proportional to the playing strength?

In other words suppose that rybka shows 10000 nodes per second in machine A and 30,000 nodes per second in machine B

Does it mean that rybka is practically doing something that is equivalent to being 3 times faster in machine B?