Albert Silver wrote:I have been testing Rybka 3, both the 32-bit and 64-bit engines, on a single CPU machine, testing in controlled matches as well as various test suites and the like.
Since Rybka 232a is still the undisputed number one engine, the announcement of an upgraded and significantly stronger version, with a long list of improvements and speculative Elo increase, has led to a lot of questions on how much is hype and how much is reality.
I don't have the final answer to this, but do believe what I know and have seen is close to the truth of it.
The first thing I did, after reading the touted tactical and kingside attack improvements, was to run it through some positions. I chose Michael Gurevich's 40 position suite, with a dozen king attack positions which had at least 3 positions that had completely stumped Rybka 232a on my rig, a modest Athlon64 2.2GHz with 512MB hash.
The result was impressive: Rybka 3 32-bit solved every single one of them and only once took as long as 4 minutes. The solution times varied from 2 times faster than Rybka 232a, to more than 10 times. I also checked to see if the 64-bit version still offered its known speedups, and the results showed a consistent 37% speedup on my rig. Note that I have been told that on machines with faster memory and the like, the speedup may even be greater, but know that 37% is what can be expected at the very least. Here are a couple of examples:
MGV34-K
12: SOLOVJOV - Lugovoj, St. Petersburg 2005
[D]1rb1qrk1/ppn1p1b1/3p3p/2pP2p1/4P1nQ/P1N2N1P/1PPBB1P1/R4RK1 w - - 0 1
Analysis by Rybka 232a: Not found in 20 mins
Analysis by Rybka 3 32-bit :
1.Qg3 Nf6 2.Qe1 Nd7 3.Rd1 Qg6 4.Bd3 Ne5 5.Nxe5
= (0.20) Depth: 12 00:00:25 694kN
1.Qg3 Nf6 2.Qe1 Nd7 3.Bd3 Ne5 4.Nxe5 Rxf1+ 5.Qxf1 Bxe5 6.Qf3 Qf7
= (0.21) Depth: 13 00:00:40 1123kN
1.Qg3 Nf6 2.Bd3 Nh5 3.Qe1 Nf4 4.Bxf4 Rxf4 5.a4 Bd7 6.Ne2 Rf8 7.Qa5
= (0.17) Depth: 14 00:01:26 2424kN
1.Qg3 Nf6 2.Bd3 Nh5 3.Qe1 Nf4 4.Bxf4 Rxf4 5.e5 c4 6.Be4 Nb5 7.Qe3 Nxc3 8.bxc3 Qd8 9.Rab1 dxe5 10.Nd2
= (0.23) Depth: 15 00:02:48 4860kN
1.Bxg5
+/= (0.37) Depth: 15 00:04:00 6741kN
1.Bxg5
+/= (0.57) Depth: 15 00:04:34 7489kN
Analysis by Rybka 3 64-bit :
1.Qg3 Nf6 2.Qe1 Nd7 3.Rd1 Qg6 4.Bd3 Ne5 5.Nxe5
= (0.20) Depth: 12 00:00:16 694kN
1.Qg3 Nf6 2.Qe1 Nd7 3.Bd3 Ne5 4.Nxe5 Rxf1+ 5.Qxf1 Bxe5 6.Qf3 Qf7
= (0.21) Depth: 13 00:00:25 1123kN
1.Qg3 Nf6 2.Bd3 Nh5 3.Qe1 Nf4 4.Bxf4 Rxf4 5.a4 Bd7 6.Ne2 Rf8 7.Qa5
= (0.17) Depth: 14 00:00:54 2424kN
1.Qg3 Nf6 2.Bd3 Nh5 3.Qe1 Nf4 4.Bxf4 Rxf4 5.e5 c4 6.Be4 Nb5 7.Qe3 Nxc3 8.bxc3 Qd8 9.Rab1 dxe5 10.Nd2
= (0.23) Depth: 15 00:01:46 4860kN
1.Bxg5
+/= (0.37) Depth: 15 00:02:32 6741kN
1.Bxg5
+/= (0.57) Depth: 15 00:02:53 7489kN
MGV39-K
14: CONQUEST - Hodgson, BL 9596 1995
[D]4kbnr/ppq2ppp/1np1p3/6P1/3r3P/2N2Q2/PPPB1P2/2KR1B1R w k - 0 1
Analysis by Rybka 232a: Not found in 20 mins
Analysis by Rybka 3 32-bit :
1.h5 Ne7 2.h6 Ned5 3.hxg7 Bxg7 4.Nxd5 cxd5 5.c3 a6 6.Kb1 Ra4
+/= (0.40) Depth: 13 00:00:35 974kN
1.h5 Ne7 2.h6 Ned5 3.hxg7 Bxg7 4.Nxd5 cxd5 5.c3 a6 6.Kb1 Ra4 7.b3 Ra5 8.Bd3 Rc5 9.Rh3 Nd7
+/= (0.35) Depth: 14 00:01:02 1747kN
1.Nb5
+/= (0.60) Depth: 14 00:01:40 3003kN
1.Nb5
+/- (0.80) Depth: 14 00:02:10 3995kN
1.Nb5
+/- (1.20) Depth: 14 00:02:19 4212kN
1.Nb5 cxb5 2.Bxb5+
+- (1.98) Depth: 14 00:03:24 6509kN
1.Nb5 cxb5 2.Bxb5+ Kd8 3.Bf4 Rd6 4.Rhe1 Kc8 5.Bxd6 Bxd6 6.Rxd6 Qxd6 7.Qc3+ Nc4 8.Bxc4 Ne7 9.Bxe6+ Kb8 10.Bxf7 Qf4+ 11.Qd2 Qc7 12.Qe3 Rc8 13.Qe5 Nc6 14.Qxc7+ Kxc7 15.Bd5 Kd6 16.Rd1
+- (2.10) Depth: 15 00:04:41 9081kN
1.Nb5 cxb5 2.Bxb5+ Kd8 3.Bf4 Rd6 4.Rhe1 Kc8 5.Bxd6 Bxd6 6.Rxd6 Qxd6 7.Qc3+ Nc4 8.Bxc4 Ne7 9.Bxe6+ Kb8 10.Bxf7 Qf4+ 11.Qd2 Qc7 12.Qe3 Rc8 13.Qe5 Nc6 14.Qxc7+ Kxc7 15.Bd5 Kd6 16.Rd1
+- (2.13) Depth: 16 00:07:17 14349kN
1.Nb5 cxb5 2.Bxb5+ Kd8 3.Bf4 Rd6 4.Rhe1 Kc8 5.Bxd6 Bxd6 6.Rxd6 Qxd6 7.Qc3+ Nc4 8.Bxc4 Ne7 9.Bxe6+ Kb8 10.Bxf7 Qf4+ 11.Qd2 Qc7 12.Qe3 Rc8 13.Qe5 Nc6 14.Qxc7+ Kxc7 15.Bd5 Kd6 16.Rd1
+- (2.13) Depth: 17 00:12:48 24568kN
Analysis by Rybka 3 64-bit:
1.h5 Ne7 2.h6 Ned5 3.hxg7 Bxg7 4.Nxd5 cxd5 5.c3 a6 6.Kb1 Ra4
+/= (0.40) Depth: 13 00:00:22 974kN
1.h5 Ne7 2.h6 Ned5 3.hxg7 Bxg7 4.Nxd5 cxd5 5.c3 a6 6.Kb1 Ra4 7.b3 Ra5 8.Bd3 Rc5 9.Rh3 Nd7
+/= (0.35) Depth: 14 00:00:39 1747kN
1.Nb5
+/= (0.60) Depth: 14 00:01:02 3003kN
1.Nb5
+/- (0.80) Depth: 14 00:01:22 3995kN
1.Nb5
+/- (1.20) Depth: 14 00:01:27 4212kN
1.Nb5 cxb5 2.Bxb5+
+- (1.98) Depth: 14 00:02:08 6509kN
I also tested it using the Silver Suite, an openings suite with 50 positions, tested with both white and black. I tested against Fruit 234n (thanks to Ryan Benitez) which had scored decently against Rybka 232a 32-bit:
Code: Select all
1 Rybka 2.3.2a 32-bit +47/-13/=40 67.0/100
2 Fruit 2.3.4n +13/-47/=40 33.0/100
Rybka 232a had a +123 Elo performance in this match-up. I expected Rybka 3 to improve on this, but figured the results we have seen for quads and octas and other latin-number configurations wouldn't be as extreme on a single-CPU machine. My, was I wrong....
Code: Select all
1 Rybka 3 32-bit +71/-10/=19 80.5/100
2 Fruit 2.3.4n +10/-71/=19 19.5/100
Rybka 3 32-bit had a +246 elo performance, in other words a
+123 Elo improvement over Rybka 232a 32-bit.
It's hard to know what to say, since the genuine numbers themselves appear straight out of a marketing blurb. The 64-bit single-CPU will truly be about 200 elo stronger than the next non-Rybka engine, which is just insane...
Albert