I downloaded and tested the suite also, and got very different results. Rolf is correct about the dubiousness of some positions. Notably 3 have no adequate defense, and thus avoiding a move does not change the lost evaluation. The positions in question are no. 3, 20, and 46. These positions cannot be saved with best play. No. 48 claims Qb6 to be the wrong move, proposing none in its stead. Even stranger is that the Excel file claims Robb gets it correct, meaning it avoids this move, but this is incorrect on both accounts. First, Qb6 is in fact the best move in the position, and second, Robbo does not avoid playing it.
Aside from this, there is also the issue of the results reported. Naturally I take into account the stronger computer I used to run the suite, but this only explains time discrepancies, not outright different results. I double-checked and triple checked my results, noting more than one discrepancy works to Robber's favor. These were all hand done, clearing the hash between engine loads, on a Core 2 Duo 3.6GHz with 1.5 Gb Ram for the hash.
In my testing, not removing the wrong positions, Robbo got 9 wrong, and Rybka 6. I used the 64-bit versions of the engines, but where results were radically different, I checked with the 32-bit version. The results were consistent.
It is also worth noting that the 64-bit version of Robber displays the exact same speedups Ryba does, when compared with the 32-bit version. Ex:
Position 18: SwissTest4_18 - Verteidigung, DeepShredder11-Bright02c, CompGame 2008
[d]r1bq1rk1/pp2n1p1/2n1p2B/3pPp1Q/2pP2P1/P1PB1N2/2P2P1P/R3K2R b KQ - 0 1
Moves to avoid: 12...Qe8, 12...Rf7, and 12...cxd3. Time allotted to find solution is 60 seconds.
RobboLito is reported to not find this within 1 min, and Rybka 3 MP in 18 seconds.
Analysis by RobberLito 0.085e4 w32:
12...Rf7 13.gxf5 cxd3 14.f6 Qa5 15.Bxg7 Qxc3+ 16.Kf1 Qxa1+ 17.Kg2 Rxg7+ 18.fxg7 Qxh1+ 19.Kxh1 Kxg7 20.cxd3 Nd8 21.Qh4 Nec6 22.Qf6+ Kg8 23.Kg2 Nf7 24.h4 b5 25.Qg6+ Kf8 26.h5
+/= (0.56) Depth: 13/43 00:00:09 10435kN
12...g6 13.Qg5 cxd3 14.Bxf8 Qxf8 15.gxf5 Qxf5 16.Qxf5 Nxf5 17.cxd3 Kg7 18.Rg1 Bd7 19.Rb1 b6 20.Kd2 Rh8 21.Kc2 Nce7
+/= (0.31) Depth: 13/43 00:00:11 12606kN
12...g6 13.Qg5 cxd3 14.Bxf8 Qxf8 15.gxf5 Qxf5 16.Qxf5 Nxf5 17.cxd3 Kg7 18.Rg1 Bd7 19.Rb1 b6 20.h3 Nce7 21.Ng5 Rc8 22.Kd2
+/= (0.34) Depth: 14/43 00:00:11 12764kN
12...g6 13.Qg5 Qa5 14.Kd2 cxd3 15.Bxf8 Kxf8 16.cxd3 fxg4 17.Qxg4 Nf5 18.Rhg1 Nce7 19.Qh3 Ke8 20.a4 Kd7 21.Qh7 Kc6 22.Kc2 b6
+/= (0.53) Depth: 15/48 00:00:33 37253kN
12...g6 13.Qg5 cxd3 14.Bxf8 d2+ 15.Kxd2 Qxf8 16.gxf5 Qxf5 17.Qxf5 Nxf5 18.h4 Nfe7 19.h5 Bd7 20.Rab1 Rf8 21.Ke2 Na5 22.Ng5 gxh5 23.Rxh5 Ng6 24.Rh7 Nf4+ 25.Kf1
+/= (0.55) Depth: 16/49 00:00:42 47548kN
12...g6 13.Qg5 cxd3 14.Bxf8 Qxf8 15.cxd3 fxg4 16.Qxg4 Qf5 17.Qxf5 Nxf5 18.h4 Bd7 19.Rb1 Na5 20.h5 gxh5 21.Rxh5 Be8 22.Rh1 Rc8 23.Kd2 Kf8 24.Rh7
+/- (0.76) Depth: 17/64 00:02:25 164mN
12...g6 13.Qg5 cxd3 14.Bxf8 Qxf8 15.cxd3 fxg4 16.Qxg4 Qf5 17.Qxf5 Nxf5 18.h4 Ng7 19.Kd2 Ne7 20.Rag1 Bd7 21.h5 Rf8 22.Ke2 Rc8 23.h6 Ngf5 24.Kd2 Kh8 25.Ng5
+/= (0.70) Depth: 18/64 00:02:53 196mN
Analysis by RobberLito 0.085e4 x64:
12...Rf7 13.gxf5 cxd3 14.f6 Qa5 15.Bxg7 Qxc3+ 16.Kf1 Qxa1+ 17.Kg2 Rxg7+ 18.fxg7 Qxh1+ 19.Kxh1 Kxg7 20.cxd3 Nd8 21.Qh4 Nec6 22.Qf6+ Kg8 23.Kg2 Nf7 24.h4 b5 25.Qg6+ Kf8 26.h5
+/= (0.56) Depth: 13/43 00:00:06 10435kN
12...g6 13.Qg5 cxd3 14.Bxf8 Qxf8 15.gxf5 Qxf5 16.Qxf5 Nxf5 17.cxd3 Kg7 18.Rg1 Bd7 19.Rb1 b6 20.Kd2 Rh8 21.Kc2 Nce7
+/= (0.31) Depth: 13/43 00:00:08 12606kN
12...g6 13.Qg5 cxd3 14.Bxf8 Qxf8 15.gxf5 Qxf5 16.Qxf5 Nxf5 17.cxd3 Kg7 18.Rg1 Bd7 19.Rb1 b6 20.h3 Nce7 21.Ng5 Rc8 22.Kd2
+/= (0.34) Depth: 14/43 00:00:08 12764kN
12...g6 13.Qg5 Qa5 14.Kd2 cxd3 15.Bxf8 Kxf8 16.cxd3 fxg4 17.Qxg4 Nf5 18.Rhg1 Nce7 19.Qh3 Ke8 20.a4 Kd7 21.Qh7 Kc6 22.Kc2 b6
+/= (0.53) Depth: 15/48 00:00:24 37253kN
12...g6 13.Qg5 cxd3 14.Bxf8 d2+ 15.Kxd2 Qxf8 16.gxf5 Qxf5 17.Qxf5 Nxf5 18.h4 Nfe7 19.h5 Bd7 20.Rab1 Rf8 21.Ke2 Na5 22.Ng5 gxh5 23.Rxh5 Ng6 24.Rh7 Nf4+ 25.Kf1
+/= (0.55) Depth: 16/49 00:00:30 47548kN
12...g6 13.Qg5 cxd3 14.Bxf8 Qxf8 15.cxd3 fxg4 16.Qxg4 Qf5 17.Qxf5 Nxf5 18.h4 Bd7 19.Rb1 Na5 20.h5 gxh5 21.Rxh5 Be8 22.Rh1 Rc8 23.Kd2 Kf8 24.Rh7
+/- (0.76) Depth: 17/64 00:01:45 164mN
12...g6 13.Qg5 cxd3 14.Bxf8 Qxf8 15.cxd3 fxg4 16.Qxg4 Qf5 17.Qxf5 Nxf5 18.h4 Ng7 19.Kd2 Ne7 20.Rag1 Bd7 21.h5 Rf8 22.Ke2 Rc8 23.h6 Ngf5 24.Kd2 Kh8 25.Ng5
+/= (0.70) Depth: 18/64 00:02:06 196mN
Analysis by Rybka 3:
12...Rf7 13.Ng5 g6 14.Qh3 fxg4 15.Qe3 cxd3 16.Nxf7 Kxf7 17.cxd3 Nf5 18.Qd2 Qh8 19.Bg5 b6 20.Rb1
= (-0.25) Depth: 9 00:00:02 276kN
12...g6 13.Qg5 Qa5 14.Kd2 Rf7
+/= (0.32) Depth: 10 00:00:08 846kN
12...g6 13.Qg5 Qa5 14.Kd2 Rf7 15.Be2 fxg4 16.Qxg4 Rh7 17.Be3
+/= (0.42) Depth: 11 00:00:09 905kN
12...g6 13.Qg5 cxd3 14.Nh4 Kh7
+/= (0.43) Depth: 12 00:00:14 1431kN
12...g6 13.Qg5 cxd3 14.Nh4 Kh7 15.Bxf8 Qxf8 16.gxf5 gxf5 17.cxd3 Qh6 18.Nf3 Qxg5 19.Nxg5+ Kg7 20.Rg1 Ng6 21.Kd2 Bd7 22.Rg3
+/= (0.47) Depth: 13 00:00:24 2454kN
12...g6 13.Qg5 cxd3 14.Nh4 Kh7 15.Bxf8 Qxf8 16.gxf5 d2+ 17.Kxd2 gxf5 18.Rhg1 Qh6 19.Nf3 Ng6 20.Rg3 Bd7 21.Qxh6+
+/= (0.59) Depth: 14 00:00:45 4437kN
12...g6 13.Qg5 cxd3 14.Nh4 Kh7 15.Bxf8 Qxf8 16.gxf5 exf5 17.cxd3 Qh6 18.Nf3 Qxg5 19.Nxg5+ Kg7 20.h4 f4 21.Kd2 Bf5 22.Rag1 Rh8 23.h5
+/= (0.62) Depth: 15 00:01:55 11791kN
Questionable positions:
No. 3: SwissTest4_03 - Verteidigung, Aristarch45-TheBaron15, CompGame 2005
[d]r1b2rk1/2p2ppp/p2b4/1p6/6Pq/1BP5/PP1P1PP1/RNBQR1K1 w - - 0 1
Moves to avoid: Qf3, Re3, g3.
Unfortunately, white is dead and has no adequate defense. The only other tries, f3 or Re4, are also insufficient.
No. 20: SwissTest4_20 - L vs L, Eigenmann 2006
[d]5k2/5ppp/p7/3p1b2/5P2/5P2/PPBK3P/8 b - - 0 1
Moves to avoid: Bxc2.
Obviously the pawn endgame is dead lost after the exchange of bishops with the queenside majority. The only problem is the endgame is lost even with the bishops. Ex:
1... Be6 2. b4 Ke7 3. Kc3 Kd6 (3... d4+ 4. Kxd4 Bxa2 5.Kc5!) 4. Bd3 Bc8 5. a4 g6 6. Kd4 f6 (6... h6 7. a5!) 7. h4!
No. 46 Naum22-Hiarcs112, CompGame 2007
[d]2r3k1/5p1p/4pP2/1p1p4/1p1R3P/8/PPP3P1/1K6 b - - 0 27
Moves to avoid: Rc4.
It is true that 27... Rc4 fails, but Black has no way to save this position against perfect play. The only other try that comes to mind, is 27...h5, but I don't see any hope for Black after 28.Rxb4 Rc5 29.g4
No. 48 - Pepito142-LGoliath35, CompGame 2002
[d]r1bq1rk1/pp2bpp1/3pp3/6P1/4PPn1/2N5/PPPQ2P1/2KR1B1R b - - 0 13
Moves to avoid: Qb6
The problem is that this is actually the best move.
If one removes the 4 positions from the equation, Robbo got a total of 6 wrong out of 60, and Rybka got 3 wrong.
Albert