a Telltale position

[lkaufman]

1. d4 Nf6 2. c4 g6 3. Nc3 d5 4. Nf3
Bg7 5. Qb3 dxc4 6. Qxc4 O-O 7. e4 b6 8. e5 Be6 9. exf6 Bxc4 10. fxg7 Kxg7 11. Bxc4.

Consider the above opening sequence, leading to the material imbalance of two bishops and a knight for queen and pawn. The move 7...b6?! is rarely played by Grandmasters because the above position is considered quite favorable for White. Material is deemed even but White has all the positional pluses. Most modern engines agree with this assessment. I ran most of them on this position for 30" each on a quad; here are the assessments:

Deep Rybka 4: +.53
Deep Shredder 12: +.88
Fritz 12: +.47
Hiarcs 13.1: +.48
Komodo 1.3: +.30
Naum 4: +1.05
Stockfish 2.0: +.20


Fire 1.31: +.09
Rybka 2.3.2 a MP: +0.02
Robbolito .085g3: 0.00
Ivanhoe 47 and 49: 0.00
Houdini 1.5: -0.13
Critter 0.90: -0.17

So Rybka 2.3.2a MP got this quite wrong with a nearly zero score, and Robbolito, which is said to have come from decompiled Rybka 2.3.2a code, also makes the same mistake with a zero score. Of course the scores won't be identical as the searches are different. The engines acknowledged to come from Robbo have of course also a zero or near-zero score. Houdini and Critter actually go negative; it is hard to imagine that a program not starting with the Robbo values would make such a big error in evaluating this position. I don't know much about Critter so I don't mean to start a debate about its status, but this is certainly strange.

As for why Rybka 2.3.2a gets it wrong, all attempts to fix the undervaluation of minor pieces vs. major pieces tested poorly in Rybka, yet seem to test okay in unrelated engines. So any program that makes this same mistake is likely to either have copied Rybka values, or to be so similar to Rybka that testing produced the same anomalous result.

[Christopher Conkie]

Want a laugh Larry? Sure you do.... Take a good long look.........


[d]rn1q1r2/p1p1ppkp/1p4p1/8/2BP4/2N2N2/PP3PPP/R1B1K2R b KQ - 0 11

Houdini 1.0 x32:
1/3 00:00 30 0 -8.70 Qxd4 Nxd4 (this bit is very funny indeed)
1/6 00:00 37 0 -0.20 Rh8
1/8 00:00 46 0 -0.18 Rg8
1/8 00:00 52 0 -0.10 Qd7
1/8 00:00 55 0 -0.04 Qd6
1/8 00:00 60 0 +0.09 Nc6
2/8 00:00 203 203,000 +0.13 Nc6 d5
3/10 00:00 617 617,000 +0.13 Nc6 d5 Nb4
4/12 00:00 1,185 592,000 +0.11 Nc6 d5 Nb4 0-0 Qd6 Ne4
5/16 00:00 2,918 729,000 +0.10 Nc6 Bf4 f6 d5 Nb4
6/20 00:00 8,351 695,000 +0.05 Nc6 Be3 Qd6 0-0 Qb4 Bb3 Rad8
7/20 00:00 10,086 720,000 +0.05 Nc6 Be3 Qd6 0-0 Qb4 Bb3 Rad8
8/20 00:00 17,531 701,000 +0.04 Nc6 Bf4 f6 d5 g5 Bg3 Na5 Bb5
9/20 00:00 33,466 712,000 +0.08 Nc6 Bf4 e6 0-0 Qf6 Be3 Qf5 d5 Ne5
10/27 00:00 177,713 740,000 +0.05 Nc6 d5 Nb4 0-0 Qd6 Rd1 Nc2 Rb1 f6 Ne4 Qb4 b3 Rad8
11/27 00:00 243,119 750,000 +0.06 Nc6 Be3 Qd6 a3 e6 0-0 h6 Ne4 Qd7 Ne5 Nxe5 dxe5
12/28 00:01 876,842 777,000 +0.03 Nc6 Bf4 e6 0-0 h6 Rfd1 Qe7 Bb5 Nb4 d5 a6 d6
13/32+ 00:01 1,119,727 790,000 +0.08 Nc6 Bf4 e6 0-0 Qf6 Be3 Qe7 d5 Qb4 Bb5 Nd8 Bd4+ f6 Bd7 e5
13/32 00:01 1,244,747 790,000 +0.10 Nc6 Bf4 Nb4 0-0 c6 Be5+ Kg8 a3 Nd5 Rac1 f6 Bf4 g5 Nxd5 cxd5
14/32 00:02 2,201,242 785,000 +0.03 Nc6 Be3 Nb4 0-0 Nc2 Rad1 f6 Bc1 Qd6 Bb3 Nb4 a3 Nc6 Ne4
15/32+ 00:03 2,790,258 775,000 +0.08 Nc6 Be3 Nb4 0-0 Nc2 Rad1 f6 Bc1 Qd6 Bb3 Nb4 a3 Nc6 Ne4 Qd7
15/32 00:03 2,987,958 772,000 +0.05 Nc6 Be3 Nb4 0-0 Nc2 Rad1 Nxe3 fxe3 Qd6 Ng5 e6 Nce4 Qb4 Bxe6 fxe6 Nxe6+ Kg8 Nxf8 Rxf8 Rxf8+ Qxf8 Rc1
16/33 00:06 5,115,258 777,000 +0.03 Nc6 d5 Na5 Be2 Qd6 b3 c5 Bb2 Kg8 0-0 Rfd8 Rad1 Qf6 Rfe1 Qf4 Bd3
17/36 00:09 7,439,534 785,000 0.00 Nc6 d5 Na5 Be2 Qd6 0-0 Rad8 Be3 c5 Rad1 h6 Rfe1 g5 h4 g4 Ne4 Qg6

RobboLito 0085g3 x32:
1/3 00:00 30 0 -8.70 Qxd4 Nxd4
1/7 00:00 43 0 -0.32 Rh8
1/7 00:00 55 0 -0.29 Rg8
1/7 00:00 65 0 -0.16 Qd7
1/8 00:00 76 76,000 -0.06 Qd6
1/8 00:00 93 93,000 +0.15 Nc6
2/8 00:00 228 228,000 +0.22 Nc6 d5
3/12 00:00 790 395,000 +0.22 Nc6 d5 Nb4
4/15 00:00 1,433 477,000 +0.18 Nc6 d5 Nb4 0-0 Qd6 Ne4
5/16 00:00 3,183 454,000 +0.17 Nc6 Bf4 f6 d5 Nb4
6/20 00:00 8,550 502,000 +0.10 Nc6 Be3 Qd6 0-0 Qb4 Bb3 Rad8
7/20 00:00 10,282 514,000 +0.10 Nc6 Be3 Qd6 0-0 Qb4 Bb3 Rad8
8/20 00:00 17,008 531,000 +0.08 Nc6 Bf4 f6 d5 g5 Bg3 Na5 Bb5
9/20 00:00 41,137 563,000 +0.14 Nc6 Bf4 e6 0-0 Qf6 Be3 Qf5 d5 Ne5
10/22 00:00 96,100 568,000 +0.10 Nc6 d5 Nb4 0-0 Qd7 a3 Nc2 Rb1 Qf5 Rd1 Rad8 Bd3
11/22 00:00 125,499 586,000 +0.10 Nc6 d5 Nb4 0-0 Qd7 a3 Nc2 Rb1 Qf5 Rd1 Rad8 Bd3
12/24 00:00 255,956 605,000 +0.09 Nc6 d5 Na5 Be2 c5 dxc6 Nxc6 0-0 Nd4 Nxd4 Qxd4 Rd1 Qe5 Rd7 Rfd8
13/27 00:00 474,089 608,000 +0.06 Nc6 d5 Na5 Ba6 Qd6 0-0 c6 Rd1 Rad8 dxc6 Qxc6 Re1 Qd6 Nb5 Qc5 Nxa7
14/30 00:03 2,290,174 617,000 +0.05 Nc6 d5 Na5 Ba6 Qd6 0-0 Rad8 Be3 c5 Rad1 f6 Nb5 Qd7 d6 Rf7
15/30 00:04 2,587,221 620,000 +0.05 Nc6 d5 Na5 Ba6 Qd6 0-0 Rad8 Be3 c5 Rad1 f6 Nb5 Qd7 d6 Rf7
16/34 00:06 3,827,124 622,000 +0.02 Nc6 d5 Na5 Ba6 Qd6 0-0 Rad8 Rd1 c6 Be3 cxd5 Rxd5 Qb4 Rb5 Qg4 Ne5 Qh4
17/37 00:11 7,282,196 623,000 0.00 Nc6 d5 Na5 Be2 c5 0-0 Qc7 Be3 Rad8 Rad1 Qd6 Ne4 Qc7 Nc3


Hehe.....



Chris

[Cubeman]

How do the games from that position end, it would be interesting for some test games between the so called Ippo clones and the traditional other strong engines.A wrong evaluation would show up in game results.Sometimes I think Human evaluations are not necessary the absolute truth.I also imagine that there could be many engines out there even before Rybka beta that would evaluate similar scores as Houdini and Critter.

[Christopher Conkie]

How do the games from that position end, it would be interesting for some test games between the so called Ippo clones and the traditional other strong engines.A wrong evaluation would show up in game results.Sometimes I think Human evaluations are not necessary the absolute truth.I also imagine that there could be many engines out there even before Rybka beta that would evaluate similar scores as Houdini and Critter.

[d]rn1q1r2/p1p1ppkp/1p4p1/8/2BP4/2N2N2/PP3PPP/R1B1K2R b KQ - 0 11

IvanHoe999963:
1 00:00 30 0 -8.70 Qxd4 Nxd4 (Ahem.....)
1 00:00 43 0 -0.32 Rh8
1 00:00 55 0 -0.29 Rg8
1 00:00 65 0 -0.16 Qd7
1 00:00 76 0 -0.06 Qd6
1 00:00 93 0 +0.15 Nc6
1 00:00 109 0 +0.15 Nc6
2 00:00 228 0 +0.22 Nc6 d5
2 00:00 288 19,000 +0.22 Nc6 d5
3 00:00 790 52,000 +0.22 Nc6 d5 Nb4
3 00:00 845 56,000 +0.22 Nc6 d5 Nb4

Let's start at the beginning first......ok?



Chris

[Gerd Isenberg]

[d]rn1q1r2/p1p1ppkp/1p4p1/8/2BP4/2N2N2/PP3PPP/R1B1K2R b KQ - 0 11

Houdini 1.0 x32:
1/3 00:00 30 0 -8.70 Qxd4 Nxd4 (this bit is very funny indeed)




Chris

Hmm, what you quote funny (Qxd4 Nxd4) looks very reasonable to me, and I would expect every engine reporting each fail high at the root that way, since Qxd4 is the only capture and tried first in the first iteration of an IID framework. What is the point?

Gerd

[Christopher Conkie]

We could even start at the very beginning.........

[d]rn1q1r2/p1p1ppkp/1p4p1/8/2BP4/2N2N2/PP3PPP/R1B1K2R b KQ - 0 11

Ippolit 0.080a x32:
1/3 00:00 30 0 -8.44 Qxd4 Nxd4 (Are we getting the picture yet?)
1/6 00:00 41 0 +0.03 Rh8



Chris

[Gian-Carlo Pascutto]

You should try Fruit 2.1 on this position

[Christopher Conkie]

You should try Fruit 2.1 on this position

As you wish Gian-Carlo.

[d]rn1q1r2/p1p1ppkp/1p4p1/8/2BP4/2N2N2/PP3PPP/R1B1K2R b KQ - 0 11

Fruit 2.1 x32:
1/2 00:00 4 0 -0.51 b5 Bxb5
1/3 00:00 9 0 +0.60 a5
1/3 00:00 18 0 +0.72 e6
1/4 00:00 40 0 +1.06 Nc6
2/8 00:00 171 0 +0.85 Nc6 Be3
3/11 00:00 666 0 +0.91 Nc6 Bf4 Qd7
4/11 00:00 2,085 0 +0.64 Nc6 Be3 Na5 Bd3
5/13 00:00 5,983 0 +0.79 Nc6 Be3 Na5 Bd5 c6
6/15 00:00 14,427 0 +0.61 Nc6 Be3 Qd6 0-0 e6 Rfe1
7/18 00:00 67,733 0 +0.65 Nc6 Bf4 e6 0-0 Qf6 Bg5 Qf5
7/18 00:00 116,861 0 +0.73 Qd6 Bd2 e6 0-0 Qb4 b3 Nc6
8/19 00:00 225,603 0 +0.60 Qd6 0-0 Nc6 Ne4 Qd7 Be3 Qf5 Bd3
9/25 00:00 541,087 0 +0.65 Qd6 0-0 Nc6 Ne4 Qd7 d5 Qf5 Re1 Ne5 Nxe5 Qxe5
10/25 00:01 1,460,830 966,469 +0.39 Qd6 0-0 Nd7 Bd2 Rfd8 Ne4 Qc6 Bd3 Nf6 Ne5
10/30 00:02 2,029,707 986,193 +0.54 Nc6 Bf4 e6 0-0 Qf6 Be3 Qe7 a3 Rad8 d5
11/30 00:03 3,699,634 982,906 +0.58 Nc6 Bf4 e6 Bb5 Nb4 0-0 Nd5 Nxd5 Qxd5 a4 c5 Rfd1
12/32 00:06 6,811,079 997,699 +0.49 Nc6 Bf4 e6 Bb5 Nb4 0-0 Nd5 Nxd5 Qxd5 Bd3 c5 Rfc1 Rad8

[Christopher Conkie]

[d]rn1q1r2/p1p1ppkp/1p4p1/8/2BP4/2N2N2/PP3PPP/R1B1K2R b KQ - 0 11

Houdini 1.0 x32:
1/3 00:00 30 0 -8.70 Qxd4 Nxd4 (this bit is very funny indeed)




Chris

Hmm, what you quote funny (Qxd4 Nxd4) looks very reasonable to me, and I would expect every engine reporting each fail high at the root that way, since Qxd4 is the only capture and tried first in the first iteration of an IID framework. What is the point?

Gerd

I would ask you to look again at the comparison Gerd.

Chris

[Damir]

same evaluation, so what? No prove that engines are are clones or of same strength just because they show exact fail high at the root...
what's your point ?