SWISSTEST: 1.RobboLito - 2.Rybka - 3.Stockfish

[kingliveson]

I posted the results of Rybka vs RobboLito yesterday. I had run the test with ponder on--though it seems RobboLito was pondering, it was made known to me that it was in vain because the code itself had not been implemented--and that gave Rybka an advantage in the test. Yet RobboLito came out ahead. I ran a test overnight with the latest RobboLito which actually supports ponder.

The test was conducted using Fritz 11 GUI on Windows 7 64-bit. The hardware is an AMD Phenom II 940 overclocked to 3.6 GHz.

Both engines used the same book renamed to bookone.ctg (Rybka 3) and booktwo.ctg (Robbolito 0.085e4). The book has no marked/priority moves. The book settings are 2/12 with learning mode disabled. The variety of play was set at approximately 71.4% which is 20/28 right arrow key.

The engines were 1 CPU 64 bit versions with hash size set to 256 MB and pondering/permanent brain was enabled. Rybka used contempt setting of 15 (default). Time control is 4 minutes with no increment.

Result:

Code: Select all

1	RobboLito 	+35/=48/-17	59.00%		59.0/100

2	Rybka 3		+17/=48/-35	41.00%		41.0/100

More games to come...

[Albert Silver]

Probably there are some "bugs" in your analysis...:

No. 3 - Much better moves than Qf3?/Re3?/g3?: f3 and Re4 (Of course Black is still worse, but that's not the question...)

No.20 - Exchange of the bishops leads to a lost pawn endgame; in contrast these endgame B vs B has a great chance for a draw (I see no forced win for white after i.e. 1...Bd7 or Bc8 followed by centralization of the king)

No.46 - Similar to No. 20: Exchange of the rooks leads to a lost pawn endgame; the two rooks increase the chance to a draw. It's a matter of chess endgame knowledge (and a big problem for engines...)

No.48 - Qb6? Absolutely not the best move in the position; on the contrary, probably he lost the game.
Alternatives with the ability to compensate: ...Bd7 (with returning of material) or ...d5 (with counter play).

Thanks for your effort: Walter

.

It is a pity you didn't read what I wrote. Aside from position 48, it is your analysis that is flawed I'm afraid.

No.3 is LOST. f3 and Re4 lead to an equally LOST position.

No. 20 is also LOST. Just because you do not understand the endgame doesn't make it any less lost. White's plan is pure textbook: pawns to a4,b4, king to d4, Bishop to d3 to keep the Black bishop tied down. Black can play his king to d6, then bishop to b7 or c8, and then only wait to die. When the timing is correct, White plays a5, then b5, Bxb5, and a6. Black will soon find himself in zugzwang, and that is it.

No. 46, again you fail to understand the endgame. Though Rc4 loses for Black, so do all other moves. There is no draw except by blunder.

No. 48, I agree Qb6 is not best. It was after a long testing period, and I failed to notice black is up a piece and can give one back to equalize. Mea culpa.

Considering the inability to evaluate the endgame, the question of your chess strength is relevant. What is your rating?

[Glarean]

Probably there are some "bugs" in your analysis...:

No. 3 - Much better moves than Qf3?/Re3?/g3?: f3 and Re4 (Of course Black is still worse, but that's not the question...)

No.20 - Exchange of the bishops leads to a lost pawn endgame; in contrast these endgame B vs B has a great chance for a draw (I see no forced win for white after i.e. 1...Bd7 or Bc8 followed by centralization of the king)

No.46 - Similar to No. 20: Exchange of the rooks leads to a lost pawn endgame; the two rooks increase the chance to a draw. It's a matter of chess endgame knowledge (and a big problem for engines...)

No.48 - Qb6? Absolutely not the best move in the position; on the contrary, probably he lost the game.
Alternatives with the ability to compensate: ...Bd7 (with returning of material) or ...d5 (with counter play).

Thanks for your effort: Walter

.

>No.3 is LOST. f3 and Re4 lead to an equally LOST position.

That's not the question, you don't have the point:
f3 & Re4 get a much better defense.

>No. 20 is also LOST.

See below: It's a long way from a lost position to the final win. So it's not the question: Who is winning? The question is: What's the biggest resistance?

>No. 46 Though Rc4 loses for Black, so do all other moves. There is no draw except by blunder.

Again, you don't understand the job of these positions. In this three (and others) positions we test the best defence; results are not really important.

>No. 48, I agree Qb6 is not best. It was after a long testing period, and I failed to notice black is up a piece and can give one back to equalize. Mea culpa.

Ok.

>Considering the inability to evaluate the endgame, the question of your chess strength is relevant. What is your rating?

Too high for you, I see...

.

[Albert Silver]

Probably there are some "bugs" in your analysis...:

No. 3 - Much better moves than Qf3?/Re3?/g3?: f3 and Re4 (Of course Black is still worse, but that's not the question...)

No.20 - Exchange of the bishops leads to a lost pawn endgame; in contrast these endgame B vs B has a great chance for a draw (I see no forced win for white after i.e. 1...Bd7 or Bc8 followed by centralization of the king)

No.46 - Similar to No. 20: Exchange of the rooks leads to a lost pawn endgame; the two rooks increase the chance to a draw. It's a matter of chess endgame knowledge (and a big problem for engines...)

No.48 - Qb6? Absolutely not the best move in the position; on the contrary, probably he lost the game.
Alternatives with the ability to compensate: ...Bd7 (with returning of material) or ...d5 (with counter play).

Thanks for your effort: Walter

.

>No.3 is LOST. f3 and Re4 lead to an equally LOST position.

That's not the question, you don't have the point:
f3 & Re4 get a much better defense.

>No. 20 is also LOST.

See below: It's a long way from a lost position to the final win. So it's not the question: Who is winning? The question is: What's the biggest resistance?

>No. 46 Though Rc4 loses for Black, so do all other moves. There is no draw except by blunder.

Again, you don't understand the job of these positions. In this three (and others) positions we test the best defence; results are not really important.

>No. 48, I agree Qb6 is not best. It was after a long testing period, and I failed to notice black is up a piece and can give one back to equalize. Mea culpa.

Ok.

>Considering the inability to evaluate the endgame, the question of your chess strength is relevant. What is your rating?

Too high for you, I see...

.

Ok, you are a weak player, that is ok.

You are right, I do not understand the point of proposing moves to avoid if everything loses. The endgames have no defense. Saying some moves are preferable when all moves lose is .... stupid. In that spirit, I have a new position for your test:

[D]3k4/7Q/3K4/8/8/8/8/8 b - - 0 0

Moves to avoid: Kc8

If required, I can provide others.

[Glarean]

Probably there are some "bugs" in your analysis...:

No. 3 - Much better moves than Qf3?/Re3?/g3?: f3 and Re4 (Of course Black is still worse, but that's not the question...)

No.20 - Exchange of the bishops leads to a lost pawn endgame; in contrast these endgame B vs B has a great chance for a draw (I see no forced win for white after i.e. 1...Bd7 or Bc8 followed by centralization of the king)

No.46 - Similar to No. 20: Exchange of the rooks leads to a lost pawn endgame; the two rooks increase the chance to a draw. It's a matter of chess endgame knowledge (and a big problem for engines...)

No.48 - Qb6? Absolutely not the best move in the position; on the contrary, probably he lost the game.
Alternatives with the ability to compensate: ...Bd7 (with returning of material) or ...d5 (with counter play).

Thanks for your effort: Walter

.

>No.3 is LOST. f3 and Re4 lead to an equally LOST position.

That's not the question, you don't have the point:
f3 & Re4 get a much better defense.

>No. 20 is also LOST.

See below: It's a long way from a lost position to the final win. So it's not the question: Who is winning? The question is: What's the biggest resistance?

>No. 46 Though Rc4 loses for Black, so do all other moves. There is no draw except by blunder.

Again, you don't understand the job of these positions. In this three (and others) positions we test the best defence; results are not really important.

>No. 48, I agree Qb6 is not best. It was after a long testing period, and I failed to notice black is up a piece and can give one back to equalize. Mea culpa.

Ok.

>Considering the inability to evaluate the endgame, the question of your chess strength is relevant. What is your rating?

Too high for you, I see...

.

.... stupid. In that spirit, I have a new position for your test:

[D]3k4/7Q/3K4/8/8/8/8/8 b - - 0 0

Moves to avoid: Kc8

Many thx, you are very... friendly.

.

[Albert Silver]

Actually, I was hoping you'd explain the purpose of testing for moves when all moves have the same end result.

Still, the bishop endgame will make a good class subject. I give a weekly class at my club, usually on tactics and analysis, but this is a good one too, as I'm sure many of the students (the highest rating is only ~1850) will have similar difficulties in evaluating it.

[Rolf]

Saying some moves are preferable when all moves lose is .... stupid. In that spirit, I have a new position for your test:

[D]3k4/7Q/3K4/8/8/8/8/8 b - - 0 0

Moves to avoid: Kc8

If required, I can provide others.

Thanks so far, Albert. The point is with testers in general, I mean those who create tests with chosen positions, that they are very sensitive towards critics. Because in their eyes it's all so clear. Everything, the choice and the results. In the original German forum, where Walter presented the test results, which were calculated by another German tester, not Walter, Walter reacted on first critical comments because of strange results, that the test cant be totally false if the actual, now smell that sound, worldwide strongest program would lead on first place!

I'm not an expert in these tests, so I have no idea if with experience on could now say, well of course such tasks in blitz, if it's for endings, Rybka isnt the overall fastest and best. I dont know this. But assumed it is a fact. Then Walter gives some of his no. 20 where a beginner like me has no clue what this should measure. What message does he have? That no 20 is better solved by a speed monster that has no experienced endgame power but it crushes Rybka anyway? What is this goiod for?

Under the impression of such weak tests, where even I could compete with lower hardware and the most rudimentary version of Rybka, I begin to understand what Vas has in mind with his online Rybka. If you are a chessplayer you often need a reliable top verdict for a situation. And this is simply impossible to do with a naked version of even Rybka and then on normal hardware. The absolute burner comes only out of Rybka if she plays online likje she plays in those tournaments where operators have so called clusters which leave nothing but burned areas behind them.

Woulkd you somehow agree with my view? On lower hardware level even patzers can compete in such tests which seems to be the message in the interest of the many programmers who have no chance to compete with the Wch. But we here should present a bit of more reliability, no?

Vas is always asked by folks who dont want to understand that quality in chess has its price. Vas explains patiently that you also pay for you car. But foks insist, no, I go to my frind in the garden. Fine, but in chess it's fatal if your grandmother is the only advisor for the next tournament. Dovoretzky or Rybka wouldnt be bad either, or not? For the poor there are many freeware engines, but if you want to drive a Porsche on the net, then you must not blink but buy Porsche-Rybka. Like all professionals. No more disk crashs, memory defects, installation probs, what over the years saves you much money.

Would you really want to drive a monster machine with anonymous, nameless reckless, robbers? I dont want that.Chess isnt a game for others than distinguished gentlemen.

My butler calls for dinner in the Main Hall.

[bob]

Probably there are some "bugs" in your analysis...:

No. 3 - Much better moves than Qf3?/Re3?/g3?: f3 and Re4 (Of course Black is still worse, but that's not the question...)

No.20 - Exchange of the bishops leads to a lost pawn endgame; in contrast these endgame B vs B has a great chance for a draw (I see no forced win for white after i.e. 1...Bd7 or Bc8 followed by centralization of the king)

No.46 - Similar to No. 20: Exchange of the rooks leads to a lost pawn endgame; the two rooks increase the chance to a draw. It's a matter of chess endgame knowledge (and a big problem for engines...)

No.48 - Qb6? Absolutely not the best move in the position; on the contrary, probably he lost the game.
Alternatives with the ability to compensate: ...Bd7 (with returning of material) or ...d5 (with counter play).

Thanks for your effort: Walter

.

>No.3 is LOST. f3 and Re4 lead to an equally LOST position.

That's not the question, you don't have the point:
f3 & Re4 get a much better defense.

>No. 20 is also LOST.

See below: It's a long way from a lost position to the final win. So it's not the question: Who is winning? The question is: What's the biggest resistance?

>No. 46 Though Rc4 loses for Black, so do all other moves. There is no draw except by blunder.

Again, you don't understand the job of these positions. In this three (and others) positions we test the best defence; results are not really important.

>No. 48, I agree Qb6 is not best. It was after a long testing period, and I failed to notice black is up a piece and can give one back to equalize. Mea culpa.

Ok.

>Considering the inability to evaluate the endgame, the question of your chess strength is relevant. What is your rating?

Too high for you, I see...

.

Ok, you are a weak player, that is ok.

You are right, I do not understand the point of proposing moves to avoid if everything loses. The endgames have no defense. Saying some moves are preferable when all moves lose is .... stupid. In that spirit, I have a new position for your test:

[D]3k4/7Q/3K4/8/8/8/8/8 b - - 0 0

Moves to avoid: Kc8

If required, I can provide others.

I think his point is about judgement of the best move to play, not the eventual result of the game. There are lots of positions where the best move is clear, even though the best move still loses.

I'm not a fan of these types of positions, as if your program is strong enough, it may well see that everything loses, and then choosing the best move is really irrelevant, if you can "see".

[Paloma]

Some analysis with Stockfish 1.5.1:

3: SwissTest4_03 - Verteidigung, Aristarch45-TheBaron15, CompGame 2005
r1b2rk1/2p2ppp/p2b4/1p6/6Pq/1BP5/PP1P1PP1/RNBQR1K1 w - - 0 1

Analysis by Stockfish 1.5.1 JA 64bit: (after 10:00 min, Depth 21)

1. -/+ (-0.80): 15.Te4 Dh2+ 16.Kf1 Dh1+ 17.Ke2 Lxg4+ 18.Txg4 Tae8+ 19.Kd3 Te1 20.Dxe1 Dxe1 21.Kc2
2. -+ (-1.45): 15.f3 Dh2+ 16.Kf1 Lg3 17.Te4 c5 18.d4 Dh1+ 19.Ke2 Dxg2+ 20.Kd3 c4+ 21.Lxc4 bxc4+
3. -+ (-3.19): 15.Df3 Lh2+ 16.Kf1 Lxg4 17.De4 Lf4 18.g3 Dh2 19.gxf4 Lh3+ 20.Ke2 Tae8 21.d3 Lg4+

( 06.12.2009)

3: SwissTest4_03 - Verteidigung, Aristarch45-TheBaron15, CompGame 2005
r1b2rk1/2p2ppp/p2b4/1p6/6Pq/1BP5/PP1P1PP1/RNBQR1K1 w - - 0 1

Analysis by Stockfish 1.5.1 JA 64bit: (after 38:00 min, Depth 23)

1. -/+ (-1.29): 15.Te4 Dh2+ 16.Kf1 Dh1+ 17.Ke2 Lxg4+ 18.Txg4 Tae8+ 19.Kd3 Te1 20.Dxe1 Dxe1 21.Kc2
2. -+ (-2.26): 15.f3 Lg3 16.Te4 Kh8 17.Ld5 f5 18.gxf5 Dh2+ 19.Kf1 Lxf5 20.d4 Tad8 21.Le3 Dh1+
3. -+ (-3.35): 15.Df3 Lh2+ 16.Kf1 Lxg4 17.De4 Lf4 18.g3 Dh2 19.gxf4 Lh3+ 20.Ke2 Tfe8 21.d3 Lg4+ 22.Ke3 Txe4+ 23.dxe4 Dh3+ 24.Kd4 Td8+ 25.Ld5 c6 26.c4 bxc4 27.Le3 cxd5 28.exd5 Tc8 29.Sa3 Df3 30.Sxc4

( 06.12.2009)

The different between 1st and 2nd move is almost 1 pawn, and Re4 is NOT the move to avoided. Is so, the Pos. ist solved.

This is the point that Walter means.

[bnemias]

I'm not a fan of these types of positions, as if your program is strong enough, it may well see that everything loses, and then choosing the best move is really irrelevant, if you can "see".

I understand this point, but at the same time I would prefer to have the program play to extend the game as long as possible when losing, or to win as quickly as possible when winning. What to do when the position is equal is more interesting.

Here's why: I have personally witnessed hundreds of games where the final result was altered by one side's flag falling. This doesn't just occur at fast time controls either-- it's a real possibility anytime there's a zero increment game.

I should add that a significant percentage of those are not really due to time management issues, but to unexpectedly long games where the result is obvious but altered anyway.