Impressive Preliminary Results of Rybka 3 by Larry Kaufman!

[mclane]

if you take shredder as example you are maybe right.
but richard lang e.g. was many years on top with the others
behind.

ed schroeder changed this - when i remember it right.
with the chess machines, and before them, excellent results on the 6502 8 bit machines while lang had (since 1985) 68000 and 68020 (factor 2 in speed difference relative to the 68000 machine)
cpu's.

i don't think the others are out of the race.
they will also find out about rybka 3.

and the group is bigger. naum e.g. came very close to rybka.
maybe others will follow.

rybka is the best that could happen to us. vas gives very much pressure
on the others. and everybody is very engaged in catching rybka.

it was the same with all the other leading programs.

rybka is a kind of magic. and the others try to find out what the magic is.

[Dann Corbit]

Great, So now you will to be able to watch Rybka win 1000-0 against everyone, instead of 999-1.

Hooooooooray.

Compe-what?

Shredder dominated for ages. Now it's Rybka's turn. I think that it is wonderful that we can get analysis of this level for the cost of going to a baseball game.

Eventually, even Rybka will be dethroned. After all, "a dragon lives forever, but not so, little boy."

Yes, But did it have such a large gap over No.2?

Did it improve 100~ ELO with every version? I remember Shredder 7.04, Shredder 8, And Shredder 9 wer'e pretty much the same, with some really minor strength improvment.

Also, It never had such a large performance gap over the competition when it was on the verge of releasing a new version. And when a new version did come out, they wer'e minimal.

What wer'e seeing with rybka is completely different if you ask me. Such a large disparity between itself and the other competitors. and wer'e not seeing them catching up at the desired rate. Sure, it will be dethroned, sooner or later. This is the nature of all things, And that's the way it should be. Ofcourse, it's wonderful to see things progress and playing level that we never seen before. I just wish there wer'e more capable competitors, so we could really the highest level of games and have some fun.

Really, no competition equals no fun. Just look at the OS and the CPU Business.

From:
http://ccrlchess.org.uk/ccrl/4040/

Code: Select all

Rank Name Rating Score Average Opponent Draws Games 
1 Rybka 2.3.2a 64-bit 4CPU 3129 +16 -16 71.4% -141.8 41.0% 1432   
2-3 Naum 3.1 64-bit 4CPU 3074 +24 -24 61.3% -74.9 44.2% 541 
2-3 Zappa Mexico II 64-bit 4CPU 3074 +20 -20 63.7% -88.2 46.4% 778 
4 Deep Shredder 11 64-bit 4CPU 3019 +14 -14 53.9% -27.2 44.4% 1580 
5 Toga II 1.4.1SE 4CPU 3012 +22 -21 52.7% -15.2 44.6% 664 
6 Hiarcs 12 4CPU 3011 +18 -18 55.2% -33.0 46.5% 920 
7 Deep Fritz 10.1 4CPU 2992 +18 -18 53.2% -22.4 41.1% 973 
8 Glaurung 2.1 64-bit 4CPU 2991 +27 -27 48.0% +12.8 43.9% 410 
9 Bright 0.3d 4CPU 2979 +30 -30 53.9% -26.0 43.4% 348 
10 Loop M1-T 64-bit 4CPU 2954 +15 -15 46.6% +21.2 45.6% 1495 
11 Deep Junior 10 4CPU 2913 +15 -16 45.3% +32.6 36.0% 1423 
12 Deep Sjeng 2.7 4CPU 2887 +18 -18 41.1% +59.0 38.8% 1020 
13 Spike 1.2 Turin 2853 +8 -8 45.2% +29.3 40.4% 5484 
14 Scorpio 2.0 4CPU 2841 +32 -33 34.9% +101.1 37.2% 312 
15 Thinker 5.1d Passive 2838 +32 -32 45.8% +27.1 37.4% 310 
16 Chessmaster 11 2CPU 2827 +35 -35 47.5% +15.2 39.6% 260 

55 Elo between Rybka and Naum

From:
http://www.husvankempen.de/nunn/40_120_ ... glist.html

Code: Select all

no Program Elo + - Games Score Av.Op. Draws 
1 Rybka 2.3.2a 64 4CPU 3095 22 22 600 73.4 % 2918 40.5 % 
2 Naum 3 x64 4CPU 3023 22 22 550 63.9 % 2924 45.3 % 
3 Zappa Mexico X64 4CPU 3006 19 19 600 61.4 % 2925 51.5 % 
4 Fruit 2.4 Test Beta A [new] 2956 21 21 600 53.8 % 2930 45.7 % 
5 Deep Shredder 11 x64 4CPU 2948 21 21 600 52.6 % 2930 45.5 % 
6 Naum 2.2 x64 4CPU 2928 20 20 550 50.6 % 2924 51.1 % 
7 HIARCS 12 ShPV 4CPU 2922 21 21 550 48.2 % 2934 45.8 % 
8 Deep Fritz 10.1 4CPU 2921 21 21 600 48.3 % 2933 43.3 % 
9 HIARCS 11.2 MP 4CPU 2915 22 22 550 47.2 % 2934 45.3 % 
10 Loop M1-P 4CPU 2870 21 21 600 40.5 % 2937 43.7 % 
11 Glaurung 2.0.1 64 4CPU 2858 21 21 600 38.8 % 2938 42.5 % 
12 Deep Junior 10.1 4CPU 2853 22 22 600 38.0 % 2938 38.3 % 
13 Bright-0.2c 4CPU 2826 23 23 600 34.1 % 2940 33.5 % 

72 Elo between Rybka and Naum

THE SSDF RATING LIST 2007-11-03 %112811 games played by 295 computers

Code: Select all

                                           Rating   +     -  Games   Won  Oppo
                                           ------  ---   --- -----   ---  ----
  10 Shredder 10 UCI  256MB Athlon 1200 MHz  2824   23   -23   980   66%  2707
  12 Shredder 9.0 UCI  256MB Athlon 1200 MHz 2808   20   -20  1280   66%  2694
  14 Shredder 8.0 CB  256MB Athlon 1200 MHz  2800   23   -22  1115   71%  2643
  15 Shredder 7.04 UCI 256MB Athlon 1200 MHz 2791   20   -20  1269   66%  2677
  19 Shredder 7.0  256MB Athlon 1200 MHz     2761   26   -25   841   69%  2624
  30 Shredder 6.0 Pad UCI 256MB Athlon 1200  2715   22   -21  1073   60%  2641
  34 Shredder 6.0  256MB Athlon 1200 MHz     2708   29   -29   587   60%  2638
  39 Shredder 7.0 UCI 128MB K6-2 450 MHz     2682   24   -24   834   43%  2734
  47 Shredder 5.32  256MB Athlon 1200 MHz    2654   25   -25   779   50%  2656
  57 Shredder 6.0 UCI 128MB K6-2 450 MHz     2612   43   -43   264   52%  2601
  61 Shredder 5.32  128MB K6-2 450 MHz       2601   20   -20  1242   42%  2662
  65 Shredder 5.0  128MB  K6-2 450 MHz       2576   16   -16  1818   44%  2618
  67 Shredder 4.0  128MB  K6-2 450 MHz       2566   23   -22   986   58%  2506
  99 Shredder 3.0  64MB P200 MMX             2416   20   -20  1262   48%  2428
 103 Shredder 2.0  58MB P200 MMX             2400   19   -19  1314   44%  2440
 128 Shredder 1.0 Pentium 90 MHz             2282   59   -58   145   53%  2263

Shredder grew a total of 542 Elo from first to last version (a lot of which is hardware).

I think that domination is perhaps even more interesting than parity.

I guess that if Alexander Naumov or Anthony Cozzie spend the same effort with their programs, they would be near current Rybka levels. However, Rybka has a partnership with ChessAssistant, which I am sure will be a big boost for advertizing, etc. and so I think that is the main reason that the others will find it hard to compete.

[Nid Hogge]

HE SSDF RATING LIST 2007-11-03 %112811 games played by 295 computers
Code:

Rating + - Games Won Oppo
------ --- --- ----- --- ----
10 Shredder 10 UCI 256MB Athlon 1200 MHz 2824 23 -23 980 66% 2707
12 Shredder 9.0 UCI 256MB Athlon 1200 MHz 2808 20 -20 1280 66% 2694
14 Shredder 8.0 CB 256MB Athlon 1200 MHz 2800 23 -22 1115 71% 2643
15 Shredder 7.04 UCI 256MB Athlon 1200 MHz 2791 20 -20 1269 66% 2677
19 Shredder 7.0 256MB Athlon 1200 MHz 2761 26 -25 841 69% 2624
30 Shredder 6.0 Pad UCI 256MB Athlon 1200 2715 22 -21 1073 60% 2641
34 Shredder 6.0 256MB Athlon 1200 MHz 2708 29 -29 587 60% 2638
39 Shredder 7.0 UCI 128MB K6-2 450 MHz 2682 24 -24 834 43% 2734
47 Shredder 5.32 256MB Athlon 1200 MHz 2654 25 -25 779 50% 2656
57 Shredder 6.0 UCI 128MB K6-2 450 MHz 2612 43 -43 264 52% 2601
61 Shredder 5.32 128MB K6-2 450 MHz 2601 20 -20 1242 42% 2662
65 Shredder 5.0 128MB K6-2 450 MHz 2576 16 -16 1818 44% 2618
67 Shredder 4.0 128MB K6-2 450 MHz 2566 23 -22 986 58% 2506
99 Shredder 3.0 64MB P200 MMX 2416 20 -20 1262 48% 2428
103 Shredder 2.0 58MB P200 MMX 2400 19 -19 1314 44% 2440
128 Shredder 1.0 Pentium 90 MHz 2282 59 -58 145 53% 2263


Shredder grew a total of 542 Elo from first to last version (a lot of which is hardware).

I think that domination is perhaps even more interesting than parity.

I guess that if Alexander Naumov or Anthony Cozzie spend the same effort with their programs, they would be near current Rybka levels. However, Rybka has a partnership with ChessAssistant, which I am sure will be a big boost for advertizing, etc. and so I think that is the main reason that the others will find it hard to compete.

From Shredder 7.04 to Shredder 9.0 - only 17 ELO points gained. That was my original point.

But up until now Rybka had tremendous success with each new version - and this is just staggering. But ofcourse we shoudln't be fooled. Everyone gets to the point where improving is getting subsequently more diffcult. And Rybka is going to hit that spot somewhen too, Sooner or Later. Honestly I was quite skeptic myself, if they can really turn out another shocker. Seems like I underestimated Larry's work, mainly because I didn't believe in the theory from the first place, But Larry seem to do alot of great work - Kudos.

RE. ChessAssistant. Child game for Chessbase ofcourse. But since their business is not under a serious threat, they are stagnating for many years now. So every year or so you get the same Fritz with a new number follwing it in a better looking box. Maybe that launch will finally make them get up their lousy a**es and start innovating once again.

I wish Convekta good luck with that, and Rybka is shaping up to be a behemoth. Well done guys.

P.S Thorsten, thanks for the history lesson, it's always good to learn new things.

[S.Taylor]

Great, So now you will to be able to watch Rybka win 1000-0 against everyone, instead of 999-1.

Hooooooooray.

Compe-what?

Shredder dominated for ages. Now it's Rybka's turn. I think that it is wonderful that we can get analysis of this level for the cost of going to a baseball game.

Eventually, even Rybka will be dethroned. After all, "a dragon lives forever, but not so, little boy."

I hope that this Rybka now will have reached the stage where it has no real weaknesses left, and that at its weakest point, and in the positions in which it is weakest, it will still be high level GM at every move. Plus, the greatest tactical virtuosity that exists in programs today, in the coming year.
If that is the case, then it may never be crushed, and only laboriously dethroned (Version 3) in a few years from now.
Small bugs may be cleared up by patches, after release. And in this case, we may have a good enough chess engine for what _I_ have been waiting for.
It might play more unbelievable major schemes than Alekhine or any other human ever could, and any major schemes and plans it doesn't play would be seen to be unsound, after deep analysis. Or atleast, answerable.
I would also love to see it beating gm's with night odds, and on a frequent basis!
If all the above is achieved, then we have something for humans to really learn from. And this is what _I_ have been waiting for.

I mean, I'd like to see the stage that GM's no longer "Know" what to do, the whole time. But that they have to REALLY know they are in for a challenge, as if playing a human GM who comes up with novel surprises, and cannot be outsmarted because of "knowing" anti-computer chess.

[Bill Rogers]

I think that the main difference between the growth of Rybak over Rebel is the fact that Larry is a GM if not an IM and not only that he understand computer chess extremely well.
Bill

[Dann Corbit]

I think that the main difference between the growth of Rybak over Rebel is the fact that Larry is a GM if not an IM and not only that he understand computer chess extremely well.
Bill

Larry is an IM.
Vas is an IM.
Iweta Rajlich is also a very strong chess player:
http://www.rybkachess.com/index.php?auswahl=Rybka+team

However, the combination of excellent chess player and excellent programmer is not new.

Vincent is a FM:
http://ratings.fide.com/card.phtml?event=1003054

Hans Berliner was a superb correspondence chess player:
http://en.wikipedia.org/wiki/Hans_Berliner
"As of March 31, 2005, Berliner still had by far the highest International Correspondence Chess Federation (ICCF) rating of any player in the United States, at 2726, 84 points above the second-highest rated player. (Chess Life 2005:37). Berliner's 2726 rating places him third on the ICCF's world list, behind Joop van Oosterom (2741) and Ulf Andersson (2736) [2] (accessed 2008-05-08)."

GMs assisted on both the Deep Blue project and the Hydra project:
http://en.wikipedia.org/wiki/Hydra_(chess)

From:
http://sjeng.org/ftp/deepblue.pdf
"8.1 Opening book
The opening book in Deep Blue was created by hand, primarily by Grandmaster Joel Benjamin, with assistance from Grandmasters Nick De Firmian, John Fedorowicz, and Miguel Illescas. The book consisted of about 4,000 positions, and every position had been checked by Deep Blue in overnight runs. The openings were chosen to emphasize positions that Deep Blue played well. In general this included tactically complex openings, but also included more positional openings that Deep Blue handled well in practice. Opening preparation was most extensive in those openings expected to arise in match play against Kasparov. In fact, none of the Kasparov-specic preparation arose in the 1997 match.
Prior to a game, a particular repertoire was chosen for Deep Blue. There were a number of possible repertoires to choose from, and the choice would be made on the basis of the match situation and the previous experience playing with the same color. Last minute changes or corrections were made in a small "override" book."

There are some other strong chess players that have written chess programs.

Gaviota's author is a Fide Master:
http://ratings.fide.com/seek.phtml?idco ... a&offset=0

Gullydeckel's author Martin Borriss is a very strong player:
http://ratings.fide.com/seek.phtml?idco ... s&offset=0

I seem to recall Sune Fischer as an IM, but I can't find a rating for him.

Uri Blass is a very strong correspondence player (Israel champion). Here is his Fide rating:
http://ratings.fide.com/seek.phtml?idco ... s&offset=0

I am not sure how much we can conclude about the combination of a good chess programmer and a strong chess player in creation of a chess program. One thing we do know for sure, is that it worked out well for the Rybka team.

[Tony Thomas]

Addendum..

Johannes Zwanzger, programmer of Jonny.

[lexdom]

Little Goliath author

http://ratings.fide.com/seek.phtml?idco ... t&offset=0

[Albert Silver]

Great, So now you will to be able to watch Rybka win 1000-0 against everyone, instead of 999-1.

Hooooooooray.

Compe-what?

From:
http://ccrlchess.org.uk/ccrl/4040/

Code: Select all

Rank Name Rating Score Average Opponent Draws Games 
1 Rybka 2.3.2a 64-bit 4CPU 3129 +16 -16 71.4% -141.8 41.0% 1432   
2-3 Naum 3.1 64-bit 4CPU 3074 +24 -24 61.3% -74.9 44.2% 541 
2-3 Zappa Mexico II 64-bit 4CPU 3074 +20 -20 63.7% -88.2 46.4% 778 
4 Deep Shredder 11 64-bit 4CPU 3019 +14 -14 53.9% -27.2 44.4% 1580 
5 Toga II 1.4.1SE 4CPU 3012 +22 -21 52.7% -15.2 44.6% 664 
6 Hiarcs 12 4CPU 3011 +18 -18 55.2% -33.0 46.5% 920

55 Elo between Rybka and Naum

Myeah... I think that at least for previous versions, Rybka scales a lot worse on multiple CPUs. Rybka 3 may do better, who can say? Compare the single-CPU results for those same engines:

Code: Select all

1	Rybka 2.3.2a 64-bit	3067
2	Naum 3.1 32-bit	2971
 	Naum 3.1 64-bit	2970
3	Deep Shredder 11 64-bit 1CPU	2963	
4	Zappa Mexico II 64-bit	2961
5	Fritz 11	2956	
 	Shredder 11	2942	
6	Toga II 1.4.1SE	2938
7	Hiarcs 12	2920	

I guess that if Alexander Naumov or Anthony Cozzie spend the same effort with their programs, they would be near current Rybka levels.

Possibly. I'm not knocking their efforts by any means, but a number of talented programmers have put in a lot of effort to do that, and few have reached and held the number one spot for any length of time on equal hardware. In fact, other than Vas, I can only think of three: Lang, SMK, and Morsch.

It is actually almost flabbergasting to note that even the 2006 version of Rybka, v1.2 would actually be neck-to-neck with the latest versions of its competitors, if not a bit ahead.

[palmer]

Thorsten,

for years you have told us this :

Classical paradigm
==================
When should we expect a major breakthrough in science ?
When will a lone developer 'step through the looking-glass' ?
Who will this developer be ?

The answer to the above two questions is of course whenever the old, classical programmers say 'we've reached perfection, there is no way to improve'; when the old paradigm says 'there is only one way'; when all the developers produce roughly equal results.

This is the situation we have today with chess programs. The classical paradigm is represented by Fritz3: fast and simple evaluation, pre-processing of the position before the search; and all strength, all hopes, in the search - nodes per second and search efficiency are the buzzwords.

For a classical program, to keep the search fast, the evaluation at each node must, of necessity, be brief. This evaluation is usually no more than a weighting given for each piece on each square (for example a knight might be worth 3.3 pawns on centre squares and 2.9 pawns on edge squares) and evaluation of the pawn structure for doubled pawns, passed pawns etc..
The classical pre-processing function looks for themes in the position and adjusts the square weightings accordingly - for example, if a knight is attacking a square next to the king, then increase the weighting for all the squares that the queen could cooperate with the knight in making a king attack, increase the knight weighting to keep it on the original square, increase other cooperating piece weightings and so on.

There is no doubt that this approach works but it cannot be the way forward.


And now a "beancounter" like Rybka is by far the leader of the pack for years.