Rybka vs Fat Fritz controversies

[Raphexon]

I think for a while, everyone thought that chess engine progress will improve ... but it will improve mostly due to the improvement in hardware running bean counter engines. Rybka changed that thinking and it made clear that the software side of chess engines is where the major gain in ELO would come from. This has been made even more true with the giant leap that Lc0 gave and later on the inclusion of NN to almost any chess engine. I think you could say with a straight face that Rybka with revolutionary system of using millions of rapid games to tune evaluation as well as some Monte Carlo experimentation was a primitive form of NN. The strength of these new NN chess engines at low depth is mind boggling. I would not be surprised if today's top NN chess engine would beat Deep Blue with only an Apple watch as hardware. A single core SF 13 against DB would probably be an incredibly lopsided affair to SF. It is too bad that If you go through games of DB with SF 13 or Lc0 ... there are so many inaccuracies and weak moves that just makes me think that DB is probably 400 or more ELO points weaker. So software turned out to be a much bigger factor in moving engine ELO forward than hardware would ever have done.

An Apple watch is already way overkill.
SF in Fishtest STC is already stronger than any human that ever existed.

Deep Blue would need somewhere between 1:1000-1:10000 time odds to not get beaten up.
If assuming 50 elo per doubling and assuming DB would be 2800 on CCRL40/4 (liberal estimate I think) then it'd need 1:250000+ time odds to beat SF.

[towforce]

I think for a while, everyone thought that chess engine progress will improve ... but it will improve mostly due to the improvement in hardware running bean counter engines. Rybka changed that thinking and it made clear that the software side of chess engines is where the major gain in ELO would come from. This has been made even more true with the giant leap that Lc0 gave and later on the inclusion of NN to almost any chess engine. I think you could say with a straight face that Rybka with revolutionary system of using millions of rapid games to tune evaluation as well as some Monte Carlo experimentation was a primitive form of NN. The strength of these new NN chess engines at low depth is mind boggling. I would not be surprised if today's top NN chess engine would beat Deep Blue with only an Apple watch as hardware. A single core SF 13 against DB would probably be an incredibly lopsided affair to SF. It is too bad that If you go through games of DB with SF 13 or Lc0 ... there are so many inaccuracies and weak moves that just makes me think that DB is probably 400 or more ELO points weaker. So software turned out to be a much bigger factor in moving engine ELO forward than hardware would ever have done.

+1 Good post.

[Dann Corbit]

All true, but if the same team built DB today, I guess it would be absurdly strong.
They could make much faster chips, and give them more features and they would be supremely well debugged.
They would have the same advantages of the modern algorithms.

The problem with the hardware approach to chess that makes it a no-go is that once you make one, it is fairly well cast in stone.
So your engine is a world beater that costs millions of dollars.
But both software and hardware march ahead exponentially.
In a few years commodity hardware and software costing $2000 will clobber a $20 million dollar hardware solution.

So does the hardware idea sound good to anyone?

I guess that Hydra can also be beaten by cell phones today

[Frank Quisinsky]

Hi there,

from the view of the style of the engine ...

Rybka have all the time for the high Elo a catastrophal mid-game and here the same problems Fruit have. The different to Fruit is the strong endgame Rybka have. And after all my information the endgame knowledge comes also not from Vas himself.

Little example:

Code: Select all

Statistics to short won / draw games:
After round 50 out of 50 (final results) = 2.000 games per engine

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

2.242 of 41.000 = 5,469% : Fast won/lost games below 60 moves (with mate ended)
1.488 of 41.000 = 3,630% : Fast draw games below 40 moves

                                  won59 lost59 draw39        won59w won59b lost59w lost59b
01. Stockfish 11 BMI2 x64           243      0     71           172     71       0       0
02. Komodo 14.0 BMI2 x64            172      0     83           125     47       0       0
03. Houdini 6.03 Pro x64            164      0     56           110     54       0       0
04. SlowChess BC 2.2 x64            121      3     98            85     36       0       3
05. Booot 6.4 POP x64               115      0    105            88     27       0       0
06. Ethereal 12.25 PEXT x64         115      1     71            79     36       1       0
07. Xiphos 0.6 BMI2 x64             103      3     84            75     28       0       3
08. Fizbo 2.0 BMI2 x64               96     31     66            62     34      10      21
09. Andscacs 0.95 BMI2 x64           90      9     81            54     36       3       6
10. Schooner 2.2 SSE x64             80      5     84            55     25       4       1
11. Defenchess 2.2 POP x64           76      6     85            50     26       0       6
12. Laser 1.7 BMI2 x64               71      2     89            45     26       1       1
13. Fire 7.1 POP x64                 69      5     96            48     21       1       4
14. rofChade 2.3 BMI x64             67     13     70            44     23       3      10
15. Fritz 17 (Ginkgo) x64            65     19     78            38     27       6      13
16. Arasan 22.0 BMI2 x64             44     31     77            31     13      13      18
17. Shredder 13 x64                  43     20     71            29     14       1      19
18. Chiron 4 x64                     43     33     69            29     14       9      24
19. Pedone 2.0 BMI2 x64              43     39     57            26     17      12      27
20. RubiChess 1.7.3 x64              40     19     83            23     17       6      13
21. Wasp 4.00 Modern x64             34     12     81            23     11       3       9
22. Vajolet2 2.8 BMI2 x64            31     26     82            24      7       7      19
23. Critter 1.6a x64                 26     70     64            17      9      28      42
24. Nemorino 5.00 BMI2 x64           25     77     55            21      4      28      49
25. Winter 0.8 x64                   25    129     57            13     12      35      94
26. Texel 1.07 BMI2 x64              24     70     63            19      5      21      49
27. Igel 2.5.0 BMI2 x64              24     80     94            16      8      24      56
28. Demolito 2020-05-14 PEXT x64     22     52     35            17      5      11      41
29. GullChess 3.0 BMI2 x64           21     52     76            19      2      15      37
30. iCE 4.0 v853 Modern x64          19     84     72            15      4      26      58
31. Fritz 16 (Rybka) x64             18    118     68            11      7      33      85
32. Protector 1.9.0 x64              17     70     77            11      6      22      48
33. Nirvanachess 2.4 POP x64         16     65     87             9      7      20      45
34. Equinox 3.30 x64                 15     93     59            10      5      27      66
35. Rodent IV 0.22 POP x64           11     85     70            10      1      28      57
36. Senpai 2.0 BMI2 x64              11    144     63             6      5      54      90
37. Minic 2.33 x64                   11    155     65             7      4      42     113
38. SmarThink 1.98 AVX2 x64          11    211     49             5      6      65     146
39. Hannibal 1.7 x64                 10     86     59             6      4      29      57
40. Combusken 1.2.0 x64               8    193     54             4      4      60     133
41. Monolith 2 PEXT x64               4    130     72             3      1      56      74

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

  654 of 41.000 = 1,596% : Fast won/lost games below 50 moves (with mate ended)
  525 of 41.000 = 1,281% : Fast draw games below 30 moves

                                  won49 lost49 draw29        won49w won49b lost49w lost49b
01. Stockfish 11 BMI2 x64            64      0     30            46     18       0       0
02. Houdini 6.03 Pro x64             52      0     13            37     15       0       0
03. Komodo 14.0 BMI2 x64             50      0     31            35     15       0       0
04. Booot 6.4 POP x64                38      0     26            31      7       0       0
05. SlowChess BC 2.2 x64             32      0     28            23      9       0       0
06. Ethereal 12.25 PEXT x64          32      0     33            25      7       0       0
07. Xiphos 0.6 BMI2 x64              31      0     29            20     11       0       0
08. Fizbo 2.0 BMI2 x64               30      6     14            15     15       1       5
09. Schooner 2.2 SSE x64             25      1     29            17      8       1       0
10. Andscacs 0.95 BMI2 x64           25      3     25            17      8       1       2
11. Defenchess 2.2 POP x64           21      0     33            15      6       0       0
12. Fritz 17 (Ginkgo) x64            21      1     24            14      7       0       1
13. Fire 7.1 POP x64                 20      2     35            14      6       1       1
14. Laser 1.7 BMI2 x64               19      0     41            10      9       0       0
15. rofChade 2.3 BMI x64             19      4     27             9     10       1       3
16. Chiron 4 x64                     15      4     20            12      3       1       3
17. Wasp 4.00 Modern x64             13      4     25            11      2       1       3
18. Critter 1.6a x64                 12     17     22             8      4       6      11
19. Winter 0.8 x64                   12     31     23             6      6       4      25
20. RubiChess 1.7.3 x64              11      0     27            10      1       0       0
21. Pedone 2.0 BMI2 x64              11      7     18             4      7       2       5
22. Shredder 13 x64                   9      5     26             7      2       0       5
23. Arasan 22.0 BMI2 x64              9      6     26             6      3       2       4
24. Nemorino 5.00 BMI2 x64            9     18     18             8      1       8      10
25. Vajolet2 2.8 BMI2 x64             8     10     33             8      0       2       8
26. Texel 1.07 BMI2 x64               7     18     23             6      1       5      13
27. Igel 2.5.0 BMI2 x64               7     30     28             5      2       8      22
28. Protector 1.9.0 x64               6     19     36             4      2       4      15
29. Demolito 2020-05-14 PEXT x64      5     11      8             3      2       3       8
30. Nirvanachess 2.4 POP x64          5     13     29             3      2       1      12
31. GullChess 3.0 BMI2 x64            5     15     19             5      0       5      10
32. iCE 4.0 v853 Modern x64           5     20     35             4      1       9      11
33. Equinox 3.30 x64                  4     26     27             4      0       9      17
34. SmarThink 1.98 AVX2 x64           4     43     20             1      3      14      29
35. Senpai 2.0 BMI2 x64               4     49     19             3      1      17      32
36. Fritz 16 (Rybka) x64              4     52     21             2      2      15      37
37. Hannibal 1.7 x64                  3     26     22             3      0       7      19
38. Minic 2.33 x64                    3     52     21             2      1      12      40
39. Rodent IV 0.22 POP x64            2     21     29             2      0       4      17
40. Monolith 2 PEXT x64               2     28     34             2      0      10      18
41. Combusken 1.2.0 x64               0    112     23             0      0      39      73

Short game stats from FCP Tourney-2020.
All the fast loses are a disaster.
No engine with +-150 Elo produced so many fast lost games!!

Nothing in Rybka chess engines is interesting ... NOTHING
For strong humans are the mid-game important, how strong is an engine after opening theory.

I never understand the publicity from a company like chessbase around Rybka because I am quite sure that most of stronger GM find out after a short time the deaster mid-game moves by Rybka.

But people are looking on Elo only and we all have not the strength to see the problems in detail. We need statistics!!

The only good point is the very strong endgame.
The same strong endgame later in Doch / Komodo is available.
The important parts from the endgames comes from Lary Kaufmann!

From Vas comes the code from Fruit, maybe parts from Crafty and a lot of updates in times he public Rybka 1. He fixed again and again simple source-code mistakes ... not what professionals do.

Not more not less ... no other opinion if I am looking in Rybka statistics.
And to look in statistics is very easy!

The community like to build a mythos.
Some are wrong!

Best
Frank

[Frank Quisinsky]

Hi there,

never I saw such a bad opening book Rybka have.
Such a passiv chess program need closed opening systems.
The book is full of open danger lines ... complete wrong.

A good example is Shredder!
Shredder at this time have around the same problems Rybka have.
Not an engine like to play aggressive chess or able to win many short games.

The different is:
The opening book is very strong.
Great closed lines and not so many danger open-lines Rybka have.

That's why Shredder have often the advantage in tournaments because the style of engine and the book of engine build a whole unit!

And a company like Chessbase with so many grandmaster working for it are not able to see such easy things?
Chessbase have also only one point in the brain ... the number one (not important are strengths and weeknesses or how build the engine). Houdini as next is the best proof for it.

Best
Frank

[carldaman]

It was very easy to build a myth when Rybka gained undisputed dominance and supremacy in the engine world. Since there was nothing better, Rybka held the final word and it was hard to pinpoint flaws in its play and most people became entranced with its abilities and blind to its shortcomings - but, with hindsight, it is now ironically viewed as a clearly weaker engine (as it should).

There ought to be a lesson in there for us to always think critically.

[Dann Corbit]

Because of the exponential advance of chess software, in five years the stockfish of today will look weak compared to whatever is best in 2026.
I don't think we need to be critical of existing software. The ancient Rybka will still clean the clock of 99.99999% of the {unassisted} humans on earth at any time control, so it was pretty good when it was best (like Shredder, Junior, etc.).
For instance, from Pohl's site, SF has risen 418 Elo in 5 years.

I expect the new AMD GPU architecture to be a jaw-dropper.
Coordinated access to video RAM by both the CPU and GPU. Think of it.

[Frank Quisinsky]

Hi Carldaman,

yes, that is right!

Looking in times Fritz are on position 1 in the World.
I produced some fast won games and some fast lost games.
Fritz engines offer a surprise often in the games.

Such engines like Pedone or Fizbo today (speculative and great for humans).

Unfortunately, with newer Fritz versions the great speculative style lost more and more.
Speaking from Quest / Fritz ... the Fritz from Frans Morsch!

Back to Rybka:
Never I am a Rybka fan for some reasons.
Own opinion is a bit subjective but statistics are fact!


Hi Dann,

if we are looking in style of engines, we have so many great attackers never we have in times before. Sure, that Stockfish dev. give the complete engine development big jumping’s. If we are looking in 10 years to the Stockfish dev. from today ... I am very sure that we cannot write that Stockfish today have in one of the gaming phases big problems.

I like to look in the own engine databases with 45-minutes games and I can see around 300-325 Elo more for Stockfish in the last 5 years (testing vs. many different opponents). That is indeed fantastic because Stockfish, 5 years old, is stronger as the most TOP-20 engines today.

Statistics from Sf 1101 (still running FCP Tourney-2021) are more as fantastic, gigantic ... MEGA!
Interesting here to looking in draw games and which opening systems produced the draw games.

Best
Frank

[Krzysztof Grzelak]

Hi there,

never I saw such a bad opening book Rybka have.
Such a passiv chess program need closed opening systems.
The book is full of open danger lines ... complete wrong.

A good example is Shredder!
Shredder at this time have around the same problems Rybka have.
Not an engine like to play aggressive chess or able to win many short games.

The different is:
The opening book is very strong.
Great closed lines and not so many danger open-lines Rybka have.

That's why Shredder have often the advantage in tournaments because the style of engine and the book of engine build a whole unit!

And a company like Chessbase with so many grandmaster working for it are not able to see such easy things?
Chessbase have also only one point in the brain ... the number one (not important are strengths and weeknesses or how build the engine). Houdini as next is the best proof for it.

Best
Frank

Hi Frank.

You write about the program Shredder. This is why he had a poor performance in tournaments TCEC. Why is he not playing in this tournament at all now?. I also noticed an interesting thing. While the author of the engine plays the program himself, he achieves great results. But games come in other tournaments, it's not so good anymore. I always wonder about such a question, is the author of the engine not changing the code and constantly improving it. I buy the program, I have a code from several years, and the engine's author has a modern code, but he does not make it available to buyers - only the old program code.

Krzysztof

[towforce]

So does the hardware idea sound good to anyone?

At the time, big expensive hardware was still the best show in town.

Ultimately, though, chess will be solved by software. Many years ago, I started a thread to discuss the limits of computing applied to chess. I can find those limits again (start with the wiki article about the limits of computing), but roughly speaking, the Planck limits suggest that the upper limit on computing operations per joule of energy is around 10^35 (we don't know how useful such an operation would be in terms of generating chess positions). As best I remember, the known universe was calculated to have generated around 10^75 joules during its existence (give of take a few orders of magnitude), meaning that if the universe had been an optimal chess computer, it could have carried out 10^110 or so operations. At the time, we concluded that this was very likely to be sufficient to search enough of the chess tree to solve chess.

However: that's not going to happen!

I have come up with two ways which are credible (IMO) to solve chess with software, though:

1. Condition tree. Here's a quick and over-simplified version: make a list of conditions that enable checkmate (L1). Generate a list of conditions that enable an L1 condition to be achieved (L2). Repeat this operation until every new condition generated already exists in the tree. You then have the list of conditions for a position to be winnable. Obviously there's a lot more to say about this, but the biggest thing is that right now, nobody knows how big the tree would be.

2. Deep complex patterns. To me, the evidence suggests that Leela's huge net is finding a large number of simple, surface-level patterns rather than a small number of deep complex patterns. For me, the evidence strongly suggests that deep, complex patterns exist in chess, and that finding them will boost the level of play (think rocket engine v lawn mower)