Some results from Larry with finalized Beta

[mclane]

to me it looks like the COMMENTS of the other chess programmers concerning Rybka were not very precise.

they said rybka is nothing special, there is no wonder in rybka, everything just good solid work. but now the others are not 50 elo behind but 250 ELO.

50 elo is not much.
but 250 elo is a world.

and i have the feeling that all these guesses about how rybka works was just a bluff and no real analyse.

how can a program be so much ahead of the other programs ?
to me it seems the other programmers have no clue how rybka works.
and the longer this situation remains the bigger the gap between them
develos.

[AdminX]

to me it looks like the COMMENTS of the other chess programmers concerning Rybka were not very precise.

they said rybka is nothing special, there is no wonder in rybka, everything just good solid work. but now the others are not 50 elo behind but 250 ELO.

50 elo is not much.
but 250 elo is a world.

and i have the feeling that all these guesses about how rybka works was just a bluff and no real analyse.

how can a program be so much ahead of the other programs ?
to me it seems the other programmers have no clue how rybka works.
and the longer this situation remains the bigger the gap between them
develos.

They may have a clue, just not and International Master Clue ...

[George Tsavdaris]

Results as of this morning, they are still being updated:

Rybka 3 vs. Rybka 2.32a mp on octal, +151=83-22 for +192 Elo.

Rybka 3 vs. Rybka 2.32a mp on quad, 40/1' repeating: +47=24-7 for +196 Elo (!!).

Rybka 3 vs. Zappa Mexico II on quad, 40/1' repeating: +85=46-11 for +200 Elo.

Rybka 3 vs. Hiarcs12 on quad, 40/1' repeating: +72=23-5 for +282 Elo.

Rybka 3 vs. Naum 3.1 on octal, 40/1' repeating: +76=22-2 for +330 Elo (!!).

Rybka 3 vs. Deep Shredder 11 on quad, 40/1' repeating:+61=21-6 for +254 Elo. Will run overnight.

These results are amazing! Actually there can't be words to describe how stunning there are.
I could never imagined that the dominant Rybka 2.3.2a could lose +151=83-22 !!

And the more amazing thing is that these were very fast blitz games(OK on octal super fast machine) and Larry believes that with the main strength Rybka 3 has(much much much better evaluation) on longer time controls the difference would be bigger!

Actually i think results on CEGT and CCRL lists would not be so big, since these lists don't play with other Rybka(=strong opponents) so practically the games against the other weaker opponents would most probably give a lower ELO performance for Rybka. But perhaps i'm wrong....

[mclane]

They may have a clue, just not and International Master Clue ...

i do not believe that an IM or a GM is so much different than any other strong chess player.

i see no GAP between the methods or knowledge of strong chess players
and IM or GMs.

some programmers are strong chess players. whittington, vincent diepeven and other strong chess players come to my mind.
stefan meyer-kahlen and mark uniacke also play chess.

i doubt the gap between rybka and lets say hiarcs or shredder has to do with the fact that vas is an IM. IMO there is a conceptual difference between
rybka (no matter which version you choose, 2.3.2 or 3)
and the other chess programs. this concept difference is worth 100 ELO
and in the new version even 200-250 because latest version of rybka seems to be tuned very well while 2.3.2 was rough tuned.

so far nobody talked about this concept difference.

we were told rybka is a plein dump brute-force engine with big tables.
but if that is so easy, why is nobody trying to copy this method ?
so i have my doubts that this analysis (done very early in the rybka-discussion) is true.

[George Tsavdaris]

a new version of Rybka which is 200 points stronger I'm not so sure about that anymore.

best regards
Roman

Not 200, only 100. I leave the rest to you...

But the results so far indicate it would be about +200 ELO better than its predecessor:

Rybka 3 vs. Rybka 2.32a mp on octal, +151=83-22 for +192 Elo.
Rybka 3 vs. Rybka 2.32a mp on quad, 40/1' repeating: +47=24-7 for +196 Elo.

Why you believe the opposite?

[mclane]

its IMO not much difference if it is 100, 200 or 7000 elo difference.

the problem is that on such a high level they were able to make another big progress. this shows that rybka 2.3.2 and the engines before were filled with bugs and misevaluations and wrong behaviour.

we have found out about those rough behaviour of rybka 2.3.2 before.
we learned that the program was still very shallow done.

now they made it more professional and were able not to make a jump about 30 or 50 elo like we normally would have expected.

but 100 or 7000 elo (whatever you choose to believe).

this is the thing that astonishes.

IMO all would have expected a normal jump.

this shows more about rybka itself than anything else.


here is my theory about it:

IMO rybka is not understood.

the magic in rybka is still not known.

and IMO the key is how the material evaluation is done.

as far as i think normal chess programs only add all white material, subtract the black material from this and get a number....

lets say (to make it easy)

1+1+1+1+1+1+1+1+5+3+3+9+12+3+3+5-1-1-1-1-1-1-1-1-5-3-3-9-12-3-3-5=0

so 0 says material is even.

if e.g. white has one bishop less and black has 1 knight less it is still 0.

and other combinations of material still give only a number.

this is IMO a very ROUGH kind of evaluation.

in the same way you can evaluate girls with giving a number between 1-10.

so your wife is 3 and my girl friend is 6.

anybody sees that reducing a human beeing to a number (3 or 6) is nonsense.

but when doing a chess program all programs work this way!
and rybka not. thats the difference. rybka tries to find out WHICH pieces are on board.
it finds out WHICH relation of pieces fight against which black group of pieces and how THIS is to be evaluated.

this seems to be a quantum difference.

like it is different to meet a person for 3 months and evaluate it THEN instead of calling somebody
3 or 6.

this is IMO the conceptual difference between rybka and the others.
this concept is maybe good for 100+ ELO.

depending in which surrounding program you implement the idea.

normal chess programs do not really talk about material.
they count it and thats it.
they don't really care which pieces are on board. they give a bonus for various things, and they can maybe give bonus/malus within a range of 1/10 up to 3 or 5 pawns.
but there is no big difference in between them at all.

they have different searches.
and different ways to use knowledge (in the tree, preprocessing before the tree). but they do not count the material different.

[mclane]

a new version of Rybka which is 200 points stronger I'm not so sure about that anymore.

best regards
Roman

Not 200, only 100. I leave the rest to you...

But the results so far indicate it would be about +200 ELO better than its predecessor:

Rybka 3 vs. Rybka 2.32a mp on octal, +151=83-22 for +192 Elo.
Rybka 3 vs. Rybka 2.32a mp on quad, 40/1' repeating: +47=24-7 for +196 Elo.

Why you believe the opposite?

maybe he believes it is marketing.

[pedrox]

I do not agree with Larry, games with this time managent can be used to estimate the power of an engine, but not the difference in ELO with another, I think that in games longer the difference will be lower, perhaps half, which is not nothing bad if finally the engine is 100 points better than the previous one.

[mclane]

yes. but this does not help us in the discussion IMO. its unimportant if the difference measured is 100 or 300.
the major problem is that the difference is not 20-30 or even 50 but 100.

[Ovyron]

Agreed, I've always been against games at such time controls and think that the results are very random compared with longer time controls. Also, note that the difference to Zappa Mexico isn't as amazing.

Still even in the case that it's only 70 elo stronger in the end, that's quite an achievement, so it's hard to critique their testing methods.