TalkChess.com

Dr.Wael Deeb wrote:

lkaufman wrote:Do those three programs generally play the same move with the same score on one core at a set depth, or do they differ substantially and frequently at the same depth?

In general they will not play the same move with the same score at the same depth,at least these are my observations roughly as I didn't test them....
Dr.D

Interesting observations you have after not testing them.

lkaufman wrote:Robbo (and I presume also Firebird) is much faster than Rybka 3 but with lower quality, so it will often see a ply deeper but with less positional knowledge and missing some tactical things as well. Therefore in a given amount of time the move choice is likely to be different. Apparently in this particular game the extra ply proved far more relevant than the missing positional stuff, but over the thousands of games in my database the two perform about the same at predicting moves at one minute per move. This also means that in assessing which is stronger, Rybka 3 or a derivative, it is important to compare results against a third unrelated engine (like stockfish for example), as in a direct matchup speed trumps knowledge in general.

Yes, my matches between Robbo and Rybka say as much. Rybka kept a minute lead in the slow TC games, as published, whereas in 5 min games, Robbo displayed a significant edge, especially with ponder on.

That said, this curiosity lead to a boon for PDA owners: Robbo is the absolute king of Pocket PC. I was questioned by a mod about suggesting this as a Pocket PC engine, and simply told him there is no Rybka for the PDA, so no original to opt for. I know Vas had planned on one eventually, but it has been at least 3 years since I have heard this. Whatever the reason, I believe the project has been shelved for the time being. As Robbo's superiority is precisely in fast time controls, it is ideal for a PDA, where the depth would be far shallower than even a netbook. The next best options are roughly 200 Elo weaker.

i'm somewhat surprised that robbo is so powerful on PPC -- i understand rybka as currently written does not translate well, while hiarcs gets a large edge (compared to other programs) for some reason. have you tested robbo vs PF4? could be interesting. at any rate, if robbo scales linearly on a 500mhz+ processor it would be a monster. apparently rybka doesn't, & hiarcs does.

i am basing this on a HIARCS forum posting some time back of PF3 vs rybka on one of the dedicated machines, & hiarcs beating it pretty easily.

Albert Silver wrote:

Dr.Wael Deeb wrote:

lkaufman wrote:Do those three programs generally play the same move with the same score on one core at a set depth, or do they differ substantially and frequently at the same depth?

In general they will not play the same move with the same score at the same depth,at least these are my observations roughly as I didn't test them....
Dr.D

Interesting observations you have after not testing them.

we are talking about FB vs RL here yes? i'd be shocked if there isn't a massive correlation on one core.

lkaufman wrote:Robbo (and I presume also Firebird) is much faster than Rybka 3 but with lower quality, How can it play better than ?
so it will often see a ply deeper but with less positional knowledge and missing some tactical things as well ??? Examples please .

Therefore in a given amount of time the move choice is likely to be different.
In short and long analysis it gives very different and good moves
Apparently in this particular game the extra ply proved far more relevant than the missing positional stuff, but over the thousands of games in my database the two perform about the same at predicting moves at one minute per move. [ I analysed yet two games and both with very different analysis form Rybka and Firebird , it doesn't act as simple Rybka clone]
This also means that in assessing which is stronger, Rybka 3 or a derivative, it is important to compare results against a third unrelated engine (like stockfish for example), as in a direct matchup speed trumps knowledge in general.

[ I agree on that ]
Unfortunately CEGT and CCRL doesn't test. But already there is a lot of testing evidence on shorter time controls , (which btw in earlier days were long games) which prove Robbolito and Firebird are the strongest engines yet.

"Robbo (and I presume also Firebird) is much faster than Rybka 3 but with lower quality, How can it play better than ?
so it will often see a ply deeper but with less positional knowledge and missing some tactical things as well ??? Examples please"

An extra ply is very important in computer chess. If one engine outsearches a similar one by a ply consistently, it will win decisively in a match. If you play Rybka vs. Robbo at any fixed depth, remembering to set Rybka for three plies less than Robbo as Rybka reports depth minus three, Rybka will always win any long match, typcially by somewhere around 55% or so. This is the basis of my statement that the search is of higher quality in Rybka. I believe this is mostly due to less chess knowledge in Robbo, but also to other shortcuts that speed up Robbo but occasionally cause a tactic to remain hidden for an extra ply or so. The extra ply Robbo often gets is clearly enough to allow it to win timed matches at short time controls. I believe there exists some time control at which Rybka would start to win, because the value of an extra ply diminishes with depth while the value of chess knowledge does not, but what this time control is I do not know. I do know that at about one minute per position the two programs become equal at matching moves from my huge database, so for my own analysis I would use Robbo (or a MP equivalent) if I want a quick answer but I would use Rybka (actually Rybka human as I consider it better for analyzing human games) if I intend to let the machine think for at least a minute.

Albert Silver wrote:

Dr.Wael Deeb wrote:

lkaufman wrote:Do those three programs generally play the same move with the same score on one core at a set depth, or do they differ substantially and frequently at the same depth?

In general they will not play the same move with the same score at the same depth,at least these are my observations roughly as I didn't test them....
Dr.D

Interesting observations you have after not testing them.

I am an interesting personality....but you should know that by now....
Dr.D

Dr.Wael Deeb wrote:

Albert Silver wrote:

Dr.Wael Deeb wrote:

lkaufman wrote:Do those three programs generally play the same move with the same score on one core at a set depth, or do they differ substantially and frequently at the same depth?

In general they will not play the same move with the same score at the same depth,at least these are my observations roughly as I didn't test them....
Dr.D

Interesting observations you have after not testing them.

I am an interesting personality....but you should know that by now....
Dr.D

Yes, nevertheless, making comparative analysis without testing is pretty odd.

lkaufman wrote:"Robbo (and I presume also Firebird) is much faster than Rybka 3 but with lower quality, How can it play better than ?
so it will often see a ply deeper but with less positional knowledge and missing some tactical things as well ??? Examples please"

An extra ply is very important in computer chess. If one engine outsearches a similar one by a ply consistently, it will win decisively in a match. If you play Rybka vs. Robbo at any fixed depth, remembering to set Rybka for three plies less than Robbo as Rybka reports depth minus three, Rybka will always win any long match, typcially by somewhere around 55% or so. This is the basis of my statement that the search is of higher quality in Rybka. I believe this is mostly due to less chess knowledge in Robbo, but also to other shortcuts that speed up Robbo but occasionally cause a tactic to remain hidden for an extra ply or so. The extra ply Robbo often gets is clearly enough to allow it to win timed matches at short time controls. I believe there exists some time control at which Rybka would start to win, because the value of an extra ply diminishes with depth while the value of chess knowledge does not, but what this time control is I do not know. I do know that at about one minute per position the two programs become equal at matching moves from my huge database, so for my own analysis I would use Robbo (or a MP equivalent) if I want a quick answer but I would use Rybka (actually Rybka human as I consider it better for analyzing human games) if I intend to let the machine think for at least a minute.

Larry,

So if I run Rybka vs Robbo at fixed depth and I set Rybka to 3 less plies than Robbo - Rybka will win approx 55% of the time?

What time control should I use and how many plies to get this result?

Did you try this to get the 55% score and what time control/plies or other settings did you use.

I would like to replicate this result.

Albert Silver wrote:

lkaufman wrote:Robbo (and I presume also Firebird) is much faster than Rybka 3 but with lower quality, so it will often see a ply deeper but with less positional knowledge and missing some tactical things as well. Therefore in a given amount of time the move choice is likely to be different. Apparently in this particular game the extra ply proved far more relevant than the missing positional stuff, but over the thousands of games in my database the two perform about the same at predicting moves at one minute per move. This also means that in assessing which is stronger, Rybka 3 or a derivative, it is important to compare results against a third unrelated engine (like stockfish for example), as in a direct matchup speed trumps knowledge in general.

Yes, my matches between Robbo and Rybka say as much. Rybka kept a minute lead in the slow TC games, as published, whereas in 5 min games, Robbo displayed a significant edge, especially with ponder on.

That said, this curiosity lead to a boon for PDA owners: Robbo is the absolute king of Pocket PC. I was questioned by a mod about suggesting this as a Pocket PC engine, and simply told him there is no Rybka for the PDA, so no original to opt for. I know Vas had planned on one eventually, but it has been at least 3 years since I have heard this. Whatever the reason, I believe the project has been shelved for the time being. As Robbo's superiority is precisely in fast time controls, it is ideal for a PDA, where the depth would be far shallower than even a netbook. The next best options are roughly 200 Elo weaker.

I think if the clones were only competing with Rybka here it wouldn't really matter. Only Vas would have any reason to complain, but as it is he is not complaining and not competing. If he wanted to compete he only needs to make a Rybka that is stronger on the PDA if it isn't already, having the clones as an example. Nobody cares if this hurts Rybka, well I should just speak for myself but it really doesn't as far as I can guess, at least not yet. Will it hurt Rybka 4 I do not know, maybe, depending on how good Rybka 4 is.

It hurts other commercial software now more, as long as they haven't caught up. That will take a while and then Rybka's advantage will be less and Vas will have more reason to worry about his sales. But the advantage would have grown less anyway, it just will be faster now. Thanks to the clones, yes. I think any 150 elo advantage as it was at some time -a rough guess- for the market leader will disappear and not come back, but that is just by looking in the crystal ball.

Eelco