Chess engines, touraments, and Rybka and Junior

[parrish]

Why not avoid all this silly arguing and just base the strength of the tournaments the way human tournaments are categorized, based upon the ratings of the participants?

[dj]

A new or revised engine will not necessarily have a rating. Engines change much more quickly than human players.

[swami]

Hi Derek,

It is obviously fallacious to suggest that 2007 was stronger because Jonny, diep and isichess perfomed worse than in 2006. The point is that there was only one weak engine in 2007 in a round robin. In 2006 there were 18 competitors, of whom 5 (Urugano, Lion, Chaturanga, FIBchess and ETABETA) were very weak, mustering only 10 points between them (mostly games against each other!) It is illegitimate to compare the results of an 11 round all play all tournament with an 11 round Swiss tournament with 18 players. Jonny and isichess both played three games each against those 5 weak engines, hence gaining 3 easy points in 2006 and boosting their overall scores. (Diep gained 2.5/3.) In addition the fact that it was a Swiss meant that in 2006 Jonny and isichess did not play all the top engines (boosting their overall scores) , but were forced to do so in 2007 (reducing their scores).

That was what I was going to say,but you have made a point quite well in this regard.

The 2007 tournament may or may not have been stronger than 2006 but not on the basis of your fallacious arguments.

It seems 2007 is not any stronger than 2006 because apart from few weaker engines there were few quite stronger engines that were missing in 2007 and they were able to gain some points against top finishers.

Deep Junior can be taken as comparison to this year's Loop but still it seems Deep Junior is preferable as a strong opponent over Loop.

Spike and Grid Chess are in almost same strength.I don't know much about GridChess.

Ikarus did perform certainly better than this year's The King.

Deep Sjeng and last year's Jonny are in almost same strength,deep sjeng scored just 1 more point over jonny this year.

Crafty,Delfi and Chiron and ParSos are quite other strong engines, and they can't be made into comparison with remaining participants in 2007.

Rybka's scores maybe better than last year's Deep Junior but this is certainly not a sensible way to make comparison,especially if you take other sports results to make comparison in similar way.

[Uri Blass]

Another point is that if you look at result of WCCC 2007 then you can see that
Rybka won with 10/11 when Junior only won with 9/11 in 2006

Participants are different,Uri.It's possible that WCCC 2006 had stronger participants than WCCC 2007.It doesn't make sense to compare scores of 2 engines in 2 different tournaments.

some players pariticipated in both tournaments
results of them in 2006 were

shredder 9
Zappa 7.5
Jonny 6.5
diep 6.5
isichess 5.5

total result 35 points

result of them in 2007 were:

Zappa 9
Shredder 7
Jonny 5
diep 4.5
isichess 3.5

total results 29 points

You can see that they scored worse in 2007 so the tournament in 2007 was stronger.

Uri

It is obviously fallacious to suggest that 2007 was stronger because Jonny, diep and isichess perfomed worse than in 2006. The point is that there was only one weak engine in 2007 in a round robin. In 2006 there were 18 competitors, of whom 5 (Urugano, Lion, Chaturanga, FIBchess and ETABETA) were very weak, mustering only 10 points between them (mostly games against each other!) It is illegitimate to compare the results of an 11 round all play all tournament with an 11 round Swiss tournament with 18 players. Jonny and isichess both played three games each against those 5 weak engines, hence gaining 3 easy points in 2006 and boosting their overall scores. (Diep gained 2.5/3.) In addition the fact that it was a Swiss meant that in 2006 Jonny and isichess did not play all the top engines (boosting their overall scores) , but were forced to do so in 2007 (reducing their scores). The 2007 tournament may or may not have been stronger than 2006 but not on the basis of your fallacious arguments.

I agree that it is more fair to compare the opponents of Junior with the opponent of rybka so here is the list of opponents.

Junior 2006:

1)Rybka
2)Spike
3)Crafty
4)Ikarus
5)Delfi
6)Uragano3D(too weak to count)

Rybka 2007:
1)Loop
2)GridChess
3)Deep Sjeng
4)The Baron
5)The king
6)Micromax(too weak to count)

Rybka is better than loop but
I think that when you compare the rest of the opponents there is some superioriry for opponents of 2007

1)Gridchess is better than toga thanks to hardware and toga has similiar level to spike.
2)Deep Sjeng is a commercial program that is better than crafty
3)I do not know if the baron is stronger or weaker than ikarus
4)The king is slightly better than delfi.

from the ccrl list
16 CM10th Xperience 2762 +15 −15 42.0% +54.4 38.1% 1513
19 Delfi 5.1 2751 +35 −35 52.9% −22.7 38.9% 257

Uri

[swami]

Rybka is better than loop but
I think that when you compare the rest of the opponents there is some superioriry for opponents of 2007

1)Gridchess is better than toga thanks to hardware and toga has similiar level to spike.

You underestimate spike,Spike can get a draw against Gridchess normally.

You can't compare last year spike and this year GridChess. (or any last year engine to this year's,Also Derek pointed out that engine changes so much quickly that the rating difference can be hardly determined.)

This year Rybka version is certainly better than last year's.
We don't know about the Junior updates.

2)Deep Sjeng is a commercial program that is better than crafty

Crafty did better than Deep sjeng in WCCC 2004 but Deep sjeng did better in WCCC 2005.They can be equalised but at best I give you a point for Deep sjeng.

)I do not know if the baron is stronger or weaker than ikarus

Ikarus is significantly stronger than Baron,IMO.

4)The king is slightly better than delfi.

from the ccrl list
16 CM10th Xperience 2762 +15 −15 42.0% +54.4 38.1% 1513
19 Delfi 5.1 2751 +35 −35 52.9% −22.7 38.9% 257

Don't agree with you on this one,Delfi is better than the King(default),The King scored 2.5 points,Delfi and Isichess had same number of points in WCCC 2006,and Delfi actually defeated Isichess last year,
Isichess also played in 2007 and has 1 point more than the King.

Instead of pointing out the rating list of Delfi and [Personality]King,you could have used the same logic you displayed in this thread about comparison of opponents of Rybka and [last year's]Junior and that would have ended with this conclusion that Delfi is better.

[Uri Blass]

Rybka is better than loop but
I think that when you compare the rest of the opponents there is some superioriry for opponents of 2007

1)Gridchess is better than toga thanks to hardware and toga has similiar level to spike.

You underestimate spike,Spike can get a draw against Gridchess normally.

You can't compare last year spike and this year GridChess. (or any last year engine to this year's,Also Derek pointed out that engine changes so much quickly that the rating difference can be hardly determined.)

This year Rybka version is certainly better than last year's.
We don't know about the Junior updates.

2)Deep Sjeng is a commercial program that is better than crafty

Crafty did better than Deep sjeng in WCCC 2004 but Deep sjeng did better in WCCC 2005.They can be equalised but at best I give you a point for Deep sjeng.

)I do not know if the baron is stronger or weaker than ikarus

Ikarus is significantly stronger than Baron,IMO.

4)The king is slightly better than delfi.

from the ccrl list
16 CM10th Xperience 2762 +15 −15 42.0% +54.4 38.1% 1513
19 Delfi 5.1 2751 +35 −35 52.9% −22.7 38.9% 257

Don't agree with you on this one,Delfi is better than the King(default),The King scored 2.5 points,Delfi and Isichess had same number of points in WCCC 2006,and Delfi actually defeated Isichess last year,
Isichess also played in 2007 and has 1 point more than the King.

Instead of pointing out the rating list of Delfi and [Personality]King,you could have used the same logic you displayed in this thread about comparison of opponents of Rybka and [last year's]Junior and that would have ended with this conclusion that Delfi is better.

1)We have no reason to believe that spike on single processor is significantly stronger than toga so if we talk about software the programs seem to be equal.
gridchess had hardware advantage that other programs had not.

2)We do not know if Ikarus is significantly stronger than Baron
Ikarus is a private version and I assume Ikarus could be commercial in case of being significantly stronger than the baron so everyone could see it.

3)The king used multi-processor version when as far as I know delfi does not support more than one processor.
I guess that the bad result of the king is mainly because it was unlucky in the tournament.

Uri

[swami]

1)We have no reason to believe that spike on single processor is significantly stronger than toga so if we talk about software the programs seem to be equal.
gridchess had hardware advantage that other programs had not.

Spike that played last year was updated(>1.3) and was not 1.2 that is freely available and it was obviously better than Toga

2)We do not know if Ikarus is significantly stronger than Baron
Ikarus is a private version and I assume Ikarus could be commercial in case of being significantly stronger than the baron so everyone could see it.

Diep is also a private engine and it is in most cases better than The Baron,by that logic you should have argued that Diep could have been commercial in order to be better than most freewares else you wouldn't know that it is better,and going by scores of previous WCCC's Ikarus seems to be almost equal to(or slightly better than?) Diep as it had won speed chess WCCC IIRC.

3)The king used multi-processor version when as far as I know delfi does not support more than one processor.
I guess that the bad result of the king is mainly because it was unlucky in the tournament.

Maybe the King was not updated and it was still Chessmaster 10000 when it played in WCCC 2007?In any case it is clear that Delfi is no worser than The King when its scores indicate that it is clearly better.

By the way,I'm not arguing for sake of winning an argument but I just couldn't resist correcting or responding to the obvious bias you have towards the engines that played in WCCC 2006.