CEGT 40/20 update (Juli 15th 2007)

[Dariusz Orzechowski]

1)This is not correct that rybka is 250 elo stronger than fruit

From the CEGT 120/40 list:
1 Rybka 2.3.2a 64 2CPU : 3026 30 29 344 73.3 % 2851 18 Fruit 2.2 : 2777 18 18 850 42.1 % 2833 40.5 %

You can see that rybka used 2 cpu so this comparison is not fair.
Note also that rybka used 64 bits when fruit used 32 bits.

As far as I know, Fruit does not support multi-CPU and does not have a 64-bit version. So, the difference between the strongest Fruit and the strongest Rybka version is now about 250 points, it is perfectly valid statement.

[George Tsavdaris]

As far as I know, Fruit does not support multi-CPU and does not have a 64-bit version. So, the difference between the strongest Fruit and the strongest Rybka version is now about 250 points, it is perfectly valid statement.

Well even this is not a perfect statement because of ELO error bars and because of testing method of CCRL (they don't play with engine's own books, who knows what will happen if they played) and because Rybka supports 8 CPU(so it may be even stronger on 8 CPU).....

And also the latest(not yet official) CEGT list has Rybka 2.3.2a 64bit 2 CPUs, 266 points above Fruit 2.2 1 CPU:

Code: Select all

1 Rybka 2.3.2a 64 2CPU           : 3045   21  21   750    77.5 %   2831   34.7 %
2 Rybka 2.3 64 2CPU              : 2996   20  20   750    72.1 %   2831   38.1 %
3 Rybka 2.1c 64 2CPU             : 2994   21  21   750    71.9 %   2831   35.9 %
4 Rybka 1.2f 64-bit              : 2959   18  18   950    70.3 %   2809   36.8 %
5 Zap!Chess Zanzibar 64 2CPU     : 2930   17  17   900    62.6 %   2841   46.2 %
6 Deep Fritz 10 2CPU             : 2884   17  17   950    56.3 %   2840   38.8 %
7 Deep Shredder 10 x64 2CPU      : 2855   17  17  1000    50.1 %   2854   40.5 %
8 Naum 2.1 x64 2CPU              : 2849   16  16  1000    49.9 %   2850   47.6 %
9 HIARCS 11 2CPU                 : 2845   17  17  1000    49.0 %   2852   39.3 %
10 List 11.64 2CPU                : 2844   18  18   850    46.8 %   2866   40.4 %
11 Loop 13.5 2CPU                 : 2837   18  18   850    46.4 %   2863   41.4 %
12 Zap!Chess Paderborn 64 2CPU    : 2837   18  18   850    48.4 %   2848   41.2 %
13 Spike 1.2 Turin 2CPU           : 2829   16  16  1000    46.4 %   2854   42.0 %
14 Deep Junior 10 2CPU            : 2829   18  18   950    47.3 %   2847   34.0 %
15 Fritz 10                       : 2803   19  19   850    45.2 %   2837   33.4 %
16 Toga II 1.2.1                  : 2800   16  16  1100    43.6 %   2845   41.5 %
17 Hiarcs 10                      : 2787   17  17   950    41.5 %   2847   38.6 %
18 Fruit 2.2                      : 2779   18  18   900    40.3 %   2848   39.2 %
19 Spike 1.2 Turin                : 2779   17  17  1000    40.8 %   2843   37.7 %
20 Hiarcs X50 UCI                 : 2764   19  19   750    42.4 %   2817   40.0 %
21 Fritz 9                        : 2762   21  21   650    44.5 %   2800   37.8 %
22 Glaurung 1.2 2CPU              : 2749   17  18   950    39.2 %   2825   38.6 %
23 Ktulu 8                        : 2738   19  20   800    39.2 %   2814   35.8 %
24 Chess Tiger 2007               : 2712   20  20   700    34.8 %   2821   39.0 %

http://husvankempen.de/nunn/phpBB2/viewtopic.php?t=784

[George Tsavdaris]

So it does seem that 2.3.2a x64 is quite a bit stronger than Rybka 2.3 The strongest engine just got even stronger ... hard to believe. I wonder how much stronger Rybka can get. Obviously Rybka still has some weaknesses and can yet be improved, but really we must be reaching some sort of wall here.

I don't like the word stronger in this cases. I prefer the expression "performing better".
And Rybka's latest version indeed has reached an amazing performance against the other engines as the ELO rating measures.

But it has still many many weaknesses in finding some tactics and especially in King attacks. Also in the endgame needs much improvement.

If these will improve a lot it will definitely improve a lot the analysis level one can reach of Rybka. But i'm not sure it will improve a lot the performance on eng-eng matches, that means i'm not sure it will have a lot higher ELO and that is because of the competition.
Since Rybka is a great amount of ELO ahead of the competition it should raise the scores even more from the amazing +80% it is now and with the draws as a natural element of Chess, i think it can't improve the performance much higher if the other engines don't improve their ELO much.....

[Uri Blass]

1)This is not correct that rybka is 250 elo stronger than fruit

From the CEGT 120/40 list:
1 Rybka 2.3.2a 64 2CPU : 3026 30 29 344 73.3 % 2851 18 Fruit 2.2 : 2777 18 18 850 42.1 % 2833 40.5 %

You can see that rybka used 2 cpu so this comparison is not fair.
Note also that rybka used 64 bits when fruit used 32 bits.

As far as I know, Fruit does not support multi-CPU and does not have a 64-bit version. So, the difference between the strongest Fruit and the strongest Rybka version is now about 250 points, it is perfectly valid statement.

The fact that fruit does not support multi-cpu is not relevant

I responded to the following words:
"I remember when Fruit was the big thing and at that time I was wondering if it was possible to improve on that."

It was obvious that the poster did not mean to 1 cpu to 2 cpu improvement and did not mean to 32 bit to 64 bit improvement.

Uri

[Dariusz Orzechowski]

The fact that fruit does not support multi-cpu is not relevant

I responded to the following words:
"I remember when Fruit was the big thing and at that time I was wondering if it was possible to improve on that."

It was obvious that the poster did not mean to 1 cpu to 2 cpu improvement and did not mean to 32 bit to 64 bit improvement.

For me, adding a parallel search is an obvious improvement of an engine. For 64-bit version it's not that straightforward but still in majority of cases it is measurable improvement and Fruit has not improved in any of these areas.

[ArmyBridge]

HI Werner, when I try to download in replay zone Chess Tiger 2007 vs Hiarcs X50 this leads me to Rybka vs Hiarcs 11 , by the way tanks for yor great job

[Heinz Van Kempen]

Hi Armando ,

thanks for the hint. I just corrected the link.

Best Regards
Heinz

[Dirt]

For me, adding a parallel search is an obvious improvement of an engine. For 64-bit version it's not that straightforward but still in majority of cases it is measurable improvement and Fruit has not improved in any of these areas.

Creating a 64-bit version is usually quite easy. Fruit just doesn't benefit significantly from it.

[Marc Lacrosse]

Hi all ,


40/20
With the permission of Ryan Benitez we test our 2nd private (beta) engine: Fruit 070327-SF1agg (search focus 1 and playing style aggressiv). The second setting provided by Marc Lacrosse was only tested by me.

Werner Schüle
CEGT-Team

Hi Werner

What is exactly this "second setting provided by Marc Lacrosse" ?

I did a lot of private tests here with this 070327 Fruit version and different sets of parameters (quite a dozen so far)...
... (and as far as I know I did not send anything regarding this to you ?).

I am the one who showed Bryan some evidence that "Search Focus ON" and "Focus depth = 1" is superior to the default config (FD = 2).

In my tests "Playing style = agressive" is not clearly better than default ("Normal").

So far the best setting in my tests is :

Code: Select all

Use Search Focus = true
Focus Depth = 1
Pruning = Aggressive
PPext = true
Use Aspiration search = false
Play Style = Normal
Pawn Shielding Hard = 200
other parameters : default value

Is this what you test under my name ?
If not please do change to these values.

I just sent this set of parameters privately to one person (who is not Ryan nor Fabien) and it was not intended to be made publicly available ...

Here are my results with it (single processor 32 bits versions at fast blitz):

Code: Select all

               Elo    +    - 
Rybka 2.3.2    3028   20   19
Hiarcs 11.1    2932   19   18
Rybka-1.0b     2916   32   31
Loop 13.6      2912   18   18
Fruit-ML       2910   18   18
Toga-1.3x4     2891   32   32
Fruit-default  2868   18   18
Spike-1.2      2808   31   32
Shredder 10    2804   18   19
Naum-2.0       2792   31   31
Alaric 7.06.17 2774   19   19
Alaric 7.04    2730   19   19

Regards

Marc

[Tony Thomas]

1)This is not correct that rybka is 250 elo stronger than fruit

From the CEGT 120/40 list:
1 Rybka 2.3.2a 64 2CPU : 3026 30 29 344 73.3 % 2851 18 Fruit 2.2 : 2777 18 18 850 42.1 % 2833 40.5 %

You can see that rybka used 2 cpu so this comparison is not fair.
Note also that rybka used 64 bits when fruit used 32 bits.

As far as I know, Fruit does not support multi-CPU and does not have a 64-bit version. So, the difference between the strongest Fruit and the strongest Rybka version is now about 250 points, it is perfectly valid statement.

It is a valid statement, but still an unfair one. If you want to compare them fairly, you should find the difference between the 32bit, single cpu versions of both engines.