Ratinglist based on positional openingpositions

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 8:44 pm
Location: Amman,Jordan

Re: Ratinglist based on positional openingpositions

Post by Dr.Wael Deeb »

Yarget wrote:Hello Oscar

Yes, I've heard the rumours about Junior 11 as well. This release should be different as Junior will become an UCI-engine. Before the release I think we will se a Junior 10 UCI release. What should we expect from Junior 11? Well, hopefully a stronger engine but hopefully also an engine which maintains this special way of playing. The world of chessengines will become more boring if Junior 11 turns out to be another strong "mainstream" engine. We'll see.

As mentioned earlier Junior is quite an extreme engine but perhaps the word "sensible" fits better. Sensible in the sense that Junior when playing a position it "likes" is very, very strong and vice versa. Basicly this is true for all engines (and humans as well) but in particular for Junior. Just take a look at the results so far for Deep Junior 10 and compare them with the results for the positional games:

versus Rybka 2.3.2a mp 3-17 (8-12 in POS-games)
versus Hiarcs 11.1 MP 5½-14½ (15-5 in POS-games!!)
versus Deep Fritz 10 6-14 (5½-14½ in POS-games)
versus Deep Shredder 11 8-12 (9-11 in POS-games)
versus Glaurung 2.0.1 currently 4½-8½ (!) (11½-8½ in POS-games)

The "sensibility" of Junior is also expressed in several ratinglists. When (Deep) Junior 10 is using a commin enginebook it has got a playingstrength clearly behind engines like Fritz 10, Zanzibar, Shredder 10 and Hiarcs 11 (check lists at CEGT and CCRL). However if Junior is allowed to play with its own well-tuned book then it's another story as the SSDF ratinglist is showing:

http://ssdf.bosjo.net/list.htm

Only Hiarcs is then in front of Junior and only by few points.

Regards
Per
You're definitely at the right track Per,Junior achieves it's top performance using it's own opening book 8-)
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
Yarget

Re: Ratinglist based on positional openingpositions

Post by Yarget »

Hello everyone!

I have completed almost 50% of the gambitgames and I thought it would be appropriate to present the current stand. First of all let me say that these games compared to the positional games as expected are totally different. I have seen many sharp knock-out games so far and the drawfrequency clearly indicates that: in this moment it is just 24,3% while it was 32,4% for all the positional games!

Here follows the current ratinglist for the Gambitgames (averagerating 2800):

Code: Select all

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Rybka 2.3.2a mp 32-bit         : 2963   61  59   120    75.0 %   2772   23.3 %
  2 HIARCS 11.1 MP UCI             : 2814   53  53   120    52.5 %   2797   28.3 %
  3 Deep Fritz 10                  : 2809   56  56   120    51.7 %   2797   20.0 %
  4 LoopMP 11A.32                  : 2809   53  53   120    51.7 %   2797   28.3 %
  5 Deep Shredder 11 UCI           : 2806   54  54   120    51.2 %   2798   25.8 %
  6 Glaurung 2.0.1                 : 2720   55  56   120    37.1 %   2812   25.8 %
  7 Deep Junior 10.1               : 2679   60  61   120    30.8 %   2819   18.3 %
Earlier in this thread I have been talking about the "sensibility" of Junior and the testgames is indeed no exception from this rule. Right now Junior is last (even behind Glaurung) and it is 114 ELO-points below the result from the positional games! Rybka has really surprised me so far in the gambitgames. At this moment it is 35 ELO-points better than the performance it made in the positional games (and this performance was already a very strong one, around 30 ELO-points more than one would expect).

Spike, Naum and Zap are being tested now and when these tests are done I'll return with more comparisons.

Regards
Per
User avatar
Ovyron
Posts: 4562
Joined: Tue Jul 03, 2007 4:30 am

Re: Ratinglist based on positional openingpositions

Post by Ovyron »

Yarget wrote:Rybka has really surprised me so far in the gambitgames. At this moment it is 35 ELO-points better than the performance it made in the positional games (and this performance was already a very strong one, around 30 ELO-points more than one would expect).
Yes, no matter the playing style, Rybka is a monster when it comes to strength ;)
Your beliefs create your reality, so be careful what you wish for.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Ratinglist based on positional openingpositions

Post by Laskos »

Yarget wrote: Rybka has really surprised me so far in the gambitgames. At this moment it is 35 ELO-points better than the performance it made in the positional games (and this performance was already a very strong one, around 30 ELO-points more than one would expect).

Regards
Per
Then you have a small calibration problem, in your test Rybka is performing generally 40-50 points stronger (comparing to other engines) than in CCRL or CEGT.

Anyway, very interesting test.
Kai
Yarget

Re: Ratinglist based on positional openingpositions

Post by Yarget »

Then you have a small calibration problem, in your test Rybka is performing generally 40-50 points stronger (comparing to other engines) than in CCRL or CEGT.
I'm not quite sure what you mean Kai. If you are talking about Rybka and the 35 ELO-points then this number is the exact difference between the current Gambitratinglist (2963) and the Positionalratinglist (2928). If your comment is referring to the list I posted earlier in this thread:

Code: Select all

1. Deep Junior 10.1 +70,33 ratingpoints 
2. Deep Fritz 10 +49,22 ratingpoints 
3. Rybka 2.3.2a mp +30,33 ratingpoints 
4. Zap!Chess Zanzibar 2CPU +17,00 ratingpoints 
5. SpikeMP 1.2 Turin +8,11 ratingpoints 
6. LoopMP 11A.32 +1,44 ratingpoints 
7. Deep Shredder 11 UCI -18,56 ratingpoints 
8. Naum 2.2 2CPU -36,33 ratingpoints 
9. Hiarcs 11.1 MP -57,44 ratingpoints 
10. Glaurung 2.0.1 2CPU -64,11 ratingpoints


then I have doublechecked my calculations. I don't use CCRL as referencelist, I only use the CEGT 40/4 ratinglist.

Regards
Per
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Ratinglist based on positional openingpositions

Post by Laskos »

Yarget wrote:
Then you have a small calibration problem, in your test Rybka is performing generally 40-50 points stronger (comparing to other engines) than in CCRL or CEGT.
I'm not quite sure what you mean Kai. If you are talking about Rybka and the 35 ELO-points then this number is the exact difference between the current Gambitratinglist (2963) and the Positionalratinglist (2928).
Regards
Per
If I am not wrong, you said that in Positionalratinglist Rybka is performing 30 points better than expected from CEGT rating compared to other engines. Then in Gambitratinglist it is performing 30+35=65 points better than expected. In the ideal situation these numbers P+G should add to 0 and not to 95, if your openings are representative of the book used by CEGT. Maybe the chosen openings suit Rybka well or maybe it is just a statistical fluctuation or calibration error.

Anyway, it doesn't distort the general picture. Very good test indeed.

Kai
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Ratinglist based on positional openingpositions

Post by Laskos »

Yarget wrote:Hello everyone!

I have completed almost 50% of the gambitgames

Regards
Per
Per, did you finish your gambitgames? I would like to see the final results (the difference between Gamb. and Pos. which is not affected by the calibration). It is very interesting to see Rybka more Gamb. and Junior more Pos.

Regards,
Kai
Yarget

Re: Ratinglist based on positional openingpositions

Post by Yarget »

Hello Kai!

Thanks for your interest in my testwork. I have been very busy this weekend and I haven't been able to test a lot. However, the tests continue and I guess that the Gambitgames will be finished Wednesday or Thursday and following that I'll present the Gambit-ratinglist and I'll compare this one with the Positional-ratinglist.

Regards
Per
Tony Thomas

Re: Ratinglist based on positional openingpositions

Post by Tony Thomas »

Per, is there any reason for not using Hiarcs 11.2?
User avatar
Ovyron
Posts: 4562
Joined: Tue Jul 03, 2007 4:30 am

Re: Ratinglist based on positional openingpositions

Post by Ovyron »

Tony Thomas wrote:Per, is there any reason for not using Hiarcs 11.2?
Maybe because the rating lists have revealed that it's weaker than 11.1?
Your beliefs create your reality, so be careful what you wish for.