Ratinglist based on positional openingpositions

[Yarget]

Hello Mike!

Thanks for your interest and comments regarding my tests.

Compared to my expectations before starting these tests I find the following Top 3 to be the most surprising (positive or negative) so far:

1. Spike turned out to be the weakest engine in the gambittests and more than 100 ratingpoints below the expected (compared to CEGT 40/4).

2. I have always considered Naum to be a solid, positional engine but it turned out to be considerable stronger in the gambits.

3. The same can be said about Rybka. It made (as expected) a strong performance in the positional games but Rybka was even stronger in the gambits. It should also be mentioned that Deep Fritz 10 performed very well in both testsets.

Just like you I wonder why Spike did so bad in the gambitgames. I have tested this fine engine for a long time (when I was testing for the CSS SMP Ratinglist) and I never noticed any tactical weaknesses. You could be right Mike when you suggest that engines like Spike have "problems to evaluate and/or use "dynamical" compensation for material".

I would like to present my testresults in the CSS Forum as well. However my german isn't too good (I have made some posts in this forum but only rather short ones). I would be grateful if you would make a post in the CSS Forum, just a short summary one (or a longer if you wish) and with a link to this thread. Das wäre toll

Best regards
Per

[Oscar L]

Thanks again Per.

Your testing and analysis are very interesting.

Regarding Junior, this "sensible" aspect you have showed is, as you said, the explanation of the better results in the SSDF list, that uses the engine book to test. And perhaps too that this is the only one where Junior 10.1 shows some improvement?

http://ssdf.bosjo.net/


Let's hope that Junior 11 UCI does not become Fruity

http://64.68.157.89/forum/viewtopic.php ... 85&t=19179


Will you test Chess Tiger?

[Yarget]

Hello Oscar!

Indeed, Junior is one of the most "sensible" (and original) engines out there. If playing under the "right" conditions (its own book or for instance my positional testset) it's able to deliver big performances. If however the conditions are changed then Junior might perform 100 ratingpoints worse. In contrast engines like Zap!Zanzibar, Shredder and Fritz "don't care" whether they play wild gambits or closed positional games.

I have decided only to test MP engines leaving Chess Tiger 2007 outside. However if Chess Tiger will appear in a MP version I might test this engine.

Best regards
Per

[Mike S.]

I have forwarded a short info to the CSS forum which will appear there soon, with links to your main postings in this thread.

[Yarget]

Thanks a lot Mike, much appreciated.

[Laskos]

So, basically

1-2 Junior & Spike each 110 ratingpoints! for Positional
3. Naum 54 ratingpoints for Gambit
4. Hiarcs 48 ratingpoints for Gambit
5. Glaurung 40 ratingpoints for Gambit
6. Rybka 28 ratingpoints for Gambit
7. Loop 24 ratingpoints for Gambit
8. Fritz 4 ratingpoints irrelevant
9. Shredder 2 ratingpoints irrelevant
10. Zap 0 ratingpoints irrelevant


This test can be taken seriously, the differences are large. It shows Junior and Spike very sensitive to opening books, and surprise, they like positional games. On the other hand Naum, Hiarcs, Glaurung and Rybka like gambit games.

Kai

[Yarget]

Yes, basicly you are right Kai. However, as I have mentioned earlier in this thread the relative small number of games and opponents mean that firm, decisive conclusions shouldn't be drawn for the time being. I think that my tests give some interesting indications regarding preferred type of positions for several engines, the "sensibility" of these engines etc.

I find the tests quite interesting and I would like to continue and therefore I have started testing Bright 0.2c 2CPU. When the tests are done I'll update the 2 ratinglists. Several places I have read that Bright is tactical very strong and I'm excited to see if my tests will confrim this. The first matches in the Gambitgames are done with these results:

against Deep Junior 10.1 12½-7½
against SpikeMP 1.2 Turin 11-9

Quite promising results although Junior and Spike are performing less well in the gambitgames.

Regards
Per

[mariaclara]

'
Hello Per,

thanks for the games

regards

[Yarget]

You are welcome Clare

By the way, I have made a break in my test of Bright. The reason is that I want to test the new Toga 1.4 MP by Thomas Gaksch as quick as possible. I hope that I in the middle of the next week will be able to present results, updated ratinglists and so on.

Best regards
Per

[maxchgr]

Sorry if I missed it, but were the games posted somewhere?