Zappa Mexico II Strength

[playjunior]

Has anyone conducted tests of the new engine?
I am especially interested in tests with:
Zappa (own book) vs Rybka (own book) on a tournament control.
But any other testing results would be interesting.
Please post the conditions of tests with the results.
Thanks!

[Mr. H.]

Here's a little tournament I finished yesterday...

Code: Select all

Conditions
==========

CPU:        AMD Athlon(tm) 64 X2 Dual Core Processor 4800+ @ 2 x 2760 MHz
OS:         Ubuntu 7.10, Gnome 2.20.1, Kernel 2.6.22-14-generic
GUI:        Xboard (UCI-engines via polyglot 1.4), Ponder off
Hash:       1024 MB each
Book:       Toga, Shredder: performance.bin, Zappa Mexico II: zappa_big.zbook, HIARCS 11.2: H11book2.hcs
TB:         Nalimov: 345-men, Shredderbases, egbb: 345-men
Time:       20 minutes per 40 moves repeating

Participants
============

HIARCS 11.2 MP         (native windows binary via WINE)
Zappa Mexico II MP     (native linux binary)
Toga II 1.4 beta 5c MP (native windows binary via WINE)
Shredder 11 SP         (native linux binary) 

Rating list
===========

    Program                            Score     %    Av.Op.  Elo    +   -    Draws

  1 HIARCS 11.2 MP                 :   7.0/ 12  58.3   2786   2844  125 108   66.7 %
  2 Zappa Mexico II                :   6.5/ 12  54.2   2793   2822  135 131   58.3 %
  3 Toga II 1.4 beta5c             :   6.0/ 12  50.0   2800   2800  147 147   50.0 %
  4 Shredder 11 UCI                :   4.5/ 12  37.5   2822   2734  186 195   25.0 %

Crosstable
==========

Computer chess game
mick-desktop, 2008.02.02 - 2008.02.06
                          Score     HIAR Zapp Toga Shre
--------------------------------------------------------
 1: HIARCS 11.2 MP       7,0 / 12   XXXX ==== 0=== 111=
 2: Zappa Mexico II      6,5 / 12   ==== XXXX =10= =110
 3: Toga II 1.4 beta5c   6,0 / 12   1=== =01= XXXX =001
 4: Shredder 11 UCI      4,5 / 12   000= =001 =110 XXXX
--------------------------------------------------------
24 games: +7 =12 -5

[M ANSARI]

I have been doing 2 tests for the last 4 days 24/7 ... one is 16 0 and the other is 5 3. At 16 0 there seems to be not much difference ... but at 5 3 there is a significant difference. Zappa Mexico used to score only 30% against Rybka ... this is now up significantly in 5 3 ... after 400 games Zappa now scores 36.7%.

Hardware used is 4.4 Ghz Quadcore at 16 0 and 3.2 Ghz Quadcore for the 5 3 match.

[playjunior]

Thanks a lot! Are you playing them with own books Ansari?

[M ANSARI]

No, I am using the best book that plays best for each engine. For Zappa it is the Perfect.ctg by Sedat and for the Rybka it is the stats2.ctg from Nick Carlin. Basically Sedats book is the most updated patched perfect.ctg which according to Anthony C is the best for Zappa. As for Rybka it is basically a RybkaII.ctg that has been patched for weak lines from the Chessbase server by Nick.

Results so far are very good for Zappa II with a 5 3 tournament ending score after 400 games at 37.88%. +53 wins -150 losses and =197 draws. I think that is the highest I have seen anyone score against Rybka 2.3.2a. Still Rybka is king though ... but the gap seems to have closed up a bit ... at least when Rybka 2.3.2a is concerned. Remember there is a new Rybka that is under development and Zappa is the latest generation ... still excellent result for Zappa. I don't think any other engine can say that they can win 1 out of 7.5 games played against Rybka or more impressively only lose once every 2.7 games against Rybka.

[Lion]

Hi,

Where can I find stats2.ctg from Nick Carlin ?

best regards

[M ANSARI]

I am afraid that is a private book.

[Tony Thomas]

I do not know about games at tournament time control, but at fast 1min+1sec time control, Zappa lagged behind every single commercial engine. With the new update Zappa is able to get in to the top Ranks. Here is the rating and results of two different versions for comparison. As Ansari pointed out the results diminish with longer time controls.

Code: Select all

15 Zappa Mexico II           : 2742  128 (+ 57,= 29,- 42), 55.9 %

Rybka v1.0 Beta.w32           :   4 (+  1,=  0,-  3), 25.0 %
WildCat 7.0                   :   4 (+  4,=  0,-  0), 100.0 %
TogaII 1.2 beta 2a KS/EHP     :   4 (+  0,=  1,-  3), 12.5 %
Spike 1.2 Turin               :   4 (+  3,=  0,-  1), 75.0 %
Smarthink 1.00                :   4 (+  1,=  1,-  2), 37.5 %
Prodeo 1.2                    :   4 (+  2,=  1,-  1), 62.5 %
Trace 1.37a                   :   4 (+  3,=  1,-  0), 87.5 %
Gandalf 6.01                  :   4 (+  0,=  2,-  2), 25.0 %
Ktulu 8.0                     :   4 (+  1,=  0,-  3), 25.0 %
Thinker 4.7a                  :   4 (+  3,=  0,-  1), 75.0 %
Pharaon 3.5.1                 :   4 (+  2,=  0,-  2), 50.0 %
SOS 5.1                       :   4 (+  4,=  0,-  0), 100.0 %
Ruffian 1.0.5                 :   4 (+  0,=  4,-  0), 50.0 %
SlowChess Blitz WV 2.1        :   4 (+  3,=  1,-  0), 87.5 %
Aristarch 4.50                :   4 (+  2,=  2,-  0), 75.0 %
CM10th D2Alos                 :   4 (+  2,=  2,-  0), 75.0 %
ChessTiger2007.1 UCI          :   4 (+  0,=  1,-  3), 12.5 %
Fruit 2.3                     :   4 (+  1,=  0,-  3), 25.0 %
Naum 2.2                      :   4 (+  1,=  0,-  3), 25.0 %
DeepSjeng27                   :   4 (+  3,=  0,-  1), 75.0 %
Delfi 5.2                     :   4 (+  3,=  1,-  0), 87.5 %
Movei00_8_438                 :   4 (+  1,=  2,-  1), 50.0 %
BugChess2_V1_5_2              :   4 (+  1,=  1,-  2), 37.5 %
Shredder11UCI                 :   4 (+  1,=  1,-  2), 37.5 %
Crafty 21.6 JA                :   4 (+  2,=  2,-  0), 75.0 %
Scorpio 2.0                   :   4 (+  1,=  1,-  2), 37.5 %
AlaricWB707                   :   4 (+  2,=  1,-  1), 62.5 %
Zappa_mexico fix              :   4 (+  3,=  1,-  0), 87.5 %
Glaurung 2.0.1 JA             :   4 (+  3,=  1,-  0), 87.5 %
Hiarcs11.2SPUCI               :   4 (+  1,=  1,-  2), 37.5 %
Bright-0.2c                   :   4 (+  1,=  0,-  3), 25.0 %
Frenzee Dec 07                :   4 (+  2,=  1,-  1), 62.5 %

Code: Select all

27 Zappa_mexico              : 2665  156 (+ 55,= 33,- 68), 45.8 %

Rybka v1.0 Beta.w32           :   4 (+  1,=  1,-  2), 37.5 %
WildCat 7.0                   :   4 (+  1,=  1,-  2), 37.5 %
TogaII 1.2 beta 2a KS/EHP     :   4 (+  1,=  0,-  3), 25.0 %
Spike 1.2 Turin               :   4 (+  1,=  1,-  2), 37.5 %
Smarthink 1.00                :   4 (+  2,=  1,-  1), 62.5 %
Prodeo 1.2                    :   4 (+  2,=  1,-  1), 62.5 %
Trace 1.37a                   :   4 (+  2,=  0,-  2), 50.0 %
Frenzee 3.0                   :   4 (+  2,=  0,-  2), 50.0 %
Gandalf 6.01                  :   4 (+  2,=  2,-  0), 75.0 %
Ktulu 8.0                     :   4 (+  0,=  2,-  2), 25.0 %
Thinker 4.7a                  :   4 (+  3,=  1,-  0), 87.5 %
Pharaon 3.5.1                 :   4 (+  1,=  0,-  3), 25.0 %
SOS 5.1                       :   4 (+  4,=  0,-  0), 100.0 %
Ruffian 1.0.5                 :   4 (+  1,=  2,-  1), 50.0 %
SlowChess Blitz WV 2.1        :   4 (+  3,=  0,-  1), 75.0 %
Aristarch 4.50                :   4 (+  1,=  1,-  2), 37.5 %
Scorpio 1.84 JA               :   4 (+  3,=  0,-  1), 75.0 %
Jonny 2.83                    :   4 (+  3,=  1,-  0), 87.5 %
CM10th D2Alos                 :   4 (+  1,=  2,-  1), 50.0 %
Zappa 1.1                     :   4 (+  2,=  1,-  1), 62.5 %
HiarcsX54UCI                  :   4 (+  0,=  0,-  4),  0.0 %
Shredder10UCI Balmung         :   4 (+  0,=  3,-  1), 37.5 %
List  5.12                    :   4 (+  2,=  0,-  2), 50.0 %
Delfi 5.1                     :   4 (+  0,=  1,-  3), 12.5 %
ChessTiger2007.1 UCI          :   4 (+  1,=  1,-  2), 37.5 %
DeepSjeng25                   :   4 (+  1,=  1,-  2), 37.5 %
Glaurung 2 Epsilon/5          :   4 (+  3,=  0,-  1), 75.0 %
Fruit 2.3                     :   4 (+  1,=  0,-  3), 25.0 %
Naum 2.2                      :   4 (+  2,=  0,-  2), 50.0 %
DeepSjeng27                   :   4 (+  1,=  3,-  0), 62.5 %
Delfi 5.2                     :   4 (+  2,=  1,-  1), 62.5 %
Movei00_8_438                 :   4 (+  0,=  1,-  3), 12.5 %
BugChess2_V1_5_2              :   4 (+  2,=  0,-  2), 50.0 %
TogaII 1.3.1                  :   4 (+  1,=  0,-  3), 25.0 %
Shredder11UCI                 :   4 (+  0,=  1,-  3), 12.5 %
Crafty 21.6 JA                :   4 (+  2,=  0,-  2), 50.0 %
Scorpio 2.0                   :   4 (+  0,=  0,-  4),  0.0 %
AlaricWB707                   :   4 (+  0,=  1,-  3), 12.5 %
Zappa_mexico fix              :   4 (+  1,=  3,-  0), 62.5 %

[M ANSARI]

Yes it does seem that this improvement in blitz is a direct result of Strelka code coming out in the open. I always wondered why Zappa was so poor in fast time controls ... apparently it was due to Anthony not wanting to change some code which would make Zappa search faster but (to him at least) not play better. It seems he has changed his mind and changed it now. The good news for Zappa is that this change has not hurt its long time control strength. So I would confidently say that Zappa Mexico II is the second strongest MP engine out there.

[Uri]

What is the strongest program to date? I thought that Zappa Mexico II and Shredder XP are the two strongest programs to date. Rybka 2.3.2 is not the strongest program because i saw it loosing on 8 processors to Zappa Zanzibar.