In Answer To Why Zappa Is So Far Down In CEGT

[geots]

In fact i dont know that it is- havent looked. Im taking this guy at his word. But the following was my answer to him in the Rybka Forum:


Why come its way down the table? News bulletin: For some reason people seem to think that when they buy the commercial program, the only difference between it and the one being used in the tournament is the hardware it is run on. Couldnt be further from the truth. Look at Junior winning the WCCC. Who really thinks the program Amir released was identical to the program that won the tournament, and it was just a matter of hardware difference, books and book cookers. Better wake up and smell the coffee. PS: But lets just assume for a moment for arguments sake, that all the versions sold were the same as those in the tournaments and matches. What will it play in a tourn. or match- 10 games at the most, often less. CETG and CCRL results will end up being many many games more than that. After all, it is a slam dunk that you cant tell much of anything between two programs after only 6 or 10 games. Add to that the fact that CETG and CCRL results will also reflect how the engines both did against outside competition also. Add all that up, and all this 10 game match proves is who will leave with the money in his pocket. They play a rematch here- you going to bet your paycheck on Zappa. Not me!

Regards,

George Speight/ Member CCRL Testing Group

[Dann Corbit]

The testing conditions are different. We also have very little data for Zappa Mexico.

Zappa is the second strongest engine in CEGT at 40/120 with 2 CPUs after Rybka:

Code: Select all

no Program Elo + - Games Score Av.Op. Draws 
1 Rybka 2.3.2a 64 2CPU 3050 18 18 1000 77.2 % 2837 36.3 % 
2 Rybka 2.3 64 2CPU 3000 18 18 933 71.7 % 2839 39.4 % 
3 Rybka 2.1c 64 2CPU 2999 18 18 969 71.6 % 2838 37.0 % 
4 Rybka 1.2f 64-bit 2969 17 17 1100 70.4 % 2818 37.3 % 
5 Zap!Chess Zanzibar 64 2CPU 2936 15 15 1143 62.9 % 2845 46.1 % 
6 Naum 2.2 x64 2CPU 2902 15 15 1066 56.7 % 2855 48.1 % 
7 Deep Fritz 10 2CPU 2884 16 16 1184 55.6 % 2845 38.7 % 
8 HIARCS 11.2 2CPU 2862 21 21 634 45.7 % 2892 41.5 % 
9 Deep Shredder 10 x64 2CPU 2855 15 15 1208 49.6 % 2858 40.8 % 
10 Naum 2.1 x64 2CPU 2854 15 15 1100 50.6 % 2849 47.1 % 

CCRL 40/40 also has Zappa second:

Code: Select all

Rank Name Rating Score Average Opponent Draws Games LOS 
1 Rybka 2.3.2a 64-bit 4CPU 3117 +27 -26 74.7% -167.3 37.0% 514   68.2% 
  Rybka 2.2 64-bit 4CPU 3107 +30 -29 73.2% -156.8 38.7% 403 65.7% 
  Rybka 2.3.2a 64-bit 2CPU 3098 +26 -25 78.0% -192.6 35.6% 571 52.5% 
  Zappa Mexico 64-bit 4CPU 3095 +82 -81 53.6% -23.0 50.0% 42 57.3% 
  Rybka 2.1 64-bit 4CPU 3087 +43 -41 74.4% -174.1 34.9% 209 54.4% 
  Rybka 2.1 32-bit 4CPU 3083 +47 -45 77.5% -192.7 30.8% 182 60.7% 
  Rybka 2.1 64-bit 2CPU 3075 +26 -25 73.4% -166.7 34.3% 571 57.5% 
  Rybka 2.3.2a 64-bit 3072 +21 -20 76.5% -191.0 31.8% 925 86.5% 
  Rybka 2.2 64-bit 2CPU 3052 +23 -23 73.0% -163.4 35.0% 697 54.2% 
2 Zap!Chess Zanzibar 64-bit 4CPU 3051 +22 -21 64.6% -91.2 46.4% 679 83.9% 
  Zap!Chess Zanzibar 64-bit 2CPU 3027 +39 -38 58.3% -57.4 45.4% 205 62.2% 
  Rybka 2.2 32-bit 2CPU 3018 +29 -29 69.9% -135.2 39.1% 399 57.1% 
  Rybka 2.3.2a 32-bit 3015 +23 -23 75.1% -171.9 36.5% 693 67.1% 
3 Naum 2.2 64-bit 4CPU 3007 +24 -24 56.6% -40.5 49.5% 525 58.5% 
  Rybka 2.1 32-bit 2CPU 3003 +26 -26 70.8% -141.9 41.1% 508 56.6% 
  Rybka 2.2 64-bit 2999 +35 -34 70.0% -128.6 47.9% 267 61.5% 
  Rybka 1.2 64-bit 2993 +21 -21 74.2% -171.1 33.6% 881 65.7% 
  Rybka 2.2 32-bit 2987 +18 -17 70.4% -139.6 36.6% 1150 54.9% 
  Rybka 1.1 64-bit 2986 +24 -24 76.1% -185.6 34.6% 665 52.3% 
4 Hiarcs 11.1 4CPU 2985 +22 -21 54.9% -32.6 42.8% 677 70.2% 
  Rybka 1.2 32-bit 2976 +18 -17 69.8% -141.8 35.3% 1174 52.3% 
  Hiarcs 11.2 4CPU 2975 +52 -53 44.0% +38.6 45.9% 109 51.2% 
  Naum 2.2 64-bit 2CPU 2974 +32 -31 62.7% -78.8 49.0% 304 58.0% 
  Rybka 2.1 32-bit 2970 +23 -23 69.5% -130.9 40.3% 645 54.1% 
  Hiarcs 11 4CPU 2967 +32 -32 53.8% -26.4 39.1% 312 50.4% 
  Naum 2.1 64-bit 4CPU 2967 +21 -21 53.8% -23.5 44.7% 700 50.8% 
  Rybka 2.1 32-bit (C+1 OSO) 2967 +31 -30 72.3% -160.2 28.5% 410 65.9% 
  Rybka 1.1 32-bit 2959 +18 -17 69.7% -137.3 38.1% 1161 53.1% 
  Hiarcs 11.2 2CPU 2958 +33 -33 59.6% -65.0 34.8% 313 50.9% 
  Deep Shredder 10 32-bit 4CPU 2957 +100 -98 54.8% -31.0 38.7% 31 51.2% 
  Deep Fritz 10 2CPU 2955 +47 -46 63.9% -85.9 47.4% 137 52.0% 
  Zap!Chess Paderborn 64-bit 4CPU 2953 +24 -24 65.2% -101.1 43.7% 558 62.8% 
5 Loop M1-T 64-bit 4CPU 2948 +21 -21 50.1% +0.1 46.3% 702 50.6% 
  Zap!Chess Zanzibar 32-bit 2CPU 2947 +172 -166 55.0% -45.5 50.0% 10 51.6% 
  Hiarcs 11.1 2CPU 2943 +20 -20 57.4% -48.5 41.8% 823 55.9% 
  Loop 13.6 64-bit 4CPU 2941 +25 -25 51.4% -7.8 41.8% 502 48.2% 
  Loop M1-T 64-bit 2CPU 2941 +25 -25 47.5% +14.3 46.8% 468 61.7% 
  Zap!Chess Paderborn 32-bit 4CPU 2932 +50 -50 44.8% +29.7 41.1% 124 51.9% 
6 Deep Shredder 10 64-bit 4CPU 2931 +18 -18 50.1% -0.6 40.1% 1037 57.2% 
  LoopMP 12.32 2CPU 2928 +20 -20 50.4% -2.7 42.6% 753 53.0% 
  Naum 2.1 64-bit 2CPU 2927 +23 -23 49.1% +5.5 43.2% 577 51.4% 

[Spock]

Well - in Zappa's case I certainly bought Zappa Mexico thinking it was the exact same version that is playing now. False marketing otherwise I would say

But generally yes - the programmers tend to use their very latest unreleased versions that are stronger than the current comemrcial release. Hiarcs a case in point for example, in last WCCC.

[AGove]

Well - in Zappa's case I certainly bought Zappa Mexico thinking it was the exact same version that is playing now.

From the Hiarcs Chess Forum http://www.hiarcs.net/forums/viewtopic. ... 4&start=75 Anthony Cozzie: "... we are using almost exactly the same version as we have distributed. I fixed a few very small bugs right before I left, perhaps good for 2 elo [Wink] Don't worry, if you ordered Zappa Mexico you too will get this amazing +2 elo version."

[geots]

Well - in Zappa's case I certainly bought Zappa Mexico thinking it was the exact same version that is playing now.

From the Hiarcs Chess Forum http://www.hiarcs.net/forums/viewtopic. ... 4&start=75 Anthony Cozzie: "... we are using almost exactly the same version as we have distributed. I fixed a few very small bugs right before I left, perhaps good for 2 elo [Wink] Don't worry, if you ordered Zappa Mexico you too will get this amazing +2 elo version."

You might be interested- i think anthony is selling some beach front property in Oklahoma with a nice view of the ocean also

[AGove]

George, be plain. You think that engine authors are liars and cheats?

[geots]

George, be plain. You think that engine authors are liars and cheats?

If you so choose, i will be plain. The words "liars and cheats" are awfully strong words and they did not come from me. Possibly you missed my smiley when you replied to my thread. In fact, i dont know a programmer that i would put in that category. I will concede that Anthony's case is unique from any that i have seen in the past. For two reasons:

1. He is the first programmer that i have seen release the version that is playing in a tournament or match BEFORE the match starts.

2. He is the first programmer that i have seen actually come out and say that the version he was using was the same version playing in match or tournament. Usually that is left to the marketing company that is pushing the programmers product.

Did i say two reasons? I must not be able to count.

3. I made the above post in the Rybka forum BEFORE Anthony made the statement that you refer to. So it would be slightly hard, if not impossible, for me to call him a liar because at that time he had not said anything. He had made no statement for me to challenge. Thats pretty elementary and not hard to understand.


But since Anthony has chosen to come out and make that statement- remembering that i have never seen a programmer do that before (im sure i will be challenged on this)- i have no choice but to take him at his word. I consider Anthony to be a person of honor and integrity.

Hope this was plain enough. No hard feelings.

Best to You

[AGove]

Okay George, You've explained yourself very well and I accept what you say.

[emerson4301972]

Maybe its Rybka that is different in the match, not Zappa. Rybka has been using different versions in the match and the new version may not have been thouroughly tested and may have some bugs.

In CEGT they use the same opening book. In Junior's case, its performance is very different in CEGT than in using its own book. With its own book, it ends up in positions where it plays well.

Also in the match they are using 8 cores, Zappa scales more efficiently than Rybka and 2 cores is very far from 8 cores.

If its more of buggy Rybka that resulted from poor result for Rybka in this match, then the owner of ChessOk must have been very relieved that the match with Junior did not push tru.

[aung]

Hi..,i new for here..forgive me if i wrong..
AFAIK tester groups using isn't same tour Computers..
eventhough they use same 8 cores for testing they will get some different result..and is about adjust engine setting too..in the tours author didn't use same defult setting( i feel it)..and another thing is memory speed..HDD speed..etc..all are different with tours Computers...
Now i hope u got answer..I Believe programmers done theirs best..they not lie to us..but is different for user's using skill..

Thanks..