GGT2

[rainhaus]

After 20 gambit-rounds I've stopped the 'Great Gambit Tournament' for an engine-update. The double round-robin tourney with the strongest multi processor engines, is running now with the newest versions of Firebird (1.2), Deep Fritz (12], Naum (4.2) and Stockfish (1.7.1).
"GGT 1" (january to march, 2010) was won by Firebird 1.0 Beta. I'll post the results soon.

[rainhaus]

Firebird 1.2 newSMP wins C02 Nimzowitsch Gambit

Opening: C02 French, Advance Variation
Round 1/2, Position 21: Nimzowich Gambit
1.e4 e6 2.d4 d5 3.e5 c5 4.Qg4

Code: Select all

                                  1  2  3  4  5  6  7    Points
1   FireBird 1.2 newSMP x 64      ** 0½ ½½ 1½ 1½ 11 11   8.5/12
2   Stockfish 1.7.1 JA x64        1½ ** 01 ½0 1½ 1½ 11   8.0/12
3   Deep Rybka 3 x64              ½½ 10 ** ½½ 0½ 11 11   7.5/12
4   Naum 4.2 x64                  0½ ½1 ½½ ** 1½ 10 ½1   7.0/12
5   Deep Shredder 12 x64          0½ 0½ 1½ 0½ ** 0½ 10   4.5/12
6   Zappa Mexico II x64           00 0½ 00 01 1½ ** ½0   3.5/12
7   Deep Fritz 12                 00 00 00 ½0 01 ½1 **   3.0/12
    ----------------------
    42 games

Naum 4.2 wins C022 Danish Gambit with the tie breaker

Opening:
Round 3/4, Position 22: Danish Gambit
1.e4 e5 2.d4 exd4 3.c3

Code: Select all

                                  1  2  3  4  5  6  7    Points  Tie Breaker*
1   Naum 4.2 x64                  ** 10 1½ ½½ ½½ ½1 ½1   7.5/12  42.25
2   FireBird 1.2 newSMP x64       01 ** ½0 ½1 ½1 1½ ½1   7.5/12  40.50
3   Stockfish 1.7.1 JA x64        0½ ½1 ** ½0 1½ ½1 11   7.5/12  39.50
4   Deep Rybka 3 x64              ½½ ½0 ½1 ** ½0 1½ 1½   6.5/12
5   Deep Fritz 12                 ½½ ½0 0½ ½1 ** ½½ 0½   5.0/12
6   Deep Shredder 12 x64          ½0 0½ ½0 0½ ½½ ** 1½   4.5/12
7   Zappa Mexico II x64           ½0 ½0 00 0½ 1½ 0½ **   3.5/12
    ----------------------
    42 games

Firebird 1.2 new SMP wins C23 Calabrese Counter Gambit

Opening: C23 Bishops Opening
Round 5/6, Position 23: Calabrese Countergambit
1.e4 e5 2.Bc4 f5

Code: Select all

                                  1  2  3  4  5  6  7    Points  Tie Breaker*
1   FireBird 1.2 newSMP x64       ** ½1 10 1½ ½1 1½ 11   9.0/12
2   Naum 4.2 x64                  ½0 ** ½1 1½ ½1 11 10   8.0/12
3   Stockfish 1.7.1 JA x64        01 ½0 ** 0½ 1½ 10 ½½   5.5/12  32.25
4   Deep Rybka 3                  0½ 0½ 1½ ** 0½ 01 1½   5.5/12  30.25
5   Zappa Mexico II x64           ½0 ½0 0½ 1½ ** ½½ ½½   5.0/12  28.50
6   Deep Fritz 12                 0½ 00 01 10 ½½ ** 1½   5.0/12  26.50
7   Deep Shredder 12 x64          00 01 ½½ 0½ ½½ 0½ **   4.0/12
    --------------------------
    42 games

Total Scores Round 1-6; 3 Gambits: Nimzo, Danish, Calabrese

Code: Select all

                             Score      SB*     Fire   Naum   Stock  DpRy   DpSh   DpFr   Zappa 
-------------------------------------------------------------------------------------------------
 1: FireBird 1.2 newSMP x64  25.0 / 36  400.00  XXXXXX 1=01=1 0==010 ===11= 1=1=11 11=11= 11=1=1
 2: Naum 4.2 x64             22.5 / 36  371.25  0=10=0 XXXXXX =11==1 ====1= 1==110 =1==11 10=1=1
 3: Stockfish 1.7.1 JA x64   21.0 / 36  343.25  1==101 =00==0 XXXXXX 01=00= 1==1== 111=10 1=111=
 4: Deep Rybka 3 x64         19.5 / 36  329.25  ===00= ====0= 10=11= XXXXXX 0=1=1= 11=001 111=0=
 5: Deep Shredder 12 x64     13.0 / 36  229.25  0=0=00 0==001 0==0== 1=0=0= XXXXXX 10==0= 0=1===
 6: Deep Fritz 12            13.0 / 36  220.50  00=00= =0==00 000=01 00=110 01==1= XXXXXX =10===
 7: Zappa Mexico II x64      12.0 / 36  208.00  00=0=0 01=0=0 0=000= 000=1= 1=0=== =01=== XXXXXX
-------------------------------------------------------------------------------------------------
126 games: +37 =55 -34

Total Performance Round 1-6; 3 Gambits

Code: Select all

Program                 Games  W    D    L     Points Perform
FireBird 1.2 newSMP x64   36   +18  =14  - 4   25.0   69.4%
Naum 4.2 x64              36   +14  =17  - 5   22.5   62.5%
Stockfish 1.7.1 JA x64    36   +14  =14  - 8   21.0   58.3%
Deep Rybka 3 x64          36   +11  =17  - 8   19.5   54.2%
Deep Shredder 12 x64      36   + 4  =18  -14   13.0   36.1%
Deep Fritz 12             36   + 6  =14  -16   13.0   36.1%
Zappa Mexico II x64       36   + 4  =16  -16   12.0   33.3%

*Tie Breaker = SB = Sonneborn/Berger-Feinwertung

EloStat-tables will be presented after further rounds.

PGN-Link: http://www.file-upload.net/download-250 ... 3.pgn.html

Next:
- GGT2 C23: Lewis Gambit, Wing Gambit, C24: Urusov Gambit
- GGT1 results
-----------
Book
50 gambit starting positions. GGT1:Eco00 - B44. GGT2:EcoC02 - E60
Test conditions
Time Control: tournament level 40/20', 20/10', 10'+12''
System: Intel Core i7 920, oc 3600-3800 MHz, 6 GB DDR3 RAM. Vista 64
Hyperthreading off, Turbo Mode off.

Engine parameters: 3 threads. Ponder off. 1,2 GB Hash.
EGTB 3,4,5: Nalimov, TotalBases, sometimes TripleBases. Stockfish don't use EGTB. Bitbases are not needed. Firebird's TotalBases and RAM-resident TripleBases don't work always properly.
Fritz12-GUI: remis late, resign late.

[beram]

Great news Rainer,

I was waiting for someone to do this.
Fire(Bird) 1.2 against the strongest engines, at long time control.

grts Bram

[gerold]

Great news Rainer,

I was waiting for someone to do this.
Fire(Bird) 1.2 against the strongest engines, at long time control.

grts Bram

I can second that. Good to see this competition of the
top engines.

Best,
Gerold.

[gerold]

Thanks Rainer.

Best,
Gerold.

[rainhaus]

Deep Rybka 3 wins C23 Lewis Countergambit
Opening: C23 Bishops Opening
Round 7/8, Position 24
1.e4 e5 2.Bc4 Bc5 3.c3 d5

Code: Select all

                             1  2  3  4  5  6  7            SB
1   Deep Rybka 3 x64         ** ½½ ½½ ½½ 1½ 11 1½   8.0/12
2   FireBird 1.2 newSMP x64  ½½ ** ½½ 1½ ½1 ½½ ½1   7.5/12
3   Naum 4.2 x64             ½½ ½½ ** ½½ ½1 ½½ ½1   7.0/12
4   Stockfish 1.7.1 JA x64   ½½ 0½ ½½ ** ½½ 1½ ½1   6.5/12
5   Deep Fritz 12            0½ ½0 ½0 ½½ ** 10 ½½   4.5/12  26.25
6   Zappa Mexico II x64      00 ½½ ½½ 0½ 01 ** 01   4.5/12  26.25
7   Deep Shredder 12 x64     0½ ½0 ½0 ½0 ½½ 10 **   4.0/12
-----------------------------------------------------------
42 games

Stockfish 1.7.2 wins C23 Wing Gambit
Opening: C23 Bishops Opening
Round 9/10, Position 25
1.e4 e5 2.Bc4 Bc5 3.b4

Code: Select all

                             1  2  3  4  5  6  7            SB
1   Stockfish 1.7.1 JA x64   ** ½½ 10 1½ 11 11 1½   9.0/12
2   FireBird 1.2 newSMP x64  ½½ ** ½½ 10 11 ½1 11   8.5/12
3   Deep Rybka 3 x64         01 ½½ ** 1½ ½0 10 1½   6.5/12
4   Deep Shredder 12 x64     0½ 01 0½ ** 01 ½½ 01   5.0/12  29.25
5   Naum 4.2 x64             00 00 ½1 10 ** 1½ 01   5.0/12  24.75
6   Deep Fritz 12            00 ½0 01 ½½ 0½ ** ½½   4.0/12  22.25
7   Zappa Mexico II x64      0½ 00 0½ 10 10 ½½ **   4.0/12  21.75
-----------------------------------------------------------
42 games

Stockfish 1.7.2 wins C24 Ponziani-Urusov Gambit
Opening: C23 Bishops Opening
Round 11/12, Position 26
1.e4 e5 2.Bc4 Nf6 3.d4 exd4 4.Nf3

Code: Select all

                             1  2  3  4  5  6  7            SB
1   Stockfish 1.7.1 JA x64   ** ½½ ½1 1½ 1½ ½1 11   9.0/12
2   Deep Rybka 3 x64         ½½ ** ½½ 1½ 0½ ½1 ½1   7.0/12  39.00
3   FireBird 1.2 newSMP x64  ½0 ½½ ** 1½ ½1 ½½ ½1   7.0/12  37.75
4   Naum 4.2 x64             0½ 0½ 0½ ** ½½ 11 11   6.5/12
5   Deep Shredder 12 x64     0½ 1½ ½0 ½½ ** 0½ ½1   5.5/12
6   Deep Fritz 12            ½0 ½0 ½½ 00 1½ ** ½½   4.5/12
7   Zappa Mexico II x64      00 ½0 ½0 00 ½0 ½½ **   2.5/12
-----------------------------------------------------------
42 games

Total Scores Round 1-12; 6 Gambits: Nimzo, Danish, Calabrese, Lewis, Wing, Urusov

Code: Select all

    Program                       
                       Fire     Stock    Naum     Rybka    DShr     DFr      Zappa    Points  SB
  1 FireBird 1.2 x     xxxxx    5.0-7.0  8.5-3.5  7.0-5.0  9.0-3.0  9.0-3,0  9.5-2,5  48.0/72
  2 Stockfish 1.7.1    7.0-5.0  xxxxx    6.0-6.0  5.0-7.0  8.5-3.5  9.0-3.0  10.02.0  45.5/72
  3 Naum 4.2           3.5-8.5  6.0-6.0  xxxxx    6.5-5.5  7.5-4.5  9.5-2.5  8.0-4.0  41.0/72 1378.50
  4 Deep Rybka 3       5.0-7.0  7.0-5.0  5.5-6.5  xxxxx    7.0-5.0  7.5-4.5  9.0-3.0  41.0/72 1344.75
  5 Deep Shredder 12   3.0-9.0  3.5-8.5  4.5-7.5  5.0-7.0   xxxxx   5.0-7.0  6.5-5.5  27.5/72
  6 Deep Fritz 12      3.0-9.0  3.0-9.0  2.5-9.5  4.5-7.5  7.0-5.0   xxxxx   6.0-6.0  26.0/72
  7 Zappa Mexico II    2.5-9.5  2.0-10.0 4.0-8.0 3.0-9.0  5.5-6.5  6.0-6.0    xxxxx   23.0/72
----------------------------------------------------------------------------------------
252 games

So far Firebird controls all the other engines with the exception of Stockfish.
New Stockfish performs stunning and is obviously not only trimmed against the former all-powerful Rybka.
New Naum is considerably stronger than the predecessor but seems to be very afraid of "Fire". Fertilized with the source codes of Ippo & Co. it could probably refresh its genes. The informed Talkchess-reader knows that the author made indeed some hints in this direction, also considering that it is easier said than done.
For the sake of many thrilling tournaments, Rybka 3 is finally dethroned and it is not even more primus inter pares. Long expected misty Rybka number 4 is lying an eternity on a sheltered and secret place. Now, the day of release draws near, but nobody knows exactly what will come to light. Perhaps a new monster again. Fire 1.3, will it be strong enough? I'm going out on a limb and say, no, it will be not.
Shredder lost within a few months its contact to the engines in front, but it still belongs to the Best of Five.
New Fritz is a bitter disappointment for the friends of pure playing strength.
Orphaned old Zappa is still about spoiling the scores of Shredder and Fritz.
Of course, the sample is still small, but tendencies often show very fast and GGT1 already has verified the tournament strength of Firebird on longer time control too. The tournament goes on and will make the necessary corrections.

Total Performance Round 1-12

Code: Select all

Program                   Games   W    D    L  Points   Perf
---------------------------------------------------------------
FireBird 1.2 x64 newSMP    72    +30  =36  -06 48.0/72  66%
Stockfish 1.7.1 JA x64     72    +29  =33  -10 45.5/72  63%
Naum 4.2 x64               72    +24  =34  -14 41.0/72  56%
Deep Rybka 3 x64           72    +22  =38  -12 41.0/72  56%
Deep Shredder 12 x64       72    +10  =35  -27 27.5/72  38%
Deep Fritz 12              72    +9   =34  -29 26.0/72  36%
Zappa Mexico II x64        72    +8   =30  -34 23.0/72  31%
-----------------------------------------------------------------
252 games

Total Elo-Ranking Round 1-12 CEGT-Calibration

Code: Select all

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws   CEGT40/20/4T 
  1 FireBird 1.2 newSMP x64        : 3244   58  56    72    66.7 %   3123   50.0 %  0000
  2 Stockfish 1.7.1 JA x64         : 3221   61  59    72    63.2 %   3127   45.8 %  3158
  3 Naum 4.2 x64                   : 3182   59  59    72    56.9 %   3133   47.2 %  3139
  4 Deep Rybka 3 x64               : 3182   56  55    72    56.9 %   3133   52.8 %  3181
  5 Deep Shredder 12 x64           : 3069   57  59    72    38.2 %   3152   48.6 %  3062
  6 Deep Fritz 12                  : 3055   58  60    72    36.1 %   3154   47.2 %  3084
  7 Zappa Mexico II x64            : 3028   62  64    72    31.9 %   3159   41.7    3016
----------------------------------------------------------------------------------------
  252 games  Starting Value EloStat: 3140,calibrated on Deep Rybka 3,CEGT 40/20,4 threads

After a few more rounds I will also calibrate the results on CCRL and Ippon. Thus you get relatively soon a good estimation within the scales of the published lists, for a still ignored top engine.

*Tie Breaker = SB = Sonneborn/Berger-Feinwertung

PGN-Link: http://www.file-upload.net/download-251 ... 6.pgn.html

Next:
- GGT2 C31: Falkbeer Counter Gambit, C32 Charousek Gambit, C37: Wild Muzio Gambit
- GGT1 results
-----------
Book
50 gambit starting positions. GGT1:Eco00 - B44. GGT2:EcoC02 - E60
Test conditions
Time Control: tournament level 40/20', 20/10', 10'+12''
System: Intel Core i7 920, oc 3600-3800 MHz, 6 GB DDR3 RAM. Vista 64
Hyperthreading off, Turbo Mode off.

Engine parameters: 3 threads. Ponder off. 1,2 GB Hash.
EGTB 3,4,5: Nalimov, TotalBases, sometimes TripleBases. Stockfish don't use EGTB. Bitbases are not needed. Firebird's TotalBases and RAM-resident TripleBases don't work always properly.
Fritz12-GUI: remis late, resign late.

[beram]

Thx again,

In my private testings I found similar results, for that Rybka 3 is not even the second strongest. In Time control 4m 2sec Fire 1.2 is better than Stockfish 1.7 it wins with about 56 % margins, as for Rybka 3 it beats with 60 %

Looking forward to final results and I am already looking forward to the match between Rybka 4and Fire 1.3. My guess is that it will become very close between those two.

grts Bram

[rainhaus]

Hi Bram,
thx for your continuing interest Concerning your private testing, a few more infos wouldn't be bad. What do you mean by 56% or 60% margins? How many games are played, number of processors, clock rate, 32 or 64 Bit?

Concerning the imminent new Rybka: I put two and two together and there comes out a very strong new version. If I remember well, Rybka 4 was not far from release approximately one year ago. In other words, the program was already at that time significant stronger than the predecessor. Ok, it's not proven, but very likely. I bet my bottom inflationary Euro that this prototype hasn't been frozen or designed backward on purpose, but it was rather improved over the last year. I guess, Rybka 4 is at least 70 Elo stronger than Rybka 3. It's a bet I can't lose. If I'm right than I have the satisfaction that it is so, otherwise I breath a sigh of relief that the tournaments of the top engines do not again become very boring .

Cheers Rainer

[beram]

Hi Bram,
thx for your continuing interest Concerning your private testing, a few more infos wouldn't be bad. What do you mean by 56% or 60% margins? How many games are played, number of processors, clock rate, 32 or 64 Bit?

Concerning the imminent new Rybka: I put two and two together and there comes out a very strong new version. If I remember well, Rybka 4 was not far from release approximately one year ago. In other words, the program was already at that time significant stronger than the predecessor. Ok, it's not proven, but very likely. I bet my bottom inflationary Euro that this prototype hasn't been frozen or designed backward on purpose, but it was rather improved over the last year. I guess, Rybka 4 is at least 70 Elo stronger than Rybka 3. It's a bet I can't lose. If I'm right than I have the satisfaction that it is so, otherwise I breath a sigh of relief that the tournaments of the top engines do not again become very boring .

Cheers Rainer

Dear Rainer,

I posted some results allready earlier and I thought it was about 56%.
In reality it was 58 %

Engine matches on my T8100, (2100Mhz, Fritzmark 6,29)
WinXP, 32bit 2CPU, Fritz 12 gui, Blitz 4m+2s - Privat and Nunn2 positions.
Sofar played, I have this score in my Fire 1.2 – Stockfish 1.7.1 games

1 FireBird 1.2 w32 +48/=57/-27 57.95% 76.5/132
2 Stockfish 1.7.1 JA +27/=57/-48 42.05% 55.5/132

as for Rybka4 I also hope it will be 70% stronger than R3, otherwise it would be disappointing for all (of his fans)

p.s. When you want the games you can pm me
grts Bram

[Frank Quisinsky]

Hi,

interesting results.

Two little things:

1. In SWCR I play on Q9550, not overclocked, with 2.83 GHz standard setting and 40 in 10 time control. I used 256Mb for Hash (more as enough). Ponder = on

You are playing with 40 in 20 and the hardware is overclocked around 1 GHz faster. I think 760Mb for Hash is more as enough for all of this engines. With too many hash the engine is a very little slower in the middle games. I think perhaps 1 ELO Means not very important but 760Mb should be enough.

2. With 5-pieces tablebases you are lost around 10 ELO compare to 4-pieces tablebases. 5-pieces are too aggressive in the important game phase. Processor is using 30% only if 12-18 pieces on the board. All this for nothing. I made in the past a lot of experients with 5-pieces and wrote in German for a lot to the topic. For eng-eng is better to play with 4-pieces, so you have the full engine power in endgames, more important as the 5-pieces tablebases.

3. You have a RAM problem with 6Gb.

Four matches are still running with ponder = off.
4x 1.2 GB RAM for hash and 5-pieces tablebases and Windows 7 (used a lot of RAM) ? So you need around ...

4x 1.2 GB for Hash
1x 1 GB for Windows 7
4x 5-pieces and 5x Chess GUI around 1.5 GB

20% of RAM should be free.
Around 8Gb !!

Install Process Explorer 12.02 (Microsoft tool, TuneUp Proezess Manager or an other good tool and check your RAM).

Keep up the good work.
I like you art of testing and your tournament idea.

Best
Frank