SWCR after 58.640 games, 4 of 6 IPP family engines tested

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Frank Quisinsky
Posts: 6928
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

SWCR after 58.640 games, 4 of 6 IPP family engines tested

Post by Frank Quisinsky »

Hi there,

at the moment the latest two IvanHoe versions are still running. Three people compiled IvanHoe: Peterpan, Vlad and Ahmed.

The Peterpan version played all 920 games. The Vlad and Ahmed version are still running.

Shortly to the Peterpan version:
After all what I saw a very stable version. No crash, no other problems, time mangement is better as from Fire 1.31. Interesting playing style, much more interesting as the playing style from Rybka, Houdini, Fire ... with more aggressiveness.

BTW:
Interesting are the results from GullChess 1.0a x64. GullChess 1.0a x64 need 730 games for a stable rating +-10 (= 10 ELO). New SWCR record, the older hold Naum 4.2 w32. Naum 4.2 w32 need 620 games for a stable rating +-10 (=20 ELO).

If one of the sill running IvanHoe comes with the same crash problems, like Houdini 1.03a or FireBird 1.1 Dr Deab I, I will stop the test directly.

Here are the rating list (all SWCR-32 / SWCR-64 games):

After 58.640 SWCR games (one game need around 40 minutes)
Last update: October 12th, 2010 (13:00)

At the moment IvanHoe T04 x64 and IvanHoe 52iUSTMO
are still running for SWCR-64!

Code: Select all

Rank Name                          Elo    +    - games score oppo. draws 
   1 Houdini 1.03a x64            2946   22   21   920   79%  2716   29% 
   2 Rybka 4 x64                  2936   17   16  1560   79%  2703   30% 
   3 IvanHoe B52aC x64            2919   20   20   920   76%  2733   36% 
   4 Rybka 3 x64                  2903   23   22   840   78%  2685   28% 
   5 Stockfish 1.8.0 JA x64       2901   18   18  1240   74%  2718   33% 
   6 Fire 1.31 x64                2899   21   20   920   75%  2718   36% 
   7 FireBird 1.1 x64 WD          2896   20   19   920   73%  2734   41% 
   8 Rybka 4                      2894   20   20   960   76%  2703   32% 
   9 Stockfish 1.7.1 JA x64       2894   19   18  1120   76%  2704   34% 
  10 Stockfish 1.7.1 JA           2874   18   18  1200   75%  2690   31% 
  11 Stockfish 1.8.0 JA           2863   20   20   920   74%  2697   37% 
  12 Rybka 3                      2860   16   16  1520   74%  2689   31% 
  13 Critter 0.80 x64             2832   17   17  1240   65%  2721   36% 
  14 Naum 4.2 x64                 2830   14   14  1800   66%  2709   36% 
  15 Stockfish 1.6.3 JA           2827   18   18  1080   71%  2680   36% 
  16 Naum 4.2                     2824   16   15  1440   68%  2700   36% 
  17 Critter 0.80                 2814   21   20   800   66%  2703   39% 
  18 Naum 4.1                     2811   20   19   920   68%  2686   35% 
  19 Critter 0.70 x64             2804   20   20   880   65%  2699   38% 
  20 Stockfish 1.6.0 JA           2802   19   18   960   68%  2687   39% 
  21 Shredder 12                  2800   10   10  3760   63%  2704   37% 
  22 Komodo 1.2 JA x64            2799   15   15  1560   63%  2706   40% 
  23 Komodo 1.0 JA x64            2788   20   20   840   64%  2690   40% 
  24 Shredder 12 x64              2785   15   15  1600   63%  2688   34% 
  25 Naum 4.0                     2784   19   19   960   65%  2682   38% 
  26 Critter 0.70                 2777   19   19   920   61%  2703   39% 
  27 Deep Fritz 12                2777   15   15  1520   61%  2704   42% 
  28 GullChess 1.0a x64           2761   18   18  1000   54%  2734   41% 
  29 Komodo 1.2 JA                2761   18   18   960   58%  2708   42% 
  30 Fritz 12                     2745   16   16  1160   59%  2688   44% 
  31 Spark 0.5 x64                2737   15   15  1560   54%  2708   37% 
  32 Hiarcs 13.1                  2735   13   13  1960   52%  2723   41% 
  33 Thinker 5.4d Inert x64       2733   14   14  1800   53%  2712   39% 
  34 Stockfish 1.5.1 JA           2731   19   19   840   59%  2672   43% 
  35 Zappa Mexico II x64          2721   14   14  1800   51%  2712   40% 
  36 Komodo 1.0 JA                2716   16   16  1200   53%  2694   40% 
  37 Spark 0.4 x64                2715   20   20   840   53%  2694   40% 
  38 Spark 0.5                    2713   18   18   960   51%  2710   41% 
  39 Thinker 5.4d Inert           2711   13   13  2000   51%  2703   42% 
  40 Protector 1.3.4 JA x64       2709   14   14  1760   50%  2713   37% 
  41 Fruit 09_07_05 x64           2705   14   14  1800   49%  2712   34% 
  42 Critter 0.60 x64             2698   20   20   840   50%  2695   38% 
  43 Doch 1.3.4 JA                2690   19   19   840   51%  2686   44% 
  44 Critter 0.60                 2688   19   19   920   50%  2688   39% 
  45 Hannibal 1.0a x64            2688   17   17  1240   45%  2725   36% 
  46 Spark 0.4                    2686   19   19   880   49%  2692   42% 
  47 Sjeng WC-2008 x64            2684   14   14  1800   46%  2713   36% 
  48 Junior 11.2                  2684   19   19   960   46%  2712   33% 
  49 Protector 1.3.5 x64          2682   20   20   840   47%  2704   39% 
  50 Junior 11.2 x64              2682   15   15  1560   47%  2709   30% 
  51 Protector 1.3.4 JA           2679   15   16  1360   46%  2706   39% 
  52 Onno 1.2.70 x64              2677   15   15  1560   46%  2709   37% 
  53 Cyclone xTreme Wrath         2675   17   17  1080   47%  2697   41% 
  54 Protector 1.3.2              2674   17   17  1160   47%  2695   41% 
  55 Protector 1.3.5 JA           2670   19   19   840   44%  2712   42% 
  56 Junior 2010                  2669   16   16  1240   47%  2691   36% 
  57 Onno 1.1.1 x64               2668   20   20   840   46%  2696   40% 
  58 Hiarcs 12.1                  2667   19   19   880   47%  2689   41% 
  59 Protector 1.3.1b             2667   19   19   840   47%  2690   42% 
  60 Doch 1.2 JA                  2665   19   19   840   48%  2679   40% 
  61 Sjeng WC-2008                2665   13   13  2000   44%  2704   37% 
  62 Hiarcs 12.1 Sharpen PV       2665   16   16  1280   45%  2699   39% 
  63 Zappa Mexico II              2662   13   13  2000   44%  2704   42% 
  64 Spark 0.3a                   2655   17   17  1120   44%  2698   41% 
  65 Doch 09.980 JA               2652   19   19   840   46%  2675   42% 
  66 Junior 11.1a                 2645   19   19   960   44%  2691   36% 
  67 Junior 11.1a x64             2645   20   20   840   43%  2697   32% 
  68 Spark 0.3                    2643   19   19   880   43%  2690   42% 
  69 Hannibal 1.0a                2640   20   20   840   39%  2715   36% 
  70 Onno 1.2.70                  2636   18   18   960   39%  2714   40% 
  71 Onno 1.1.1                   2630   14   15  1520   40%  2695   41% 
  72 Loop M1-T x64                2626   19   19   960   36%  2732   36% 
  73 Loop 2007 x64                2626   15   15  1600   38%  2714   35% 
  74 Fruit 05/11/03               2617   13   13  2000   37%  2705   41% 
  75 Loop 13.6                    2614   15   15  1520   38%  2696   39% 
  76 Loop 2007                    2614   19   19   960   35%  2715   36% 
  77 Critter 0.52b                2613   18   18  1040   38%  2698   37% 
  78 Umko 1.0 x64                 2610   17   17  1240   33%  2728   37% 
  79 Glaurung 2.2 JA              2609   18   18  1080   37%  2700   36% 
  80 Ktulu 9.03                   2609   15   16  1520   36%  2708   30% 
  81 Twisted Logic 20100131x x64  2609   18   18  1120   35%  2714   32% 
  82 SmarThink 1.20 x64           2599   14   14  1800   35%  2715   34% 
  83 Equinox 0.83 x64             2598   18   18  1200   32%  2737   32% 
  84 SmarThink 1.20               2597   13   13  2000   34%  2705   37% 
  85 Crafty 23.3 JA x64           2596   17   17  1240   32%  2728   33% 
  86 Twisted Logic 20100131x      2574   15   15  1600   32%  2707   30% 
  87 Spike 1.2 Turin              2573   16   16  1480   31%  2708   34% 
  88 Cipollino 3.25 x64           2566   20   20   960   28%  2735   30% 
  89 BugChess2 1.7 x64            2556   20   20   960   28%  2718   31% 
  90 Scorpio 2.6 JA x64           2553   18   19  1120   28%  2716   32% 
  91 Crafty 23.2 JA x64           2552   18   19  1120   28%  2716   30% 
  92 Chronos 1.99 x64             2550   18   18  1120   27%  2716   33% 
  93 Crafty 23.3 JA x64 NP        2546   20   20   960   25%  2736   30% 
  94 Daydreamer 1.75 JA x64       2519   19   19  1120   24%  2717   30% 
  95 Tornado 3.6.7 x64            2478   23   24   840   19%  2724   24% 
The results from IvanHoe T0.4 x64 and IvanHoe 52iUSTMO x64 can be follow in SWCR Live-Mode.

After this test a bigger SWCR-32 tournament will start. So in SWCR-64 are also 24 different engines activ.

Notice:
In SWCR-32 no IPP family engines playing.
In SWCR-64 later only one of the IvanHoe versions are still playing. The games from the others can be found in my download selection later.

Next database update will be available around October 18th, 2010.

Frank's Chess Page, SWCR
http://www.amateurschach.de

Best
Frank
gerold
Posts: 10121
Joined: Thu Mar 09, 2006 12:57 am
Location: van buren,missouri

Re: SWCR after 58.640 games, 4 of 6 IPP family engines teste

Post by gerold »

Thanks for your testing Frank.

Your top 6 is in line with my top six after 1000 games at 6/3 tc.
I found Peterpan version to be the best of the 3 you are testing.
I am running w32 versions.

I had no crash for Houdini in Arena 1.1.

Best,
Gerold.
Frank Quisinsky
Posts: 6928
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: SWCR after 58.640 games, 4 of 6 IPP family engines teste

Post by Frank Quisinsky »

Hi Gerold,

the crashes in Houdini 1.03a x64 and FireBird 1.1 WD x64 have to do with the ponder mode. Furthermore, I am playing in SWCR without resign. Crahes comes often before the games ended, means before mate or clearly remis.

38x for Houdini 1.03a x64 after 920 games.
53x for FireBird 1.1 WD x64 after 920 games.

A lot crashes and many work by myself. For each crash I have to start task-manager, delete the engine from task-manager, edit the Shredder *.sto file, stop and start the tourney and so on ... I will not do all this again if one the two IvanHoe comes again with the same ponder problems.

No problems with Fire 1.31 x64 and IvanHoe B52aC x64. Time mangement is better for IvanHoe B52aC x64 if I compare with Fire 1.31 x64. So my favorit at the moment is clearly IvanHoe B52aC x64. But let us wait of the other two running IvanHoe versions.

Perhaps interesting.
Start of the year I am testing Robolito 0.9 w32 with 620 games. No problems, no crash ... ELO is 23 better as from Rybka 3. So its clear that no bigger ELO improvements in the IvanHoe B52ac x64 I tested. But perhaps the other two are better with longer time controls. I don't know.

At the moment IvanHoe 52iUSTMO x64 played 20 games without one crash. Seems to be OK :-)

Best
Frank

PS: Could you send me your list per PM or post here. Interesting ... each rating is interesting for myself :-)
gerold
Posts: 10121
Joined: Thu Mar 09, 2006 12:57 am
Location: van buren,missouri

Re: SWCR after 58.640 games, 4 of 6 IPP family engines teste

Post by gerold »

Still testing Frank Don't have a complete rating list yet.
My test are for one of the chess companies.

Best,
Gerold.
gerold
Posts: 10121
Joined: Thu Mar 09, 2006 12:57 am
Location: van buren,missouri

Re: SWCR after 58.640 games, 4 of 6 IPP family engines teste

Post by gerold »

Have you tested Grapefruit. Is Gullchess by the same person
that wrote Grapefruit.

Best,
Gerold.
Frank Quisinsky
Posts: 6928
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: SWCR after 58.640 games, 4 of 6 IPP family engines teste

Post by Frank Quisinsky »

Hi Gerold,

before I start SWCR around August / September last year I tested Grapefruit in my SWCR-blitz and in a Fruit Clone tournament on my Notebook with around 300 games per engine.

In the Fruit clone tournament I added all what is available and stronger ... after my investigation.

Later one of the members of CSS Forum gave me the tip that the Grapefruit 1.0 Beta is the strongest fromt he Grapefruit family. And yes in my blitz list this version is clearly better as the Grapefruit versions I tested before.

But with longer time controls (40 in 10, on my 2.0 GHz Dual Core T7300 Notebook with ponder = On are the strongest Fruit clone ... very clear ... Cyclone xTreme Wrath. Grapefruit 1.0 Beta is on third, on second on other Cyclone xTreme version, I believe Fury ... don't know ... have the results on my notebook and can at the moment not look.

Later I started SWCR-32 ... I tested Cyclone xTreme Wrath only.

The different between Cyclone xTreme Wrath and Grapefruit 1.0 Beta are around 25 ELO with 40 in 10 on my T7300 notebook with ponder = on and 256Mb for Hashtables.

Best
Frank

Edit: Correction ... I tested on my Notebook with 40 in 20. I found the results. Wrath is 6 ELO better as Fury and 22 ELO better as Grapefruit 1.0 Beta. 380 games played for each of 14 Fruit clones I added.
gerold
Posts: 10121
Joined: Thu Mar 09, 2006 12:57 am
Location: van buren,missouri

Re: SWCR after 58.640 games, 4 of 6 IPP family engines teste

Post by gerold »

Thanks Frank.
I was thinking of testing Grapefruit again vs. Gullchess. on one
of my old computers.

Best,
Gerold.
Frank Quisinsky
Posts: 6928
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: SWCR after 58.640 games, 4 of 6 IPP family engines teste

Post by Frank Quisinsky »

Hi Gerold,

Grapefruit 1.0 Beta:
194.048 bytes
30.11.2008

This one is the strongest!

Best
Frank
User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 8:44 pm
Location: Amman,Jordan

Re: SWCR after 58.640 games, 4 of 6 IPP family engines teste

Post by Dr.Wael Deeb »

Hi Frank,
Very pleased with the result of FireBird 1.1 WD :D
Actualy the engines from 4th to 7th place are nearly identical in strength and with more games my settings could climb easily to number 4....
Cheers,
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
Frank Quisinsky
Posts: 6928
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: SWCR after 58.640 games, 4 of 6 IPP family engines teste

Post by Frank Quisinsky »

Hi Wael,

that's right!

Interesting is, that Robbolito 0.9 x64 is stronger as Fire 1.31 x64 and FireBird 1.1 WD x64, around the same level as the IvanHoe B52aC x64 version.

Now it's interesting how strong are the other two still running IvanHoe compiles. But I believe not stronger as 2.920 ELO too. So after around 9 months no improvements in ELO strength for all the IvanHoe's, Robbolito and Fire versions. But a clearly more interesting playing style from IvanHoe B52aC as the others.

Not sure, let us wait of the latest two results.
So far no GUI crash in testing IvanHoe T0.4 x64 and IvanHoe 52iUSTMO x64 after the first 25/39 of 920 games.

If you like ... look in the games!
Download area from my webpage, 4 IPP family engines with Shredder Classic 4.0 GUI comments added.

After this test with the two still running IvanHoe versions I have enough from the IPP family :-)

Very interesting is the comming soon SWCR-32 tournament with 6 updates (will be start around October 19th, 2010):

Code: Select all

103. w32 Jonny 4.00                NEW      Scheduled: ~ 19.10.2010 private status
102. w32 Stockfish 1.9.1 JA        Update   Scheduled: ~ 19.10.2010
101. w32 GullChess 1.0a            NEW      Scheduled: ~ 19.10.2010
100. w32 Equinox 0.83              NEW      Scheduled: ~ 19.10.2010 private status
099. w32 WB Crafty 23.3 JA         NEW      Scheduled: ~ 19.10.2010
098. w32 Bright 0.5c               NEW      Scheduled: ~ 19.10.2010
Best
Frank