Naum 4 vs various Engines (1000 games) + rating list

[IWB]

Hello all,

1000 games with Naum 4 are played:

Code: Select all

Naum 4                    : 1000 (+431,=394,-175), 62.8 %

Zappa Mexico II x64           : 100 (+ 33,= 50,- 17), 58.0 %
Deep Shredder 10 x64          : 100 (+ 54,= 35,- 11), 71.5 %
Spike 1.2 Turin               : 100 (+ 62,= 31,-  7), 77.5 %
Fruit 05/11/03                : 100 (+ 54,= 34,- 12), 71.0 %
HIARCS 12 MP                  : 100 (+ 56,= 34,- 10), 73.0 %
Toga II 1.4 beta5c BB         : 100 (+ 50,= 38,- 12), 69.0 %
Rybka 3                       : 100 (+  5,= 38,- 57), 24.0 %
Rybka 2.2n2 mp                : 100 (+ 26,= 58,- 16), 55.0 %
Shredder Bonn                 : 100 (+ 43,= 37,- 20), 61.5 %
DSjeng WC2008 x64             : 100 (+ 48,= 39,- 13), 67.5 %

All this gives this Bayesrating:

Code: Select all

Rank Name                           Elo    +    - games score oppo. draws 
   1 Rybka 3                       2811   13   13  2500   77%  2615   31% 
   2 Rybka 2.3.2a mp               2708    8    8  5500   68%  2588   37% 
   3 Naum 4 x64                    2683   18   18  1000   63%  2597   39%
   4 Rybka 2.2n2 mp                2679   14   14  1700   61%  2606   41% 
   5 Rybka 1.2f                    2666    7    7  7100   65%  2561   34% 
   6 Deep Fritz 11                 2635   17   17  1200   56%  2593   39% 
   7 Zappa Mexico II x64           2620    9    9  4000   51%  2613   41% 
   8 Shredder Bonn                 2613   14   14  1700   52%  2602   36% 
   9 Deep Shredder 11 x64          2600   10   10  3400   52%  2584   40% 
  10 Strelka 2.0 B x64             2597   18   18  1000   53%  2578   44% 
  11 Rybka 1.0 Beta x64            2593   12   12  2400   53%  2571   36% 
  12 Naum 3.1 x64                  2591    8    8  5600   48%  2604   39% 
  13 Zappa Mexico I X64            2590   10   10  3600   53%  2569   40% 
  14 HIARCS 12 MP                  2574   10   10  3700   43%  2622   37% 
  15 Toga II 1.4 beta5c BB         2573   10   10  3500   44%  2611   39% 
  16 DSjeng WC2008 x64             2572   17   17  1100   49%  2580   38% 
  17 DSjeng 3.0 x64                2563   11   10  3200   41%  2625   34% 
  18 Naum 2.2                      2547   11   11  2700   43%  2587   43% 
  19 Rybka 1.0 Beta 32-bit         2541    9    9  4400   50%  2541   33% 
  20 HIARCS 11.2                   2536    8    7  6600   46%  2566   36% 
  21 Fruit 05/11/03                2535    8    8  5600   41%  2595   39% 
  22 Fruit 2.3                     2530   23   23   600   52%  2518   39% 
  23 DS 10 Balmung                 2529   11   11  2600   50%  2530   43% 
  24 LoopMP 12.32                  2525    9    9  4900   46%  2550   35% 
  25 Loop 13.5                     2524    9    9  4700   43%  2568   37% 
  26 Loop M1-P                     2522   25   25   500   49%  2528   40% 
  27 Toga II 1.2.1a                2519    8    7  6600   47%  2542   35% 
  28 ListMP 11.64b                 2518   12   12  2200   44%  2558   36% 
  29 Glaurung 2.1                  2510   20   20   800   44%  2553   34% 
  30 Deep Shredder 10 x64          2510    7    7  7400   41%  2571   36% 
  31 HIARCS 11 MP                  2506   20   20   800   45%  2541   36% 
  32 Naum 2.1 NoLearn              2505   10   10  3700   45%  2541   36% 
  33 Toga II 1.3x4                 2500   20   20   800   45%  2536   39% 
  34 Hiarcs X54 64bit              2496   23   23   600   44%  2538   37% 
  35 Spike 1.2 Turin               2481    7    7  9000   36%  2576   35% 
  36 DS 9.02                       2464   16   16  1400   38%  2552   29% 
  37 Deep Sjeng 2.7                2461   13   14  2000   31%  2591   33% 
  38 Glaurung 2-epsilon/5          2446   17   17  1300   35%  2554   31% 
  39 Deep Sjeng 2.5                2393   20   20   900   30%  2534   31% 

In case someone is interested these are the tournament conditions:

Ponder ON
6min + 3sec on Core2@2.4GHz
Opening positions
changing colors
no learning (of course!)
ONE thread
256 MB Hash
4 pc endgamedtatabases max


Bye
Ingo

[M ANSARI]

Thanks for the thorough testing ... but could more information on some of the data. Is the "one thread" for N4 or for all the engines? Also what OS is this running? With "ponder on" is this with one machine or your dual core using 1 core each to keep poner on. Thanks again.

[IWB]

Hello

Thanks for the thorough testing ... but could more information on some of the data. Is the "one thread" for N4 or for all the engines? Also what OS is this running? With "ponder on" is this with one machine or your dual core using 1 core each to keep poner on. Thanks again.

Is this serious?

OK, anyhow:

1. Of course one thread for all Engines!
2. XP64, but as long as all engines have the same OS does this really matter?

And the last Question about "one machine or your dual core using 1 core each to keep poner on" I have to ask again about the relevance?
But ok, it was 3 x Q6600 Quad machines running Shredder Classic GUI two times on each machine, so 6 ponder on games running at the same time. Then a network drive where all the tournament data is stored and all 6 GUIs where playing on the same tournament simultaniously. (Try that with CB ) Thats why just UCI Engines take part.

Bye
Ingo

[swami]

5 wins and 57 losses against Rybka 3? Still a long way to go for Naum, but yes it's improved a great deal.

[IWB]

Hello

5 wins and 57 losses against Rybka 3? Still a long way to go for Naum...

I support the conclusion, but I doubt that it can be drawn out of just 100 games!

In my list are several engines which lose vs another program but have a better overall rating. So the direct comparision is ... doubtfull.

Bye
Ingo