THE SSDF RATING LIST 2022-12-31

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Lars Sandin
Posts: 2032
Joined: Sun Nov 18, 2007 5:17 pm
Location: Sundsvall, Sweden

THE SSDF RATING LIST 2022-12-31

Post by Lars Sandin »

Code: Select all

THE SSDF RATING LIST 2022-12-31    161668 games played by  429 computers							
                                           Rating   +	  -  Games   Won  Oppo
                                           ------  ---   --- -----   ---  ----
   1 Lc0 0.28.2 Cuda-611213 3060Ti           3575   27   -25   772   66%  3447
   2 Stockfish 13 x64 1800X 3.6 GHz          3574   35   -32   480   69%  3436
   3 Stockfish 15 x64 1800X 3.6 GHz          3574   42   -39   320   66%  3458
   4 Lc0 0.26.3 Cuda(67362) 3060Ti           3571   29   -27   680   68%  3444
   5 Dragon Komodo 3 x64 1800X 3.6 GHz       3571   43   -41   285   62%  3487
   6 Dragon Komodo 2.51 x64 1800X 3.6 GHz    3566   42   -39   320   65%  3457
   7 Stockfish 14 x64 1800X 3.6 GHz          3557   39   -37   360   66%  3445
   8 Stockfish 12 NNUE x64 1800X 3.6 GHz     3554   28   -27   658   61%  3464
   9 Dragon by Komodo x64 1800X 3.6 GHz      3542   34   -32   460   63%  3448
  10 Stockfish 11 x64 1800X 3.6 GHz          3535   36   -34   450   70%  3391
  11 Dragon Komodo 3.1 MCTS 1800X 3.6 GHz    3519   42   -39   306   63%  3430
  12 Stockfish 10 x64 1800X 3.6 GHz          3510   25   -23   920   67%  3375
  13 Dragon Komodo 2 MCTS x64 1800X 3.6 GHz  3478   45   -44   240   55%  3443
  14 Stockfish 9 x64 1800X 3.6 GHz           3474   25   -24   922   68%  3329
  15 Booot 7 x64 1800X 3.6 GHz               3471   50   -51   185   48%  3486
  16 Komodo 13.1 x64 1800X 3.6 GHz           3463   29   -28   600   60%  3152
  17 Komodo 14 x64 1800X 3.6 GHz             3461   30   -30   520   52%  3445
  18 Komodo 13.02 x64 1800X 3.6 GHz          3455   30   -29   600   65%  3344
  19 Arasan 23.01 x64 1800X 3.6 GHz          3452   33   -34   420   48%  3464
  20 Stockfish 9 x64 Q6600 2.4 GHz           3437   32   -31   480   56%  3372
  21 Pedone 3.1 x64 1800X 3.6 GHz            3433   38   -39   320   45%  3470
  22 Booot 6.5 x64 1800X 3.6 GHz             3429   34   -34   400   48%  3440
  23 Dragon Komodo MCTS x64 1800X 3.6 GHz    3401   38   -38   320   52%  3389
  24 Wasp 5.5 x64 1800X 3.6 GHz              3391   41   -43   280   41%  3455
  25 Wasp 5 x64 1800X 3.6 GHz                3379   41   -44   280   38%  3468
  26 Marvin 6.0.0 x64 1800X 3.6 GHz          3365   47   -53   220   31%  3499
  27 Arasan 22.3 x64 1800X 3.6 GHz           3363   39   -42   320   35%  3471
  28 Pedone 3.1 x64 Q6600 2.4 GHZ            3356   41   -39   300   60%  3288
  29 Deep Shredder 13 x64 1800X 3.6 GHz      3351   24   -23   920   62%  3265
  30 Chiron 5.01 x64 1800X 3.6 GHz           3347   33   -35   440   34%  3458
  31 Chiron 5.01 x64 Q6600 2.4 GHz           3328   43   -41   280   62%  3243
  32 Marvin 5.2.0 x64 1800X 3.6 GHz          3328   37   -40   360   32%  3459
  33 Wasp 4.5 x64 1800X 3.6 GHz              3303   43   -47   280   30%  3453
  34 Vajolet2 2.8 x64 1800X 3.6 GHz          3287   27   -28   650   38%  3370
  35 Deep Hiarcs 14 1800X 3.6 GHz            3210   23   -24   880   38%  3283
  36 Deep Rybka 4 x64 Q6600 2.4 GHz          3193   18   -18  1528   62%  3106
  37 Revelation 2 Hiarcs 14.1 PXA320 800 MHz 2924   47   -45   228   56%  2881
  38 Chessmaster King 3.5 x64 Q6600 2.4 GHz  2857   24   -25   932   30%  3006
  39 Revelation Hiarcs 13.3 PXA255 500 MHz   2772   57   -52   177   66%  2660
  40 Revelation Shredder 12 PXA255 500 MHz   2706   60   -58   140   56%  2666
  41 Revelation Rybka 2.2 PXA255 500 MHz     2635   45   -43   260   63%  2546
  42 Revelation Deep Sjeng 3 PXA255 500 MHz  2600   62   -67   120   41%  2664
  43 Millennium The King Exclusive 300 MHz   2534   51   -49   190   55%  2499
  44 Revelation Ruffian 2.1 PXA255 500 MHz   2346   68   -71   100   45%  2385
  45 Millennium ChessGenius Excl. M7 300 MHz 2245   50   -48   200   57%  2194
  46 Mephisto London 68030 33 MHz            2195   31   -31   482   50%  2192
  47 Millennium ChessGenius Pro M4 120 MHz   2162   59   -54   160   63%  2066
  48 Mephisto London 68020 12 MHz            2089   60   -53   171   68%  1955
  49 Millennium ChessGenius ARM M4 48 MHz    2069   45   -43   251   58%  2015
  50 Mephisto Roma II 68000 10 MHz           1891   65   -63   119   54%  1862

 3 Stockfish 15 x64 1800X 3.6 GHz, 3574											
Lc028 3060Ti    20,5-19,5  Dra3 1800X      20,5-19,5  SF12NU 1800X      20-20
Boo7 1800X      25,5-14,5  Arasa23 1800X     25-15    Mar60 1800X       32-8 
Chi5 1800X      33,5-6,5   Mar52 1800X     33,5-6,5   

 5 Dragon Komodo 3 x64 1800X 3.6 GHz, 3571											
Lc028 3060Ti    19,5-20,5  Stoc13 1800X    19,5-20,5  Stoc15 1800X    19,5-20,5
Stoc10 1800X      24-16    Boo7 1800X       3,5-1,5   Ped31 1800X     27,5-12,5
Wasp55 1800X    29,5-10,5  DShre13 1800X   33,5-6,5   

 11 Dragon Komodo 3.1 MCTS 1800X 3.6 GHz, 3519											
Lc028 3060Ti      17-23    SF12NU 1800X    12,5-13,5  Ko14 1800X        22-18
Ped31 1800X       36-24    Mar60 1800X       42-18    Chi5 1800X        62-18

 15 Booot 7 x64 1800X 3.6 GHz, 3471											
Lc028 3060Ti    31,5-48,5  Stoc15 1800X    14,5-25,5  Dra3 1800X       1,5-3,5 
Wasp45 1800X      41-19    

 26 Marvin 6.0.0 x64 1800X 3.6 GHz, 3365											
Lc028 3060Ti       6-34    Stoc15 1800X       8-32    Dra31M 1800X      18-42
Boo65 1800X     16,5-23,5  Wasp55 1800X    19,5-20,5

In this last rating list of the year 2022, we can present five new entries. 
First one out is the latest version from team Stockfish, namely Stockfish 15. 
This version landed just a hairline from the number one spot this time and 
grabbed the third position, but only one single point behind the leader Lc0! 
After 480 played games it has earned a rating of 3574, which is 17 points ahead 
of Stockfish 14 and exactly on par with Stockfish 13 at the moment. As opening
book for Stockfish 14, we have used Fauzi Dabat's "Aggressive 5.4 by Fauzi.abk".

Next one out is Dragon Komodo 3 by Mark Lefler and Larry Kaufman. We have tested
both the "normal" version and the Monte Carlo Tree Search-version. The non-MCTS
version of Dragon Komodo 3 earned a fifth place in this rating list, with a
rating of 3571 after 285 games played. It has positioned itself five points 
ahead of the 2.51 version and just four points behind the number one spot at 
the moment!

In the testing of the Monte Carlo Tree Search (MCTS) Dragon Komodo, we have 
used the 3.1 version. It has recieved a rating of 3519 after 306 games
played. It is 41 points stronger than the Dragon Komodo 2 MCTS and took the 
eleventh spot on this rating list. As opening book for both the ordinary and 
the MCTS-version, we have used Erdogan Günes opening book "out10-35.bin".
The sharp-sighted has probably noticed that the nomenclature "x64" is omitted
from the name this time. We can insure that the testing has been done with the
strongest settings, AVX2 and also 64-bit, and that the omission is made just 
because of the length limitations in the naming of the engines. 

I can also add that since most engines that we have tested on both the Q6600 
and 1800X hardware has been 64-bit programs, we will soon make an overhaul to
upcoming rating lists and strip this rather unnecessary designation from the 
names - only adding 32 bit to the names of the few engines that hasn't been 
64-bit compatible.

Well, to get back on track. The next program that we can present in our latest 
rating list, is Alex Morozov's - Booot 7. This time he has used his own 
implementation of Neural Net technology (NN) in evaluation, using Python, 
Delphi and SIMD. This has proved to been useful to the already strong Booot-
program and he has managed to get a rating of 3471 after the first 185 played 
games. This is 42 points ahead of the "ordinary evaluation"-version of 
Booot 6.5 and it now holds the first spot of the engines outside the 
three in the top: Lc0, Stockfish and Dragon Komodo. 
As opening book for Booot 7, we have used the "Perfect2021.abk" opening book 
by Sedat Canbaz. 

Last one out in this rating list is Martin Danielsson with his Marvin-program.
We have tested the 6.0.0-version this time, and it has recieved a rating 
of 3365 after 220 played games. It is 37 points stronger than the last Marvin 
version (5.2.0) which we have tested and we have used Marvin's own opening book 
for the testing.

Since the last rating list, Wasp 4.5 1800X gained 14 points, Pedone 3.1 1800X
gained 10 and Arasan 23.01 1800X gained 9 points. Biggest loss since the last
list was Lc0 0.28.2 which dropped 16 points in the lead. Other notable losses
was Roma II which lost 13 points and Chiron 5.01 1800X, which lost 8 points.  

The next rating list will probably be released relatively soon and will feature
the manually tested Revelation II A.E. Komodo 12.1, and hopefully also contain
some more engines on both the 1800X and Q6600 hardware level. We will also 
begin testing of a more recent version of Lc0 soon, so hopefully it will have 
enough games to be presented as an early bird in the next rating list.
 
Lars Sandin       
Lars Sandin, SSDF
User avatar
Dariusz
Posts: 379
Joined: Sat Jun 13, 2015 10:08 am
Location: Poland
Full name: Dariusz Domagała

Re: THE SSDF RATING LIST 2022-12-31

Post by Dariusz »

Thank you :)
Regards, Darius
https://chessengeria.eu
mehmet123
Posts: 692
Joined: Sun Jan 26, 2020 10:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: THE SSDF RATING LIST 2022-12-31

Post by mehmet123 »

If Lc0 0.30dev 784968 was used in this test, the difference in elo between Lc0 and Stockfish could be greater.
In this rating list, Stockfish 13 and Stockfish 15 seem to have the same rating, I think this is due to the low number of tested games. I don't think there is a big difference between these two engines in long time games, maybe 10 - 15 elo difference.
In this rating list Dragan Komodo 3 has an elo rating close to Stockfish 15. Dragon Komodo is very close to Stockfish in long-time games and multiprocessor tests.
Dragon Komodo performs very well, especially in tests with 4-16 cpus. Stockfish performs much better in tests with 1-2 and 32-128 cpus
Jouni
Posts: 3713
Joined: Wed Mar 08, 2006 8:15 pm
Full name: Jouni Uski

Re: THE SSDF RATING LIST 2022-12-31

Post by Jouni »

This is more like book test. Aggressive 5.4 by Fauzi.abk for Stockfish, out10-35.bin for Komodo and Perfect2021.abk for Lc0 :) .
Jouni