SSDF Rating List 23-05-24

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Lars Sandin
Posts: 2032
Joined: Sun Nov 18, 2007 5:17 pm
Location: Sundsvall, Sweden

SSDF Rating List 23-05-24

Post by Lars Sandin »

Code: Select all

THE SSDF RATING LIST 2023-05-24*   162991 games played by  435 computers							
                                           Rating   +	  -  Games   Won  Oppo
                                           ------  ---   --- -----   ---  ----
   1 Lc0 0.29.0 Cuda-808544 3060Ti           3610   45   -41   287   68%  3466
   2 Lc0 0.28.2 Cuda-611213 3060Ti           3571   26   -25   780   66%  3456
   3 Lc0 0.26.3 Cuda(67362) 3060Ti           3570   29   -27   680   68%  3441
   4 Stockfish 15 x64 1800X 3.6 GHz          3570   39   -36   367   66%  3456
   5 Stockfish 13 x64 1800X 3.6 GHz          3568   35   -32   480   69%  3432
   6 Dragon Komodo 3 x64 1800X 3.6 GHz       3567   40   -38   329   62%  3482
   7 Dragon Komodo 2.51 x64 1800X 3.6 GHz    3565   42   -39   320   65%  3454
   8 Stockfish 14 x64 1800X 3.6 GHz          3555   37   -35   400   65%  3452
   9 Stockfish 12 NNUE x64 1800X 3.6 GHz     3549   26   -25   760   60%  3479
  10 Rebel 16.2 x64 1800X 3.6 GHz            3544   48   -47   209   53%  3522
  11 Dragon by Komodo x64 1800X 3.6 GHz      3536   34   -32   460   63%  3443
  12 Stockfish 11 x64 1800X 3.6 GHz          3529   36   -34   450   70%  3386
  13 Arasan 23.4 x64 1800X 3.6 GHz           3524   54   -55   160   47%  3560
  14 Dragon Komodo 3.1 MCTS 1800X 3.6 GHz    3514   34   -33   440   60%  3447
  15 Stockfish 10 x64 1800X 3.6 GHz          3501   25   -23   920   67%  3378
  16 Booot 7 x64 1800X 3.6 GHz               3477   39   -40   300   46%  3507
  17 Dragon Komodo 2 MCTS x64 1800X 3.6 GHz  3473   45   -44   240   55%  3439
  18 Stockfish 9 x64 1800X 3.6 GHz           3464   24   -23   962   67%  3345
  19 Komodo 14 x64 1800X 3.6 GHz             3455   30   -30   520   52%  3441
  20 Arasan 23.01 x64 1800X 3.6 GHz          3449   33   -34   420   48%  3462
  21 Arasan 23.4 x64 Q6600 2.4 GHz           3439   40   -37   361   68%  3314
  22 Pedone 3.1 x64 1800X 3.6 GHz            3433   38   -39   320   45%  3468
  23 Wasp 6.5 x64 1800X 3.6 GHz              3433   44   -45   242   44%  3478
  24 Stockfish 9 x64 Q6600 2.4 GHz           3427   32   -31   480   56%  3384
  25 Booot 6.5 x64 1800X 3.6 GHz             3426   34   -34   400   48%  3436
  26 Wasp 5.5 x64 1800X 3.6 GHz              3378   37   -39   360   36%  3486
  27 Arasan 22.3 x64 1800X 3.6 GHz           3360   39   -42   320   35%  3468
  28 Marvin 6.0.0 x64 1800X 3.6 GHz          3358   47   -53   220   31%  3494
  29 Pedone 3.1 x64 Q6600 2.4 GHZ            3356   38   -37   340   58%  3303
  30 Deep Shredder 13 x64 1800X 3.6 GHz      3347   24   -23   920   62%  3261
  31 Chiron 5.01 x64 1800X 3.6 GHz           3344   33   -35   440   34%  3455
  32 Marvin 5.2.0 x64 1800X 3.6 GHz          3325   37   -40   360   32%  3456
  33 Chiron 5.01 x64 Q6600 2.4 GHz           3318   34   -34   400   52%  3301
  34 Vajolet2 2.8 x64 1800X 3.6 GHz          3279   27   -28   652   38%  3364
  35 Deep Hiarcs 14 1800X 3.6 GHz            3206   23   -24   880   38%  3289
  36 Revelation 2 A.E. Komodo 12.1 A7 1 GHz  3177   73   -68   100   59%  3113
  37 Revelation 2 Hiarcs 14.1 PXA320 800 MHz 2924   47   -45   228   56%  2881
  38 Chessmaster King 3.5 x64 Q6600 2.4 GHz  2857   24   -25   932   30%  3005
  39 Revelation Hiarcs 13.3 PXA255 500 MHz   2772   57   -52   177   66%  2660
  40 Revelation Shredder 12 PXA255 500 MHz   2706   60   -58   140   56%  2666
  41 Revelation Rybka 2.2 PXA255 500 MHz     2635   45   -43   260   63%  2546
  42 Revelation Deep Sjeng 3 PXA255 500 MHz  2600   62   -67   120   41%  2665
  43 Millennium The King Exclusive 300 MHz   2534   49   -48   200   55%  2501
  44 Revelation Ruffian 2.1 PXA255 500 MHz   2346   68   -71   100   45%  2385
  45 Millennium ChessGenius Excl. M7 300 MHz 2249   50   -48   204   58%  2193
  46 Mephisto London 68030 33 MHz            2195   31   -31   482   50%  2192
  47 Millennium ChessGenius Pro M4 120 MHz   2163   59   -54   160   63%  2067
  48 Mephisto London 68020 12 MHz            2089   60   -53   171   68%  1957
  49 Millennium ChessGenius ARM M4 48 MHz    2070   45   -43   251   58%  2015
  50 Mephisto Roma II 68000 10 MHz           1898   63   -59   132   58%  1843

 1 Lc0 0.29.0 Cuda-808544 3060Ti, 3610											
Stoc15 1800X       4-3     Reb162 1800X    21,5-18,5  Aras234 1800X     46-34
Stock9 1800X      29-11    Wasp65 1800X    28,5-11,5  Wasp55 1800X      66-14

 10 Rebel 16.2 x64 1800X 3.6 GHz, 3544											
Lc029 3060Ti    18,5-21,5  Dra3 1800X       4,5-4,5   Stoc14 1800X      19-21
Aras234 1800X   20,5-19,5  Boo7 1800X      23,5-16,5  Wasp65 1800X      24-16

 13 Arasan 23.4 x64 1800X 3.6 GHz, 3524											
Lc029 3060Ti      34-46    Reb162 1800X    19,5-20,5  Boo7 1800X        21-19

 21 Arasan 23.4 x64 Q6600 2.4 GHz, 3439											
Komo12 Q6600    21,5-18,5  Stock8 Q6600    21,5-18,5  Ped31 Q6600     22,5-17,5
Chi5 Q6600      85,5-34,5  DShre13 Q6600     24-16    Vaj28 Q6600        1-0 
Mar51 Q6600     33,5-6,5   DRybka3 Q6600   34,5-5,5   

 23 Wasp 6.5 x64 1800X 3.6 GHz, 3433											
Lc029 3060Ti    11,5-28,5  Stoc15 1800X    11,5-28,5  Reb162 1800X      16-24
Ped3 1800X      44,5-35,5  Komo11 1800X    21,5-18,5  Vaj28 1800X        2-0 

 36 Revelation 2 A.E. Komodo 12.1 A7 1 GHz, 3177											
DShre13 Q6600      6-14    DHiar14 Q6600    9,5-10,5  DJuni12 Q6600   11,5-8,5 
Baro343 Q6600   15,5-4,5   Naum 4  A1200   16,5-3,5   

Since the last rating list we can now present six new entrants in the list. 

First one out is our new leader in the list: Lc0 0.29.0, and also the first 
program that has broken the 3600-barrier in our rating list. More games will 
be needed to lower the error bars, but after the first 297 games it has placed
itself 39 points over the last version tested with a huge rating of 3610! 
It will be interesting to see how it will hold up as more games are played 
against the top programs. 
The Lc0-version which we have tested this time is: Lc0 0.29.0, with the 
19 filter + 512 block network, named 808544 (date: 2023-01-12). As with the
earlier Lc0-versions, we have used the free opening book "perfect2021.abk" 
by Sedat Canbaz for the testing of Lc0.

Our next newcomer in the list are one really strong comeback from the team of
Ed Schroeder and Chris Wittington. Chris Wittington is formely known for 
programs such as: Chess Player 2150/2175, Complete Chess System 
and Chess System Tal (I and II). 
Ed Schroeder is of course known for many of the old dedicated chess computers 
like for instance: Mephisto Polgar, MM IV/V, RISC (I and II), Rebell
and also for versions of his program in the ChessMachine ISA-plugin card for PC.
On the PC-scene he has made the Gideon and Rebel-program famous and lately also 
made several interesting and strong versions of his ProDeo-program.

Since last year they have utilized Neural Net-technology in their joint
effort and has seemingly improved fast. We have waited some time to test it, 
since we wanted it to share most of the functions - like for instance Pondering 
and Multi-Processor functionality, which the others programs that we test on our
latest hardware uses. The version which we have tested is named: Rebel 16.2 and
we have used its own opening book, made by the famous Jeroen Noomen, for the 
testing. After the first 209 games, this strong combo of the program: Rebel 16.2,
and the opening book of Jeroen Noomen, has managed to get an impressive rating 
of 3544 on our 1800X hardware. It is placed 10th in the list at the moment, 
just 5 points shy of the Stockfish 12-program. 
A very strong comeback by Ed, Chris and Jeroen!

Our next addition to the list are Jon Dart's strong and reliable Arasan-program. 
The version that we have tested on our 1800X and Q6600 hardware are Arasan 23.4.
On our 1800X hardware it has reached a rating of 3524 after the initial 160 
games played. That is at the moment 75 points ahead of the formerly tested 23.01
version and a very nice improvement indeed! More games will of course be needed 
to lower the error bars a bit. 
On our Q6600 hardware, Arasan 23.4 has reached a rating of 3439 after 361 games 
played. We never tested the 23.01 version on the Q6600 hardware, but Arasan 23.4 
are a whopping 202 points ahead of the 21.2 version at least! 
We have, as always, used Arasan's own opening book for the testing.                   

We are also glad to be able to present a new strong Wasp version by John Stanback, 
namely Wasp 6.5, on our latest rating list. After the first 242 games played, it 
has reached a rating of 3433! This is 55 points stronger than the 5.5 version 
which we tested before, so a nice improvement there also! We have used Wasp's own
opening book for the testing. 

And we can now finally show the first rating of the manually tested:
Phoenix Revelation II Anniversary Edition chess computer by Ruud Martin/DGT. This
Anniversary Edition of the Revelation II, features an updated hardware relative
to the ordinary Revelation II. The hardware for the Anniversary Edition is an 
Colibri iMX7D with 2 ARM Cortex A7 processors running at 1GHz. The software which 
we have used for the testing is the program Komodo 12.1 by Mark Lefler and
Larry Kaufman. 
The rating that has been reached after the first 100 games is 3177! This is a 
rating that is 253 points ahead of Hiarcs 14.1, which we tested formerly
on the ordinary Revelation II. It is clearly the strongest dedicated chess 
computer which we have tested so far and it has been able to perform admirably 
against the strong Q6600 opposition which it mostly has faced in our testing 
so far. 

Aside from producing more games with the already mentioned newcomers in the
rating list, we are at the moment testing Richard Pijl's - The Baron 3.45, 
and also Alex Morozov's - Booot 7.1. We also hope to have some more programs
ready for the next rating list.  

* This rating list was produced and based on the games/results that were finished 
on the first of May, but the publication of this rating list was unfortunately a 
bit delayed this time.
The testing has of course not stood still since then, and the games/results 
that has been played since this date will of course feature in the next rating 
list, which we plan to have out sometime in august/september.

Lars Sandin
Lars Sandin, SSDF
ernest
Posts: 2042
Joined: Wed Mar 08, 2006 8:30 pm

Re: SSDF Rating List 23-05-24

Post by ernest »

In order to compare Lc0 and Stockfish hardware (for "fairness"...), what is your "Leela ratio "
or, more simply, their kn/s around the start of games ?
Lars Sandin
Posts: 2032
Joined: Sun Nov 18, 2007 5:17 pm
Location: Sundsvall, Sweden

Re: SSDF Rating List 23-05-24

Post by Lars Sandin »

ernest wrote: Thu May 25, 2023 5:18 am In order to compare Lc0 and Stockfish hardware (for "fairness"...), what is your "Leela ratio "
or, more simply, their kn/s around the start of games ?
It is hard to compare kn/s and depths between Lc0 and SF, as they are using entirely different techniques. The 3060Ti machine is currently playing in an endgame (169 kN/s) with Booot 7.1 (that has 14.1 MN/s). The depths reached are even harder to compare, as they are extremely different in their representation. Both use a 1800X CPU as a base, but the machine with the 3060Ti graphic card are only using 2 threads on the CPU (relatively unimportant as most of it's power comes from the GPU) and Booot are of course using all 8 CPU-cores on the 1800X.
Lars Sandin, SSDF