CCRL engines with more than 500 games

[Uri Blass]

list of popular engines at 40/40(at least 500 games)

comments:
1)for engines 3,10
I am not sure what is the version of 2.1 that is rated and the blitz list has 2.1o and has no 2.1 with no additions

2)11 was not tested by CCRL blitz and only 32 bit was tested but even for 15 I am not sure which version of 1.2is it and the blitz list has 1.2n and 1.2f

3)I did not look at all this list to find which program has the bigger difference between blitz and long time control(I will try to do comparison later) but usually the difference is small.



1 Rybka 2.3.2a 64-bit 4CPU 3117 +23 −22 72.1% −148.3 39.0% 689
63.9%
2 Rybka 2.3.2a 64-bit 2CPU 3096 +23 −22 77.2% −187.8 36.5% 727
66.3%
3 Rybka 2.1 64-bit 2CPU 3076 +26 −25 73.4% −167.6 34.3% 571
60.5%
4 Rybka 2.3.2a 64-bit 3071 +18 −18 75.3% −179.3 34.8% 1195
73.5%
5 Zappa Mexico 64-bit 4CPU 3062 +21 −21 64.6% −95.4 43.4% 721
64.6%
6 Rybka 2.2 64-bit 2CPU 3056 +24 −23 72.9% −164.4 34.8% 661
54.4%
7 Zap!Chess Zanzibar 64-bit 4CPU 3054 +22 −22 64.6% −91.2 46.4% 679
87.1%
8 Rybka 2.3.2a 32-bit 3016 +22 −21 74.9% −169.9 36.6% 774
66.5%
9 Naum 2.2 64-bit 4CPU 3009 +21 −21 55.6% −34.5 47.4% 665
63.6%
10 Rybka 2.1 32-bit 2CPU 3003 +26 −26 70.8% −141.9 41.1% 508
56.9%
11 Rybka 1.2 64-bit 2993 +21 −21 74.2% −171.7 33.0% 851
63.6%
12 Hiarcs 11.1 4CPU 2988 +19 −19 52.7% −18.0 43.7% 882
53.8%
13 Rybka 2.2 32-bit 2987 +18 −17 70.4% −140.3 36.6% 1150
52.0%
14 Rybka 1.1 64-bit 2985 +24 −24 76.1% −185.0 34.6% 665
54.4%
15 Rybka 1.2 32-bit 2976 +18 −18 70.1% −144.9 35.1% 1125
64.0%
16 Naum 2.1 64-bit 4CPU 2971 +21 −21 53.8% −24.4 44.7% 700
50.9%
17 Rybka 2.1 32-bit 2970 +23 −23 69.5% −130.9 40.3% 645
56.3%
18 Rybka 1.1 32-bit 2959 +18 −17 69.7% −137.7 38.1% 1161
54.9%
19 Loop M1-T 64-bit 4CPU 2955 +18 −18 49.1% +5.7 45.9% 923
51.3%
20 Zap!Chess Paderborn 64-bit 4CPU 2954 +24 −24 65.2% −101.5 43.7% 558
56.7%
21 Hiarcs 11.1 2CPU 2944 +20 −20 56.8% −45.0 42.2% 793
48.5%
22 Loop 13.6 64-bit 4CPU 2944 +25 −24 51.9% −10.0 42.3% 527
57.3%
23 Deep Shredder 10 64-bit 4CPU 2936 +17 −17 50.5% −3.1 39.9% 1133
51.0%
24 Deep Fritz 10 4CPU 2931 +19 −19 48.8% +8.3 40.5% 881
51.4%
25 LoopMP 12.32 2CPU 2930 +20 −20 50.4% −2.4 42.6% 753
53.4%
26 Naum 2.1 64-bit 2CPU 2925 +23 −23 49.1% +5.8 43.1% 576
56.4%
27 Deep Shredder 10 32-bit 2CPU 2920 +19 −19 57.2% −48.0 37.7% 951
48.9%
28 Rybka 1.0 64-bit 2920 +16 −16 68.7% −126.9 35.5% 1415
63.0%
29 Loop 13.5 64-bit 2CPU 2915 +24 −24 55.9% −37.7 36.9% 574
52.7%
30 Deep Junior 10 4CPU 2914 +18 −18 47.0% +21.9 35.9% 1061
49.1%
31 Deep Junior 10 2CPU 2899 +21 −21 53.4% −26.8 36.6% 727
51.3%
32 Toga II 1.3.1 2897 +13 −13 50.3% −3.4 42.2% 1997
55.5%
33 Zappa Mexico 32-bit 2889 +24 −24 56.2% −39.1 42.0% 526
53.3%
34 Hiarcs 11.1 2888 +17 −17 53.0% −20.3 43.1% 1160
52.3%
35 Naum 2.2 32-bit 2887 +20 −20 53.5% −20.1 49.2% 731
59.0%
36 Rybka 1.0 32-bit 2884 +16 −16 60.9% −73.4 40.3% 1279
51.1%
37 Zap!Chess Zanzibar 32-bit 2883 +22 −22 57.1% −47.0 43.6% 645
58.1%
38 Fritz 10 2881 +16 −16 51.4% −9.8 36.5% 1401
48.9%
39 Loop 13.6 32-bit 2880 +16 −16 53.5% −21.4 44.2% 1220
55.4%
40 Shredder 10 2872 +13 −13 56.9% −45.8 37.8% 2023
61.5%
41 Glaurung 1.2.1 64-bit 4CPU 2867 +20 −20 38.7% +74.6 38.9% 858
49.8%
42 Loop 10.32f 2867 +18 −18 50.7% −3.2 41.4% 961
53.1%
43 Strelka 1.8 2865 +21 −21 44.7% +34.2 41.4% 752
51.9%
44 Toga II 1.2.1a 2861 +11 −11 48.3% +9.1 39.3% 2877
52.1%
45 Glaurung 2 epsilon/5 64-bit 2CPU 2860 +23 −23 39.0% +76.0 38.1% 606
54.2%
46 Spike 1.2 Turin 2854 +10 −10 47.4% +15.5 41.3% 3713
54.4%
47 Deep Shredder 9 2CPU 2853 +18 −18 57.0% −45.7 36.4% 1081
49.9%
48 Deep Sjeng 2.7 1CPU 2847 +19 −19 48.3% +12.1 41.8% 880
50.1%
49 Hiarcs X50 2847 +17 −17 45.9% +30.0 36.1% 1162
51.9%
50 Junior 10 2845 +15 −15 49.0% +6.6 36.0% 1484
51.3%
51 Naum 2.1 32-bit 2843 +19 −19 51.3% −10.2 39.5% 870
57.6%
52 Fritz 9 2841 +13 −13 49.4% +3.8 34.5% 2161
49.9%
53 Fruit 2.2.1 2840 +13 −13 52.2% −14.3 40.9% 2088
65.6%
54 Hiarcs 10 2836 +15 −15 49.5% +4.8 42.4% 1544
66.2%
55 Deep Fritz 8 2CPU 2831 +19 −19 50.8% −4.4 34.0% 908
55.2%
56 Toga II 1.1a 2825 +18 −18 55.4% −34.7 36.9% 1036
59.4%
57 Deep Junior 9 2CPU 2823 +18 −18 52.6% −20.6 34.1% 1105
47.8%
58 Shredder 9 2823 +17 −17 55.7% −38.2 37.3% 1162
56.9%
59 Junior 10.1 2820 +26 −26 45.2% +32.5 32.5% 511
52.2%
60 Glaurung 1.2.1 64-bit 2CPU 2818 +24 −24 38.1% +85.1 34.9% 599
51.0%
61 Hiarcs 10 Hypermodern 2818 +19 −19 51.7% −8.7 42.0% 889
50.0%
62 Ktulu 8.0 2804 +13 −13 40.6% +62.2 37.7% 2110
54.8%
63 Naum 2.0 64-bit 2802 +22 −23 50.1% +2.5 40.4% 633
54.9%
64 Naum 2.0 32-bit 2801 +18 −18 51.2% −6.5 38.8% 990
50.6%
65 Chess Tiger 2007 2793 +16 −16 42.4% +52.0 39.9% 1353
52.7%
66 Chess Tiger 2007.1 2790 +15 −15 43.6% +43.6 41.4% 1391
54.8%
67 Spike 1.1 2785 +15 −15 48.4% +10.0 39.8% 1527
61.2%
68 Zap!Chess Paderborn 32-bit 2781 +21 −21 45.2% +34.5 39.2% 732
56.4%
69 Junior 9 2779 +16 −16 46.6% +22.7 36.7% 1236
52.6%
70 Glaurung 1.2.1 64-bit 2770 +23 −23 46.0% +27.4 36.4% 640
51.0%
71 Ktulu 7.5 2763 +15 −15 42.9% +48.0 35.2% 1421
47.6%
72 Scorpio 1.91 2763 +21 −21 46.3% +22.8 38.7% 754
56.9%
73 CM10th Xperience 2761 +14 −14 41.4% +58.5 37.6% 1675
57.3%
74 Glaurung 1.2.1 32-bit 2758 +16 −16 44.4% +39.2 33.1% 1345
53.8%
75 CM10th Paralyse 2757 +18 −18 41.4% +56.9 36.3% 1049
53.9%
76 Smarthink 1.00 32-bit 2756 +15 −15 43.4% +43.6 38.8% 1451
52.9%
77 Alaric 707 2752 +23 −23 44.1% +42.5 36.4% 645
50.1%
78 Spike 1.0a 2752 +19 −19 43.9% +41.1 40.0% 871
56.8%
79 Movei 0.08.438 2747 +22 −22 53.5% −22.0 37.4% 693
52.4%
80 CM10th Magic II 2746 +17 −17 41.3% +57.0 43.7% 1096
53.6%
81 Scorpio 1.8 2744 +20 −20 47.4% +17.9 35.6% 860
64.3%
82 Slow Chess Blitz WV2.1 2740 +15 −15 48.8% +7.8 37.7% 1531
55.4%
83 Ktulu 7.1 2738 +19 −19 40.9% +63.0 35.6% 949
51.3%
84 Ruffian 2.1.0 2735 +23 −23 46.4% +28.7 35.9% 640
51.0%
85 CM10th Default 2733 +16 −16 41.3% +59.5 37.3% 1326
50.8%
86 Chess Tiger 15 2733 +17 −17 41.3% +60.0 40.2% 1129
51.6%
87 Zappa 1.1 64-bit 2732 +24 −24 45.3% +33.4 37.8% 545
53.7%
88 WildCat 7 2730 +17 −17 46.5% +24.7 37.7% 1158
59.4%
89 Gandalf 6 2724 +15 −15 40.5% +65.4 37.2% 1490
51.0%
90 Glaurung 1.0.2 32-bit 2724 +18 −18 42.3% +55.8 31.8% 1072
48.9%
91 Pro Deo 1.2 2722 +19 −19 48.5% +7.0 29.8% 1005
58.0%
92 List 5.12 2719 +19 −19 46.4% +24.4 37.2% 879
55.8%
93 Hiarcs 9 2717 +21 −21 37.8% +82.1 36.6% 784
53.3%
94 Chess Tiger 2004 2716 +22 −22 40.4% +62.0 45.5% 653
54.9%
95 Ruffian 1.0.5 2710 +20 −20 46.0% +25.9 41.5% 803
49.2%
96 Slow Chess Blitz WV2 2710 +19 −19 40.1% +69.8 34.1% 950
55.4%
97 Naum 1.91 64-bit 2709 +19 −19 40.0% +68.5 38.3% 972
51.8%
98 Pharaon 3.5.1 2708 +17 −17 45.8% +29.6 33.2% 1195
64.8%
99 Naum 1.91 32-bit 2703 +18 −18 39.3% +72.3 37.1% 1042
53.1%
100 Delfi 5.0 2702 +20 −21 47.5% +15.6 36.8% 774
53.9%
101 Aristarch 4.50 2698 +15 −15 44.0% +43.4 34.3% 1529
54.1%
102 WildCat 6 2696 +22 −22 47.1% +18.9 37.5% 680
52.9%
103 Pseudo 0.7c 2691 +19 −19 49.8% +0.2 38.7% 860
51.7%
104 Jonny 2.83 32-bit 2678 +22 −22 47.8% +13.6 30.9% 712
49.5%
105 SOS 5.1 2661 +21 −21 45.3% +29.8 36.0% 744
53.2%
106 Colossus 2006f 2660 +23 −23 47.4% +17.0 31.5% 660
52.6%
107 Little Goliath Evolution 3.12 2612 +25 −25 42.4% +51.5 32.1% 532
53.7%
108 AnMon 5.60 2610 +22 −22 36.3% +96.3 29.3% 743
50.4%

[PauloSoare]

I wonder about the few elo diference betewen 4 and 2 cores.

1 Rybka 2.3.2a 64-bit 4CPU 3117 +23 −22 72.1% −148.3 39.0% 689
63.9%
2 Rybka 2.3.2a 64-bit 2CPU 3096 +23 −22 77.2% −187.8 36.5% 727
66.3%

I don´t remember Vas talking about that on Rybka Forum.

Paulo Soares

[Spock]

I don´t remember Vas talking about that on Rybka Forum.

Paulo Soares

It was discussed at length on the Rybka forum at the time. Some people had the view that Rybka's multi-cpu scaling/performance had got significantly worse since 2.1

[Uri Blass]

Comparing the data with the ccrl blitz show that movei earns the most from long time control out of engines seriously tested at least 500 games in both lists.

I am surprised because Movei was not designed to earn from long time control and I consider my move ordering as poor move ordering and movei even does not use internal iterative deepening.

There are good tricks for better move ordering that are not used in movei and I wll probably try to use them after I rewrite movei.

Uri

[Shaun]

Hi Uri,

in the 40/40 there was some combinations of 1.2 and 2.1 Rybka versions following a recomendation from Vas.

In the 40/4 I kept them seperate.

Shaun

P.S. check out this link I will add the latest 40/40 list for selection in the next update

[Uri Blass]

Hi Uri,

in the 40/40 there was some combinations of 1.2 and 2.1 Rybka versions following a recomendation from Vas.

In the 40/4 I kept them seperate.

Shaun

P.S. check out this link I will add the latest 40/40 list for selection in the next update

I do not understand the numbers after the rating

I see

1 Rybka 2.3.2a 64-bit 4CPU 3137 +20

I thought it may be difference between CCRL 40/40 and CCRL 40/4 because we have
1 Rybka 2.3.2a 64-bit 4CPU 3117
but it is clearly not the case because

when I compare movei rating I have

in
http://computerchess.org.uk/ccrl/4040/c ... librate=no


22 Movei 0.08.438 2747 +63 +22 −22 53.5% −22.0 37.4% 693 +680
52.4%
against the link that you post

22 Movei 0.08.438(10 10 10) 2731 +18 −18 51.6% −11.8 32.7% 1031
54.0%
Movei 0.08.438 2702 +18 +25 −25 44.2% +39.2 29.4% 540 +527
55.4%

18 elo is not the right number for improvement in movei.
It is 45 elo for the default version and the question what is the improvement for the 10 10 10 personality is not tested.

Uri

[Shaun]

Hi Uri,

it is the rating difference but the 40/40 list compared was an old one...

I will add the latest list for the next blitz update (tomorrow?)

Shaun

[Kirill Kryukov]

I do not understand the numbers after the rating

I see

1 Rybka 2.3.2a 64-bit 4CPU 3137 +20

In the header of that rating list there is:

"Comparing with reference list CCRL-4040.2007-09-03 (no recalibration)"

The "+20" is the rating difference compared to the reference list. Reference list can be selected at the bottom of the page.

Best,
Kirill

[Uri Blass]

Hi Uri,

it is the rating difference but the 40/40 list compared was an old one...

I will add the latest list for the next blitz update (tomorrow?)

Shaun

I tried the following link that has reference list from 09.11.2007 and I see no number for Movei 00.8.438

Maybe the reason is that Movei is written as Movei 0.08.438 in the 40/40 list


http://computerchess.org.uk/ccrl/404/cg ... librate=no

[Uri Blass]

Here are the list of programs with more than 40 elo difference if you do not include movei and do not include programs with less than 100 games
in one of the lists.

Good blitzers:

Colossus 2007b 2674 +98 +20 −20 47.9% +13.8 30.5% 858 +660
Hiarcs 11.2 4CPU 3000 +50 +17 −17 55.0% −32.4 39.6% 1118 +851
Loop 10.32f 2910 +43 +21 −21 59.5% −73.2 31.7% 777 −184


Good long time control programs
The Baron 1.7.0 2560 −89 +16 −16 47.0% +17.8 23.2% 1631 +1330
Booot 4.10.1 2502 −82 +41 −43 26.8% +168.0 25.7% 222 +39
Twisted Logic 0.065e37 2544 −80 +31 −32 34.6% +115.8 23.4% 380 +200
Hamsters 0.5 2584 −74 +19 −19 47.5% +15.7 27.2% 980 +793
Deep Sjeng 2.5 1CPU 2773 −67 +28 −27 54.0% −27.5 34.5% 435 +205
Quark 2.35 2515 −64 +30 −30 47.2% +21.8 24.1% 395 +77
Frenzee 3.0 2602 −63 +24 −24 48.0% +15.0 27.1% 616 +178
Arasan 9.5 2498 −61 +27 −27 50.8% −9.1 23.2% 535 +281
Muse 0.899b 2465 −59 +38 −38 57.5% −62.7 26.8% 254 −17
Delfi 5.2 2693 −55 +22 −23 36.9% +93.6 28.5% 710 +471
Hamsters 0.4 2556 −54 +26 −27 42.3% +55.9 28.9% 506 +280
Hermann 2.0 2473 −53 +21 −21 35.0% +118.8 21.6% 928 +800
Booot 4.13.0 2582 −51 +34 −35 39.1% +74.4 31.2% 285 +45
39 The Baron 1.8.1 2649 −50 +16 −16 48.8% +9.2 24.5% 1462 +1122
Zappa 1.1 32-bit 2637 −50 +25 −25 43.2% +48.9 29.4% 575 +251
Comet B68 2480 −50 +21 −21 37.0% +92.4 26.3% 853 +636
50.6%
Matacz 1.1 2480 −49 +41 −41 55.7% −55.4 22.5% 227 −5

Petir 3.99d 2621 −48 +17 −17 49.8% −7.4 28.9% 1254 +894
Gosu 0.16 2473 −48 +37 −37 56.8% −61.5 23.3% 279 +10
Petir 3.95 2602 −47 +22 −22 41.3% +63.0 27.1% 768 +516
Colossus 2006d 2585 −43 +29 −30 38.4% +86.5 26.4% 425 +195
Twisted Logic 0.099x 2586 −42 +40 −41 36.1% +98.3 29.2% 219 +1
51.5%
SpiderChess 070603 2558 −42 +21 −22 44.1% +39.6 29.5% 793 +558

I stopped at 2465 and it seems that something is wrong at the rating of weak programs because I cannot believe that they are stronger at long time control.

Uri