My Final Testsuite

[wolfman]

My Final Testsuite.

I composed out of a lot of known testsuites my final testsuite (FTS). It consists of 3000 positions.
Clean-up 1.4 used to get rid of duplicates.

FTS

Compu's : Intel i5 M430 @ 2,27 Ghz and AMD QL60 @ 1,9 Ghz.
Engines using 1 cpu.
I let engines run for 10 seconds ( Intel i5) or 15 seconds ( AMD QL60) to find the matching move.
% MM : percentage of matching moves.
AT : average time in seconds to find matching moves.
AD : average deep to find matching moves.
RT : rated time.
TSR : Testsuite rating. This figure is based on a correlation between testsuites results and the CCRL 40/4 rating.
TSR will be recalculated as more results will be obtained.
i : Intel i5
a : AMD QL60

Results sofar.

Rank // Engine // Version // // %MM // AD // AT // TSR
1 // Houdini // 3 64 // i // 40,13 // 12,79 // 1,83 // 3188
2 // Critter // 1.6a 64 // i // 41,33 // 12,19 // 1,77 // 3187
3 // Strelka // 5.5.1 64 // i // 40,60 // 12,55 // 1,86 // 3176
4 // Fritz // 13 32 // a // 27,00 // 13,24 // 2,16 // 2968
5 // Shredder // 12 32 // a // 26,97 // 10,35 // 2,55 // 2959
6 // Fritz // 12 32 // a // 26,40 // 12,15 // 2,49 // 2917
7 // Quazar // 0.4 64 // i // 27,50 // 14,47 // 2,91 // 2897
8 // Nemo // Spn 1.0.1b 64 // i // 23,67 // 11,78 // 2,37 // 2830
9 // Godel // 2.3.7 64 // i // 20,77 // 13,70 // 3,54 // 2629
10 // Ruffian // 20040408 32 // a // 19,90 // 9,78 // 3,27 // 2572
11 // Nimzo // 8cb 32 // a // 20,47 // 8,73 // 2,98 // 2563
12 // Anaconda // 2.01 32 // a // 18,20 // 9,12 // 3,46 // 2526
13 // Cheese // 1.5 pop 64 // i // 17,90 // 10,63 // 2,99 // 2526
14 // Rhetoric // 0.10d 32 // i // 8,83 // 8,11 // 3,78 // 2331
15 // Resp // 0.19JA 64 // i // 12,40 // 10,75 // 3,95 // 2248
16 // Bubble // 1.5 32 // a // 12,97 // 9,08 // 2,57 // 2096
17 // Gerbil // 0.2JA 64 // i // 9,97 // 8,59 // 2,56 // 1989

I like to post the testsuite, but how?

[wolfman]

i intel i5
a amd QL60
TSR Test Suite Rating
%MM % Mating Moves
AD Average Deep to reach MM
AT Average Time in seconds to reach MM, for a divided by 1.5
RT Rated Time
Date 21 jul 2013

Rank Engine Version Compu TSR %MM AD AT RT

1 Critter 1.6a 64 i 3197 41,33 12,19 1,77 660
2 Houdini 3 64 i 3193 40,13 12,79 1,83 672
3 Strelka 5.5.1 64 i 3188 40,60 12,55 1,86 670
4 Firenzina 2.3.1 64 i 3179 37,37 13,12 1,87 696
5 Bouquet 1.7 64 i 3166 37,63 13,07 2,01 699
6 Komodo 5.1r2 64 i 3144 35,90 12,46 2,10 716
7 Arrester Eagle 32 i 3133 36,03 12,26 2,20 719
8 Komodo CCT 64 i 3129 35,23 12,55 2,17 724
9 Stockfish 3 sse2 JA 64 i 3094 36,27 14,26 2,53 729
10 Fritz 13 32 a 2958 27,00 13,24 2,16 788
11 Booot 5.2.0 64 i 2953 28,50 13,85 2,69 792
12 Quazar 0.4 64 i 2906 27,50 14,47 2,91 805
13 Shredder 12 32 a 2905 26,97 10,35 2,55 799
14 Fritz 12 32 a 2894 26,40 12,15 2,49 802
15 Murka 3 64 i 2861 24,83 12,58 2,40 811
16 Nemo 1.0.1b 64 i 2817 23,67 11,78 2,37 819
17 Crafty 23.5JA 64 i 2813 23,93 14,02 2,67 825
18 Crafty 23.6JA 64 i 2795 23,83 14,61 2,91 831
19 Belka 2.0.0 32 a 2786 23,37 10,74 2,49 825
20 Fruit 2.3.1 32 a 2740 22,17 11,24 2,43 832
21 CT 2007.1 32 a 2729 22,87 11,75 2,99 840
22 The King 3.50 32 a 2704 22,00 8,86 2,88 843
23 Discocheck 4.3 sse2 64 i 2682 20,90 13,18 2,90 852
24 Cerebro 3.03d 32 a 2668 21,20 9,29 2,95 851
25 Cyclone xTr Fear 32 a 2657 20,90 11,38 2,92 852
26 Fruit 2.2.1 32 a 2656 20,80 10,59 2,84 851
27 Godel 2.3.7 64 i 2642 20,77 13,70 3,54 866
28 Nimzo 8cb 32 a 2636 20,47 8,73 2,98 856
29 Ruffian 20040408 32 a 2605 19,90 9,78 3,27 866
30 Brutus 8.05Ja 64 i 2574 18,53 11,15 2,92 869
31 Cheese 1.5 pop 64 i 2542 17,90 10,63 2,99 875
32 Anaconda 2.01 32 a 2533 18,20 9,12 3,46 881
33 Yace 0.9987 32 a 2532 18,20 9,45 3,43 880
34 Greko 10.2JA 64 i 2446 16,07 10,63 2,83 885
35 Resp 0.19JA 64 i 2273 12,40 10,75 3,95 925
36 Rhetoric Lite 32 a 2226 13,43 8,91 2,54 900
37 Bubble 1.5 32 a 2195 12,97 9,08 2,57 904
38 Rhetoric 0.10d 32 i 2051 8,83 8,11 3,78 945
39 Gerbil 0.2JA 64 i 2037 9,97 8,59 2,56 926

[wolfman]

Update 2 aug 2013.

Rank Engine Version Compu TSR %MM AAD ATMM ART
1 Critter 1.6a 64 i 3195 41,33 12,19 1,77 660
2 Houdini 3 64 i 3189 40,13 12,79 1,83 672
3 Strelka 5.5.1 64 i 3186 40,60 12,55 1,86 670
4 Firenzina 2.3.1 xTr 64 i 3173 37,37 13,12 1,87 696
5 Bouquet B1-1.7 64 i 3160 37,63 13,07 2,01 699
6 Komodo 5.1r2 64 i 3136 35,90 12,46 2,10 716
7 Arrester Eagle 32 i 3129 36,03 12,26 2,20 719
8 Sting SF 3 VE 64 i 3123 36,37 13,99 2,40 724
9 Komodo CCT 64 i 3122 35,23 12,55 2,17 724
10 Stockfish 3 sse2 JA 64 i 3119 36,27 14,26 2,53 729
11 Sting SF 4b 64 i 3107 35,53 13,85 2,56 736
12 Sting SF 2 pop VE 64 i 3097 34,40 14,12 2,34 736
13 Gandalf 6.02 32 a 2977 27,10 6,36 2,01 783
14 Fritz 13 32 a 2948 27,00 13,24 2,16 788
15 Booot 5.2.0 64 i 2944 28,50 13,85 2,69 792
16 Hannibal 1.3 64 i 2934 27,07 12,85 2,31 792
17 Texel 1.02JA 64 i 2915 27,93 13,92 3,22 810
18 Quazar 0.4 64 i 2904 27,50 14,47 2,91 805
19 Shredder 12 32 a 2901 26,97 10,35 2,55 799
20 Fritz 12 32 a 2888 26,40 12,15 2,49 802
21 Toga II 3.0JA 32 i 2873 26,17 12,24 2,68 808
22 Glaurung 081204JA 64 i 2865 25,67 11,83 2,56 809
23 Murka 3 64 i 2855 24,83 12,58 2,40 811
24 Glaurung 2.1JA 64 i 2825 24,47 11,78 2,56 818
25 Nemo Spn 1.0.1b 64 i 2818 23,67 11,78 2,37 819
26 Crafty 23.5JA 64 i 2796 23,93 14,02 2,67 825
27 Belka 2.0.0 32 a 2778 23,37 10,74 2,49 825
28 CM11 Leonidas 32 a 2778 24,07 8,99 2,88 829
29 Crafty 23.6JA 64 i 2775 23,83 14,61 2,91 831
30 CM11 Wolfman 32 a 2768 23,73 8,96 2,79 829
31 CM9 Tancredi 32 a 2760 23,57 8,65 2,82 831
32 CM10 Warrior 32 a 2750 23,27 8,71 2,78 832
33 Fruit 2.3.1 32 a 2735 22,17 11,24 2,43 832
34 ETChess 130108 32 a 2736 23,07 9,68 3,34 846
35 CT 2007.1 32 a 2725 22,87 11,75 2,99 840
36 Gaviota 0.86 64 i 2720 20,63 11,05 2,24 840
37 CM11 Default 32 a 2691 22,00 8,86 2,88 843
38 Arasan 16.0 64 i 2694 20,93 12,62 2,54 844
39 GambitFruit 1.0b4bx 32 a 2687 21,77 10,79 2,76 842
40 Arasan 15.6 64 i 2683 20,77 12,72 2,58 846
41 Discocheck 4.3 sse2 64 i 2661 20,90 13,18 2,90 852
42 Cerebro 3.03d 32 a 2652 21,20 9,29 2,95 851
43 Cyclone xTr Fear 32 a 2639 20,90 11,38 2,92 852
44 Fruit 2.2.1 32 a 2639 20,80 10,59 2,84 851
45 Godel 2.3.7 64 i 2624 20,77 13,70 3,54 866
46 Nimzo 8cb 32 a 2617 20,47 8,73 2,98 856
47 Ruffian 20040408 32 a 2583 19,90 9,78 3,27 866
48 Brutus 8.05Ja 64 i 2552 18,53 11,15 2,92 869
49 Hamsters 0.7.1 32 a 2534 18,87 10,61 3,10 870
50 Cheese 1.5 pop 64 i 2516 17,90 10,63 2,99 875
51 Anaconda 2.01 32 a 2495 18,20 9,12 3,46 881
52 Yace 0.9987 32 a 2495 18,20 9,45 3,43 880
53 Greko 10.2JA 64 i 2442 16,07 10,63 2,83 885
54 Tjchess 1.1 64 i 2303 13,67 10,23 2,99 904
55 Cinnamon 1.1 pop 64 i 2290 11,73 8,59 2,24 909
56 Rhetoric Lite 32 a 2279 13,43 8,91 2,54 900
57 Bubble 1.5 32 a 2247 12,97 9,08 2,57 904
58 Resp 0.19JA 64 i 2164 12,40 10,75 3,95 925
59 Gerbil 0.2JA 64 i 2142 9,97 8,59 2,56 926
60 Warrior 1.03 32 a 2048 10,60 7,66 3,15 927
61 Mint 2.3.1 32 a 2045 10,90 7,61 4,00 935
62 Rhetoric 0.10d 32 i 1900 8,83 8,11 3,78 945
63 BSC 3.9 64 i 1819 2,63 8,73 1,26 977
62 Darky 0.5e 32 a 1749 6,93 7,36 4,44 961
63 Darky 0.5d 32 a 1744 6,87 7,37 4,35 961

[velmarin]

I like to post the testsuite, but how?

For example:

http://limelinx.com/

No need to register.

[wolfman]

Thank you !

http://limelinx.com/-eqrrs-7d506

[velmarin]

Error - File Not Found

Something went no well.

[wolfman]

I am a donkey, so ihope there is not a second stone.

http://limelinx.com/4g6o

[velmarin]

It's okay.

Thanks,

[wolfman]

Update 8 aug 2013

Rank Engine Version Compu TSR %MM AAD ATMM ART
1 Critter 1.6a 64 i 3200 41,33 12,19 1,77 660
2 Houdini 3 64 i 3196 40,13 12,79 1,83 672
3 Robodini 1.1 64 i 3194 39,83 12,73 1,84 675
4 Strelka 5.5.1 64 i 3192 40,60 12,55 1,86 670
5 Firenzina 2.3.1 xTr 64 i 3182 37,37 13,12 1,87 696
6 Saros 3.3b 64 i 3178 38,33 12,90 1,97 692
7 Akkad 0.52b popcnt 64 i 3171 37,87 12,97 2,02 698
8 Bouquet B1-1.7 64 i 3169 37,63 13,07 2,01 699
9 Komodo 5.1r2 64 i 3147 35,90 12,46 2,10 716
10 Arrester Eagle 32 i 3140 36,03 12,26 2,20 719
11 Stockfish 20130802JS 64 i 3140 37,17 14,39 2,48 720
12 Komodo CCT 64 i 3133 35,23 12,55 2,17 724
13 Sting SF 3 VE 64 i 3132 36,37 13,99 2,40 724
14 Stockfish 3 sse2 JA 64 i 3127 36,27 14,26 2,53 729
15 Sting SF 4b 64 i 3116 35,53 13,85 2,56 736
16 Sting SF 2 pop VE 64 i 3109 34,40 14,12 2,34 736
17 Tankist 3.1 32 a 3061 33,03 11,89 2,43 750
18 Gandalf 6.02 32 a 2994 27,10 6,36 2,01 783
19 Fritz 13 32 a 2964 27,00 13,24 2,16 788
20 Booot 5.2.0 64 i 2957 28,50 13,85 2,69 792
21 Hannibal 1.3 64 i 2950 27,07 12,85 2,31 792
22 Texel 1.02JA 64 i 2922 27,93 13,92 3,22 810
23 Quazar 0.4 64 i 2915 27,50 14,47 2,91 805
24 Shredder 12 32 a 2912 26,97 10,35 2,55 799
25 Fritz 12 32 a 2901 26,40 12,15 2,49 802
26 Toga II 3.0JA 32 i 2887 26,17 12,24 2,68 808
27 Glaurung 081204JA 64 i 2881 25,67 11,83 2,56 809
28 Murka 3 64 i 2871 24,83 12,58 2,40 811
29 Glaurung 2.1JA 64 i 2841 24,47 11,78 2,56 818
30 Nemo 1.0.1b 64 i 2835 23,67 11,78 2,37 819
31 Crafty 23.5JA 64 i 2812 23,93 14,02 2,67 825
32 Belka 2.0.0 32 a 2796 23,37 10,74 2,49 825
33 CM11 Leonidas 32 a 2789 24,07 8,99 2,88 829
34 Crafty 23.6JA 64 i 2789 23,83 14,61 2,91 831
35 CM11 Wolfman 32 a 2781 23,73 8,96 2,79 829
36 CM9 Tancredi 32 a 2773 23,57 8,65 2,82 831
37 CM10 Warrior 32 a 2764 23,27 8,71 2,78 832
38 Fruit 2.3.1 32 a 2755 22,17 11,24 2,43 832
39 ETChess 130108 32 a 2742 23,07 9,68 3,34 846
40 Gaviota 0.86 64 i 2739 20,63 11,05 2,24 840
41 CT 2007.1 32 a 2737 22,87 11,75 2,99 840
42 Bright 0.5c 32 a 2720 22,30 11,24 3,66 859
43 Scorpio 2.7.5 64 i 2714 21,50 13,52 2,73 844
44 Arasan 16.0 64 i 2712 20,93 12,62 2,54 844
45 CM11 Default 32 a 2706 22,00 8,86 2,88 843
46 Movei 08438_101010 32 a 2705 22,23 10,33 3,35 852
47 GambitFruit 1.0b4bx 32 a 2704 21,77 10,79 2,76 842
48 Arasan 15.6 64 i 2701 20,77 12,72 2,58 846
49 Scorpio 2.7.6 64 i 2683 19,87 13,01 2,42 849
50 Delfi 5.4 32 a 2680 21,47 10,61 2,93 848
51 Discocheck 4.3 sse2 64 i 2673 20,90 13,18 2,90 852
52 Cerebro 3.03d 32 a 2666 21,20 9,29 2,95 851
53 Fruit 2.2.1 32 a 2656 20,80 10,59 2,84 851
54 Cyclone xTr Fear 32 a 2655 20,90 11,38 2,92 852
55 Nimzo 8cb 32 a 2632 20,47 8,73 2,98 856
56 Godel 2.3.7 64 i 2631 20,77 13,70 3,54 866
57 Goliath Light 1.5cb 32 a 2610 19,83 9,69 2,86 858
58 Ruffian 20040408 32 a 2595 19,90 9,78 3,27 866
59 Amyan 1.72 32 a 2593 19,87 11,08 3,38 868
60 Brutus 8.05Ja 64 i 2568 18,53 11,15 2,92 869
61 Hamsters 0.7.1 32 a 2550 18,87 10,61 3,10 870
62 Cheese 1.5 pop 64 i 2532 17,90 10,63 2,99 875
63 Anaconda 2.01 32 a 2507 18,20 9,12 3,46 881
64 Yace 0.9987 32 a 2507 18,20 9,45 3,43 880
65 Greko 10.2JA 64 i 2460 16,07 10,63 2,83 885
66 Delphil 3.0 64 i 2432 16,53 8,71 3,41 891
67 Delphil 2.9g 64 i 2400 16,00 8,41 3,49 896
68 Tjchess 1.1 64 i 2320 13,67 10,23 2,99 904
69 Cinnamon 1.1 pop 64 i 2312 11,73 8,59 2,24 909
70 Rhetoric Lite 32 a 2298 13,43 8,91 2,54 900
71 Bubble 1.5 32 a 2266 12,97 9,08 2,57 904
72 Resp 0.19JA 64 i 2173 12,40 10,75 3,95 925
73 Gerbil 0.2JA 64 i 2161 9,97 8,59 2,56 926
74 Maverick 0.05 64 i 2070 9,73 7,18 3,24 934
75 Warrior 1.03 32 a 2063 10,60 7,66 3,15 927
76 Mint 2.3.1 32 a 2057 10,90 7,61 4,00 935
77 MadChess 1.2 32 i 2050 10,70 7,49 4,32 939
78 Rhetoric 0.10d 32 i 1966 8,83 8,11 3,78 945
79 BSC 3.9 64 i 1836 2,63 8,73 1,26 977
77 Darky 0.5e 32 a 1761 6,93 7,36 4,44 961
78 Darky 0.5d 32 a 1756 6,87 7,37 4,35 961

[Paloma]

It's NOT !!! okay.

Only one file: Final.cbh

With this one file you can't open the database