The Speedy Rating List

[matejst]

Thank you, Ed.

I still use both Pro Deo and Benjamin when analyzing, so it is meaningful to me, although depth has a big influence on the Speedy Rating List.

I rather miss an "anti-GM" option in many engines, and I was more interested in the ERL, but it was a project nobody was interested in. Lately, most engines have an evaluation tuned automatically, it gives good results in eng-eng matches, but it is ugly chess.

[xr_a_y]

Thanks for considering Minic in here !

Can you please give a shot to Minic 2.32 (unofficial release that can be found here https://github.com/tryingsomestuff/Mini ... ter/Minic2)

I suspect it might be stronger on this type of test ...

[Rebel]

Thank you, Ed.

I still use both Pro Deo and Benjamin when analyzing, so it is meaningful to me, although depth has a big influence on the Speedy Rating List.

True that, search rules.

I rather miss an "anti-GM" option in many engines, and I was more interested in the ERL, but it was a project nobody was interested in. Lately, most engines have an evaluation tuned automatically, it gives good results in eng-eng matches, but it is ugly chess.

Same cause, search.

I have seen it happen to my hand-tuned evaluation because back then there were no eng-eng facilities, nowadays search with all its reductions and pruning (while adding hundreds of elo points) damages what you carefully have programmed and tuned, the evaluation function and with it the playing style. As a positional oriented chess player I hate it, as computer chess programmer I have no choice

BTW, there is a replacement for the anti-GM option, actually 3, all of them function better than the original anti-GM.

[ATTACKING = 100]
[CHAOS = 100]
[CONFUSION = 100]

They all 3 try to complicate the position.

The activation is a bit non-standard, 100 means not active, but any other value than 100 it becomes alive. Forgot why I did it this strange way.

[Rebel]

Thanks for considering Minic in here !

Can you please give a shot to Minic 2.32 (unofficial release that can be found here https://github.com/tryingsomestuff/Mini ... ter/Minic2)

I suspect it might be stronger on this type of test ...

Oki doki, but which executable to pick?

Code: Select all

   minic_2.32_mingw_x64_nehalem.exe  minic 2.32   May 28, 2020  
   minic_2.32_mingw_x64_skylake.exe  minic 2.32   May 28, 2020  
   minic_2.32_mingw_x64_x86-64.exe  minic 2.32   May 28, 2020  

[xr_a_y]

Thanks for considering Minic in here !

Can you please give a shot to Minic 2.32 (unofficial release that can be found here https://github.com/tryingsomestuff/Mini ... ter/Minic2)

I suspect it might be stronger on this type of test ...

Oki doki, but which executable to pick?

Code: Select all

   minic_2.32_mingw_x64_nehalem.exe  minic 2.32   May 28, 2020  
   minic_2.32_mingw_x64_skylake.exe  minic 2.32   May 28, 2020  
   minic_2.32_mingw_x64_x86-64.exe  minic 2.32   May 28, 2020  

In this order, best to worst, depending on your hardware :
skylake == avx2/bmi2
nehalem == sse4.2
x86-64 == just popcnt

[lkaufman]

Added Winter 0.8, ProDeo 2.2 and Benjamin, provisional (text) result as I adding more:

Code: Select all

    EPD  : epd\lc1.epd
    Time : 1000ms
                                                               Max            Time   Hash          
    Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
 1  Stockfish 11     324287  11:30:00.8  23191  40000  3242  400000  81.07%   1000   128    1     0
 2  Komodo 14        317110  11:09:21.4  22398  40000  3171  400000  79.28%   1000   128    1     0
 3  Houdini 6.03     314596  11:27:54.8  22218  40000  3146  400000  78.65%   1000   128    1     0
 4  rofChade 2.3     309642  11:24:20.2  21587  40000  3096  400000  77.41%   1000   128    1    41
 5  Ethereal 12      309088  11:30:20.9  21840  40000  3090  400000  77.27%   1000   128    1     0
 6  Laser 1.7        308178  11:29:30.5  21320  40000  3081  400000  77.04%   1000   128    1     0
 7  Schooner 2.2     306808  10:22:01.9  21335  40000  3068  400000  76.70%   1000   128    1     0
 8  Xiphos 0.6       306796  10:03:55.9  21333  40000  3068  400000  76.70%   1000   128    1     0
 9  RubiChess-1.7.2  305900  11:26:03.9  21215  40000  3059  400000  76.48%   1000   128    1     0
10  Booot 6.4        305082  12:56:19.6  21267  40000  3050  400000  76.27%   1000   128    1   124
11  Defenchess 2.2   303754  10:12:00.7  21208  40000  3037  400000  75.94%   1000   128    1    41
12  Andscacs 0.95    301560  12:10:30.8  20745  40000  3015  400000  75.39%   1005   128    1    41
13  Shredder 13      300619  11:28:47.0  20946  40000  3006  400000  75.15%   1000   128    1     0
14  Arasan 22        298663  11:27:32.7  20460  40000  2986  400000  74.67%   1000   128    1     0
15  ice 4.0          297995  11:49:51.1  20767  40000  2980  400000  74.50%   1000   128    1     0
16  Fire 7.1         296996  11:26:35.7  20356  40000  2970  400000  74.25%   1000   128    1     0
17  Vajolet 2.8      296269  11:25:51.6  20327  40000  2962  400000  74.07%   1000   128    1     0
18  Demolito 200514  295686  10:53:58.5  20429  40000  2956  400000  73.92%   1000   128    1     0
19  Wasp 3.75        293908  11:44:35.3  20203  40000  2939  400000  73.48%   1000   128    1     0
20  Gogobello 2.2    293824  11:44:24.1  20296  40000  2938  400000  73.46%   1000   128    1    41
21  igel 2.40        293056  11:28:38.5  20261  40000  2930  400000  73.26%   1000   128    1    41
22  Rodent 4         291647  11:45:03.3  19774  40000  2916  400000  72.91%   1000   128    1     0
23  Winter 0.8       291596  10:02:03.0  20057  40000  2916  400000  72.90%   1000   128    1    41
24  Monolith 2       290807  11:31:27.5  19972  40000  2908  400000  72.70%   1000   128    1     0
25  Deuterium 2019   290578  11:31:42.7  19819  40000  2905  400000  72.64%   1000   128    1     0
26  Amoeba 3.1       289712  09:40:26.2  19682  40000  2897  400000  72.43%   1000   128    1     0
27  FabChess 1.15    289296  11:50:40.2  19849  40000  2892  400000  72.32%   1000   128    1    41
28  Marvin 3.60      288732  10:48:52.8  19848  40000  2887  400000  72.18%   1000   128    1    41
29  Minic 2.25       287730  10:54:48.6  19780  40000  2877  400000  71.93%   1000   128    1     0
30  Counter 3.5      285137  11:25:20.1  19329  40000  2851  400000  71.28%   1000   128    1     0
31  Asymptote 0.8    284305  12:19:43.9  19245  40000  2843  400000  71.08%   1000   128    1    41
32  Weiss 0.10       281037  11:06:09.2  18927  40000  2810  400000  70.26%   1000   128    1     0
33  Fruit 2.3        275618  11:30:55.1  18270  40000  2756  400000  68.90%   1000   128    1     0
34  ProDeo 2.2       273793  11:29:46.3  18263  40000  2738  400000  68.45%   1000   128    1     0
35  Benjamin         271801  11:29:55.8  17985  40000  2718  400000  67.95%   1000   128    1     0
36  Fruit 2.1        270830  11:29:45.7  17982  40000  2708  400000  67.71%   1000   128    1     0
37  Ruffian 2        265592  11:13:01.7  17740  40000  2656  400000  66.40%   1000   128    1     0

The range of the ratings shown is roughly half that of the range on the 1' + 0.6" FastGM list, which should be the closest comparison (although it doesn't have all of these engines). I suppose the scaling was matched to some rating list, but what rating list has such a narrow range of ratings from top to bottom? Other than this scaling detail, it does look surprisingly accurate.

[Rebel]

Added Winter 0.8, ProDeo 2.2 and Benjamin, provisional (text) result as I adding more:

Code: Select all

    EPD  : epd\lc1.epd
    Time : 1000ms
                                                               Max            Time   Hash          
    Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
 1  Stockfish 11     324287  11:30:00.8  23191  40000  3242  400000  81.07%   1000   128    1     0
 2  Komodo 14        317110  11:09:21.4  22398  40000  3171  400000  79.28%   1000   128    1     0
 3  Houdini 6.03     314596  11:27:54.8  22218  40000  3146  400000  78.65%   1000   128    1     0
 4  rofChade 2.3     309642  11:24:20.2  21587  40000  3096  400000  77.41%   1000   128    1    41
 5  Ethereal 12      309088  11:30:20.9  21840  40000  3090  400000  77.27%   1000   128    1     0
 6  Laser 1.7        308178  11:29:30.5  21320  40000  3081  400000  77.04%   1000   128    1     0
 7  Schooner 2.2     306808  10:22:01.9  21335  40000  3068  400000  76.70%   1000   128    1     0
 8  Xiphos 0.6       306796  10:03:55.9  21333  40000  3068  400000  76.70%   1000   128    1     0
 9  RubiChess-1.7.2  305900  11:26:03.9  21215  40000  3059  400000  76.48%   1000   128    1     0
10  Booot 6.4        305082  12:56:19.6  21267  40000  3050  400000  76.27%   1000   128    1   124
11  Defenchess 2.2   303754  10:12:00.7  21208  40000  3037  400000  75.94%   1000   128    1    41
12  Andscacs 0.95    301560  12:10:30.8  20745  40000  3015  400000  75.39%   1005   128    1    41
13  Shredder 13      300619  11:28:47.0  20946  40000  3006  400000  75.15%   1000   128    1     0
14  Arasan 22        298663  11:27:32.7  20460  40000  2986  400000  74.67%   1000   128    1     0
15  ice 4.0          297995  11:49:51.1  20767  40000  2980  400000  74.50%   1000   128    1     0
16  Fire 7.1         296996  11:26:35.7  20356  40000  2970  400000  74.25%   1000   128    1     0
17  Vajolet 2.8      296269  11:25:51.6  20327  40000  2962  400000  74.07%   1000   128    1     0
18  Demolito 200514  295686  10:53:58.5  20429  40000  2956  400000  73.92%   1000   128    1     0
19  Wasp 3.75        293908  11:44:35.3  20203  40000  2939  400000  73.48%   1000   128    1     0
20  Gogobello 2.2    293824  11:44:24.1  20296  40000  2938  400000  73.46%   1000   128    1    41
21  igel 2.40        293056  11:28:38.5  20261  40000  2930  400000  73.26%   1000   128    1    41
22  Rodent 4         291647  11:45:03.3  19774  40000  2916  400000  72.91%   1000   128    1     0
23  Winter 0.8       291596  10:02:03.0  20057  40000  2916  400000  72.90%   1000   128    1    41
24  Monolith 2       290807  11:31:27.5  19972  40000  2908  400000  72.70%   1000   128    1     0
25  Deuterium 2019   290578  11:31:42.7  19819  40000  2905  400000  72.64%   1000   128    1     0
26  Amoeba 3.1       289712  09:40:26.2  19682  40000  2897  400000  72.43%   1000   128    1     0
27  FabChess 1.15    289296  11:50:40.2  19849  40000  2892  400000  72.32%   1000   128    1    41
28  Marvin 3.60      288732  10:48:52.8  19848  40000  2887  400000  72.18%   1000   128    1    41
29  Minic 2.25       287730  10:54:48.6  19780  40000  2877  400000  71.93%   1000   128    1     0
30  Counter 3.5      285137  11:25:20.1  19329  40000  2851  400000  71.28%   1000   128    1     0
31  Asymptote 0.8    284305  12:19:43.9  19245  40000  2843  400000  71.08%   1000   128    1    41
32  Weiss 0.10       281037  11:06:09.2  18927  40000  2810  400000  70.26%   1000   128    1     0
33  Fruit 2.3        275618  11:30:55.1  18270  40000  2756  400000  68.90%   1000   128    1     0
34  ProDeo 2.2       273793  11:29:46.3  18263  40000  2738  400000  68.45%   1000   128    1     0
35  Benjamin         271801  11:29:55.8  17985  40000  2718  400000  67.95%   1000   128    1     0
36  Fruit 2.1        270830  11:29:45.7  17982  40000  2708  400000  67.71%   1000   128    1     0
37  Ruffian 2        265592  11:13:01.7  17740  40000  2656  400000  66.40%   1000   128    1     0

The range of the ratings shown is roughly half that of the range on the 1' + 0.6" FastGM list, which should be the closest comparison (although it doesn't have all of these engines). I suppose the scaling was matched to some rating list, but what rating list has such a narrow range of ratings from top to bottom? Other than this scaling detail, it does look surprisingly accurate.

Thanks Larry for your kind words. Regarding the elo calculation, it's quite simple: Komodo 14 scores 79.28% multiply this with 40 and you get the elo rating, 3.171. I could easily increase the value to 41, 42, 43 till it matches the known rating lists. But that step has to be taken when the 4000ms list is more voluminous because 4000ms is comparable with for instance CCRL 40/2. I don't recall an elo list based on 1 second per move.

[lkaufman]

Added Winter 0.8, ProDeo 2.2 and Benjamin, provisional (text) result as I adding more:

Code: Select all

    EPD  : epd\lc1.epd
    Time : 1000ms
                                                               Max            Time   Hash          
    Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
 1  Stockfish 11     324287  11:30:00.8  23191  40000  3242  400000  81.07%   1000   128    1     0
 2  Komodo 14        317110  11:09:21.4  22398  40000  3171  400000  79.28%   1000   128    1     0
 3  Houdini 6.03     314596  11:27:54.8  22218  40000  3146  400000  78.65%   1000   128    1     0
 4  rofChade 2.3     309642  11:24:20.2  21587  40000  3096  400000  77.41%   1000   128    1    41
 5  Ethereal 12      309088  11:30:20.9  21840  40000  3090  400000  77.27%   1000   128    1     0
 6  Laser 1.7        308178  11:29:30.5  21320  40000  3081  400000  77.04%   1000   128    1     0
 7  Schooner 2.2     306808  10:22:01.9  21335  40000  3068  400000  76.70%   1000   128    1     0
 8  Xiphos 0.6       306796  10:03:55.9  21333  40000  3068  400000  76.70%   1000   128    1     0
 9  RubiChess-1.7.2  305900  11:26:03.9  21215  40000  3059  400000  76.48%   1000   128    1     0
10  Booot 6.4        305082  12:56:19.6  21267  40000  3050  400000  76.27%   1000   128    1   124
11  Defenchess 2.2   303754  10:12:00.7  21208  40000  3037  400000  75.94%   1000   128    1    41
12  Andscacs 0.95    301560  12:10:30.8  20745  40000  3015  400000  75.39%   1005   128    1    41
13  Shredder 13      300619  11:28:47.0  20946  40000  3006  400000  75.15%   1000   128    1     0
14  Arasan 22        298663  11:27:32.7  20460  40000  2986  400000  74.67%   1000   128    1     0
15  ice 4.0          297995  11:49:51.1  20767  40000  2980  400000  74.50%   1000   128    1     0
16  Fire 7.1         296996  11:26:35.7  20356  40000  2970  400000  74.25%   1000   128    1     0
17  Vajolet 2.8      296269  11:25:51.6  20327  40000  2962  400000  74.07%   1000   128    1     0
18  Demolito 200514  295686  10:53:58.5  20429  40000  2956  400000  73.92%   1000   128    1     0
19  Wasp 3.75        293908  11:44:35.3  20203  40000  2939  400000  73.48%   1000   128    1     0
20  Gogobello 2.2    293824  11:44:24.1  20296  40000  2938  400000  73.46%   1000   128    1    41
21  igel 2.40        293056  11:28:38.5  20261  40000  2930  400000  73.26%   1000   128    1    41
22  Rodent 4         291647  11:45:03.3  19774  40000  2916  400000  72.91%   1000   128    1     0
23  Winter 0.8       291596  10:02:03.0  20057  40000  2916  400000  72.90%   1000   128    1    41
24  Monolith 2       290807  11:31:27.5  19972  40000  2908  400000  72.70%   1000   128    1     0
25  Deuterium 2019   290578  11:31:42.7  19819  40000  2905  400000  72.64%   1000   128    1     0
26  Amoeba 3.1       289712  09:40:26.2  19682  40000  2897  400000  72.43%   1000   128    1     0
27  FabChess 1.15    289296  11:50:40.2  19849  40000  2892  400000  72.32%   1000   128    1    41
28  Marvin 3.60      288732  10:48:52.8  19848  40000  2887  400000  72.18%   1000   128    1    41
29  Minic 2.25       287730  10:54:48.6  19780  40000  2877  400000  71.93%   1000   128    1     0
30  Counter 3.5      285137  11:25:20.1  19329  40000  2851  400000  71.28%   1000   128    1     0
31  Asymptote 0.8    284305  12:19:43.9  19245  40000  2843  400000  71.08%   1000   128    1    41
32  Weiss 0.10       281037  11:06:09.2  18927  40000  2810  400000  70.26%   1000   128    1     0
33  Fruit 2.3        275618  11:30:55.1  18270  40000  2756  400000  68.90%   1000   128    1     0
34  ProDeo 2.2       273793  11:29:46.3  18263  40000  2738  400000  68.45%   1000   128    1     0
35  Benjamin         271801  11:29:55.8  17985  40000  2718  400000  67.95%   1000   128    1     0
36  Fruit 2.1        270830  11:29:45.7  17982  40000  2708  400000  67.71%   1000   128    1     0
37  Ruffian 2        265592  11:13:01.7  17740  40000  2656  400000  66.40%   1000   128    1     0

The range of the ratings shown is roughly half that of the range on the 1' + 0.6" FastGM list, which should be the closest comparison (although it doesn't have all of these engines). I suppose the scaling was matched to some rating list, but what rating list has such a narrow range of ratings from top to bottom? Other than this scaling detail, it does look surprisingly accurate.

Thanks Larry for your kind words. Regarding the elo calculation, it's quite simple: Komodo 14 scores 79.28% multiply this with 40 and you get the elo rating, 3.171. I could easily increase the value to 41, 42, 43 till it matches the known rating lists. But that step has to be taken when the 4000ms list is more voluminous because 4000ms is comparable with for instance CCRL 40/2. I don't recall an elo list based on 1 second per move.

The FastGM 1' + 0.6" list comes the closest of major lists, it's about 2 seconds per move. Multiplying by 80 rather than 40 and subtracting a suitable constant, somewhere around 2950 I think, should produce a fairly good match. Of course a regression would produce a closer match, this is just a quick estimate.

[Rebel]

Thanks for considering Minic in here !

Can you please give a shot to Minic 2.32 (unofficial release that can be found here https://github.com/tryingsomestuff/Mini ... ter/Minic2)

I suspect it might be stronger on this type of test ...

Oki doki, but which executable to pick?

Code: Select all

   minic_2.32_mingw_x64_nehalem.exe  minic 2.32   May 28, 2020  
   minic_2.32_mingw_x64_skylake.exe  minic 2.32   May 28, 2020  
   minic_2.32_mingw_x64_x86-64.exe  minic 2.32   May 28, 2020  

In this order, best to worst, depending on your hardware :
skylake == avx2/bmi2
nehalem == sse4.2
x86-64 == just popcnt

Code: Select all

                                                               Max            Time   Hash          
    Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
29  Minic 2.32       289394  11:01:03.5  19880  40000  2894  400000  72.35%   1000   128    1     0
32  Minic 2.25       287730  10:54:48.6  19780  40000  2877  400000  71.93%   1000   128    1     0

Used the sse4.2 executable for 2.32, not sure which one I used for 2.25, likely the sse4.2 also.

[cucumber]

Wow, this is really cool. Do you think you could talk about the methodology behind Lc1.epd? Also, would you be able to test Fire 7.1 again? I'm surprised to see it so low. It seems like a major outlier compared to what rating lists show.