Good. I usually run my own regression suite of 500 games per opponent against a set of opponents (usually top 50 engines, tc=all/60+0.6 with tbs), this allows me to get an average elo increase estimate because the suite is mandatory for an Igel release.
For example,
Igel 2.4.0
Code: Select all
Rank Name Elo +/- Games Score Draws
0 Igel 2.4.0 64 POPCNT -80 6 10000 38.7% 32.1%
1 Stockfish 10 64 POPCNT 501 48 500 94.7% 10.6%
2 Ethereal 11.75 (PEXT) 324 31 500 86.6% 24.4%
3 Fire 7.1 x64 popcnt 311 31 500 85.7% 23.8%
4 Xiphos 0.6 BMI2 299 30 500 84.8% 26.4%
5 rofChade 2.202 BMI 217 25 500 77.7% 35.8%
6 Laser 1.7 215 27 500 77.5% 33.0%
7 Defenchess 2.2 x64 192 27 500 75.1% 32.6%
8 Andscacs 0.95 161 26 500 71.7% 33.0%
9 Pedone 2.0 109 24 500 65.2% 38.8%
10 RubiChess 1.6 81 23 500 61.5% 44.2%
11 Arasan 22.0 66 24 500 59.4% 39.6%
12 Strelka 5.5 x64 62 24 500 58.8% 36.8%
13 Deep iCE 4.0.853 x64/popcnt 48 24 500 56.8% 36.8%
14 Texel 1.07 41 22 500 55.9% 47.0%
15 Vajolet2 2.8.0 39 24 500 55.6% 37.6%
16 Nemorino 27 24 500 53.9% 37.4%
17 Protector 1.9.0 -13 24 500 48.1% 36.2%
18 Winter 0.7 BMI2 -136 27 500 31.4% 30.0%
19 zurichess neuchatel -234 27 500 20.6% 31.2%
20 GreKo 2018.08 -487 54 500 5.7% 7.8%
Code: Select all
Rank Name Elo +/- Games Score Draws
0 Igel 2.3.1 64 POPCNT -106 6 10000 35.2% 31.0%
1 Stockfish 10 64 POPCNT 527 55 500 95.4% 8.0%
2 Fire 7.1 x64 popcnt 346 35 500 88.0% 19.6%
3 Ethereal 11.75 (PEXT) 340 31 500 87.6% 23.6%
4 Xiphos 0.6 BMI2 318 31 500 86.2% 24.4%
5 Laser 1.7 259 28 500 81.6% 29.6%
6 rofChade 2.202 BMI 220 27 500 78.0% 32.8%
7 Defenchess 2.2 x64 206 26 500 76.6% 34.0%
8 Andscacs 0.95 172 27 500 72.9% 30.2%
9 RubiChess 1.6 161 24 500 71.7% 41.0%
10 Pedone 2.0 143 25 500 69.5% 34.6%
11 Strelka 5.5 x64 119 25 500 66.5% 34.6%
12 Deep iCE 4.0.853 x64/popcnt 98 24 500 63.7% 39.0%
13 Arasan 22.0 95 25 500 63.4% 35.6%
14 Texel 1.07 89 24 500 62.6% 38.8%
15 Vajolet2 2.8.0 65 23 500 59.2% 44.4%
16 Nemorino 44 25 500 56.3% 34.2%
17 Protector 1.9.0 2 23 500 50.3% 40.6%
18 Winter 0.7 BMI2 -111 26 500 34.5% 29.8%
19 zurichess neuchatel -215 26 500 22.5% 33.8%
20 GreKo 2018.08 -411 44 500 8.6% 11.2%
Code: Select all
Rank Name Elo +/- Games Score Draws
0 Igel 2.3.0 64 POPCNT -178 14 2000 26.5% 24.9%
1 Stockfish 10 64 POPCNT 531 132 100 95.5% 9.0%
2 Xiphos 0.6 BMI2 494 115 100 94.5% 11.0%
3 Ethereal 11.75 (PEXT) 478 121 100 94.0% 10.0%
4 Fire 7.1 x64 popcnt 436 103 100 92.5% 13.0%
5 Laser 1.7 330 68 100 87.0% 26.0%
6 rofChade 2.202 BMI 323 74 100 86.5% 23.0%
7 Defenchess 2.2 x64 295 73 100 84.5% 23.0%
8 Andscacs 0.95 263 72 100 82.0% 22.0%
9 Pedone 2.0 230 65 100 79.0% 28.0%
10 RubiChess 1.6 230 61 100 79.0% 32.0%
11 Strelka 5.5 x64 220 71 100 78.0% 20.0%
12 Arasan 22.0 186 61 100 74.5% 31.0%
13 Texel 1.07 168 63 100 72.5% 27.0%
14 Deep iCE 4.0.853 x64/popcnt 143 54 100 69.5% 41.0%
15 Nemorino 143 61 100 69.5% 29.0%
16 Vajolet2 2.8.0 119 58 100 66.5% 33.0%
17 Protector 1.9.0 100 56 100 64.0% 36.0%
18 Winter 0.7 BMI2 -31 58 100 45.5% 29.0%
19 zurichess neuchatel -63 53 100 41.0% 40.0%
20 GreKo 2018.08 -295 83 100 15.5% 15.0%