A complete 2000 lines of code engine

[xr_a_y]

You've had a rate of over one release per week since the beginning of the year. Even with about half of your dev-versions not being made into releases, that's still a huge amount of releases, no tester can hope to test all of your official releases.

Also, your versioning scheme doesn't help to differentiate these releases from normal dev versions. With Xiphos bumping the middle version number, or Eth now doing a release every 25 commits, it's more obvious.

If I had to give release-for-testing guidelines, it would be something like this :

Code: Select all

Less than 1 month : never except for bugfixes
1 to 1.5 months : 100+ elo
1.5 to 2 months : 50+ elo
2 to 3 months : 30+ elo
3 to 6 months : 20+ elo
Over 6 months : 15+ elo (looking at you, Komodo)

Major features such as FRC can replace some elo.

Of course I don't expect all releases to be tested. And I am very thankful to all testers for the kind work and CPU time. Very valuable inputs !
Again my initial message about being "stuck" was referring to Minic strength, not a all Minic "tested version". Especially at CCRL 40/15 TC, there is absolutely no way testers can test each versions.

My hope is to release when "needed", meaning at least +10 or a bad bug fix. Which makes sense to me.

Indeed, december and january were very impacted by many fixes (many for TCEC). The mean rate being around 1 per week.

Code: Select all

2020-02-21 19:33:52 +0100  (tag: 1.44)                                                         
2020-02-14 21:31:39 +0100  (tag: 1.42)                                                         
2020-02-09 19:16:42 +0100  (tag: 1.39)                                                         
2020-01-18 12:26:07 +0100  (tag: 1.35)                                                         
2020-01-14 18:59:15 +0100  (tag: 1.34)                                                         
2020-01-12 21:09:03 +0100  (tag: 1.32)                                                         
2020-01-10 08:03:06 +0100  (tag: 1.31)                                                         
2020-01-09 19:49:15 +0100  (tag: 1.30)                                                         
2020-01-01 13:44:25 +0100  (tag: 1.26)                                                         
2020-01-01 10:55:56 +0100  (tag: 1.25)                                                         
2019-12-31 20:14:17 +0100  (tag: 1.24)                                                         
2019-12-29 19:06:19 +0100  (tag: 1.22)                                                         
2019-12-27 23:59:55 +0100  (tag: 1.21)                                                         
2019-12-13 16:20:58 +0100  (tag: 1.19)
2019-12-05 19:04:21 +0100  (tag: 1.15)
2019-11-27 19:05:00 +0100  (tag: 1.14)
2019-11-09 09:29:39 +0100  (tag: 1.09)
2019-10-16 08:27:48 +0200  (tag: 1.00)
2019-09-14 08:46:30 +0200  (tag: 0.99)
2019-09-04 21:09:09 +0200  (tag: 0.98)
2019-08-31 09:22:16 +0200  (tag: 0.92)
2019-08-29 07:54:30 +0200  (tag: 0.89)
2019-08-21 10:38:16 +0200  (tag: 0.87)
2019-07-27 08:04:03 +0200  (tag: 0.82)
2019-07-08 12:45:46 +0200  (tag: 0.76)
2019-07-03 20:27:59 +0200  (tag: 0.74)
2019-06-10 09:00:55 +0200  (tag: 0.65)
2019-06-04 06:47:09 +0200  (tag: 0.62)
2019-05-28 07:33:06 +0200  (tag: 0.61)
2019-05-23 20:40:04 +0200  (tag: 0.60)
2019-05-22 07:32:41 +0200  (tag: 0.59)
2019-05-14 07:21:39 +0200  (tag: 0.57)
2019-05-02 15:06:36 +0200  (tag: 0.53)
2019-04-28 10:18:29 +0200  (tag: 0.50)
2019-04-03 07:34:35 +0200  (tag: 0.47)
2019-03-31 15:34:51 +0200  (tag: 0.45)
2019-03-05 20:38:40 +0100  (tag: 0.43)
2019-03-05 13:07:22 +0100  (tag: 0.42)
2019-02-28 09:39:49 +0100  (tag: 0.41)
2019-02-19 07:27:15 +0100  (tag: 0.40)
2019-02-18 07:43:31 +0100  (tag: 0.39)
2019-02-17 15:47:27 +0100  (tag: 0.38)
2019-02-13 07:33:55 +0100  (tag: 0.37)
2019-01-27 09:32:51 +0100  (tag: 0.34)
2019-01-15 19:25:40 +0100  (tag: 0.32)
2019-01-10 20:43:01 +0100  (tag: 0.30)
2019-01-05 16:41:37 +0100  (tag: 0.29)
2018-12-25 09:38:26 +0100  (tag: 0.28)
2018-12-22 21:57:10 +0100  (tag: 0.26)
2018-12-19 21:18:35 +0100  (tag: 0.25)
2018-11-27 20:34:51 +0100  (tag: v0.20)

You are very right about the naming scheme to differentiate "official" and "dev".
I will introduce something to tackle this issue soon. Thanks for the suggestion !

[fastgm]

FGRL 60 + 0.6 sec
http://www.fastgm.de

Minic 1.44 +42 Elo better than Minic 1.22

Code: Select all

   # Engine       :    Elo  Error   Games    (%)      W       D      L     D(%)   CFS
--------------------------------------------------------------------------------------
 247 Minic 1.44   :   2738      4   22000   39.29    5692   5903  10405   26.83    51
 273 Minic 1.22   :   2696      4   25500   34.08    5391   6600  13509   25.88    79

[xr_a_y]

FGRL 60 + 0.6 sec
http://www.fastgm.de

Minic 1.44 +42 Elo better than Minic 1.22

Code: Select all

   # Engine       :    Elo  Error   Games    (%)      W       D      L     D(%)   CFS
--------------------------------------------------------------------------------------
 247 Minic 1.44   :   2738      4   22000   39.29    5692   5903  10405   26.83    51
 273 Minic 1.22   :   2696      4   25500   34.08    5391   6600  13509   25.88    79

Thanks a lot for testing

[xr_a_y]

Recent progress (past 3 months)

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   0 minic_1.49                      9       6    7842   51.3%   46.2%
   1 RubiChess                     185      28     412   74.4%   35.2%
   2 demolito                      104      27     413   64.5%   36.1%
   3 igel-last                      76      25     413   60.8%   45.0%
   4 PeSTO_bmi2                     19      26     413   52.7%   38.5%
   5 Winter0.7                       5      26     412   50.7%   40.3%
   6 minic_1.48                     -1      19     412   49.9%   68.7%
   7 Hakkapeliitta                  -5      27     413   49.3%   33.2%
   8 amoeba                         -6      24     413   49.2%   47.0%
   9 minic_1.45                    -13      20     413   48.2%   64.4%
  10 Topple_master                 -17      26     412   47.6%   39.8%
  11 rodentIV                      -23      26     413   46.7%   39.7%
  12 minic_1.46                    -24      21     413   46.6%   61.3%
  13 cheng4_linux_x64              -30      25     413   45.6%   43.3%
  14 minic_1.39                    -45      20     413   43.6%   65.4%
  15 minic_1.35                    -66      22     413   40.7%   55.7%
  16 combusken-linux-64            -70      27     413   40.1%   38.0%
  17 zurichess-neuchatel           -74      26     412   39.6%   39.8%
  18 minic_1.19                    -76      23     413   39.2%   51.3%
  19 FabChessv1.13                -102      28     413   35.7%   34.6%

[xr_a_y]

Last "1.51" release is a bit stronger

Code: Select all

   1 demolito                      135      21     733   68.6%   35.3%
   2 igel-last                      66      19     733   59.4%   42.7%
   3 minic_1.51                     13      19     734   51.8%   42.9%
   4 Winter0.7                      10      20     733   51.4%   37.9%
   5 PeSTO_bmi2                      3      20     734   50.5%   35.3%
   6 minic_1.48                    -13      18     734   48.1%   47.4%
   7 minic_1.44                    -26      19     735   46.3%   44.4%
   8 minic_1.35                    -53      18     734   42.4%   47.8%
   9 rodentIV                      -63      20     733   41.1%   39.0%
  10 cheng4_linux_x64              -67      20     735   40.5%   39.3%

[xr_a_y]

New release of Minic (1.54) ! quite stronger than 1.48 thanks to a re-tune of mostly everything

Code: Select all

Score of minic_1.48 vs minic_1.54: 111 - 246 - 501 [0.421]
Elo difference: -55.1 +/- 14.9, LOS: 0.0 %, DrawRatio: 58.4 %

[xr_a_y]

Here's some results of Minic 1.54 versus other at blitz TC

Code: Select all

   1 Defenchess_2.2                230      37     257   79.0%   33.5%
   2 RubiChess                     166      34     256   72.3%   40.6%
   3 demolito                       96      35     256   63.5%   35.5%
   4 igel-last                      35      31     257   55.1%   46.3%
   5 Winter0.7                     -11      34     256   48.4%   35.2%
   6 PeSTO_bmi2                    -16      32     256   47.7%   44.5%
   7 cheng4_linux_x64              -71      34     257   39.9%   38.5%
   8 rodentIV                     -109      34     257   34.8%   37.0%
   9 combusken-linux-64           -143      35     257   30.5%   36.2%

[voffka]

igel-last is Igel 2.4.0?

[xr_a_y]

igel-last is Igel 2.4.0?

git master of yesterday c03d39dfadf7017032a5777ff501f8a6a72f9c9b.

[voffka]

igel-last is Igel 2.4.0?

git master of yesterday c03d39dfadf7017032a5777ff501f8a6a72f9c9b.

Cool. I am curios if you have done the same test against Igel 2.3.1? The purpose is to see relative strength between the two versions of Igel if you have the data.

The 2.4.0 is showing some good numbers against top dogs like Arasan, Rubichess, etc, relative performance increase is currently measured to be between 50 and 80 elo against top dogs, but I am interested against Minic as I am not using it in my testing.

But maybe I should