SF130118

[APassionForCriminalJustic]

At fishtest C=20 had shown 0 measurable regression, so it probably should not result in any meaningful regression vs houdini or komodo (also it didn't in some 1000-games samples of other testers).
Right now it scores slightly more vs houdini and slightly less vs komodo but I guess it's just a noise. But it for sure helps vs weaker engines a lot.

So far 60 percent versus Houdini CT 20 certainly not regression fitting. Clearly there is absolutely no boost performance for the top tier engines. Stockfish has always been the strongest engine. It was only half a point from making TCEC superfinal. With contempt 20 it would have more than likely made it. It is great to see the fish right where it should be. Houdini already faltering in January is bad news since a new release won't be for close to another year at best.

[Eelco de Groot]

At fishtest C=20 had shown 0 measurable regression, so it probably should not result in any meaningful regression vs houdini or komodo (also it didn't in some 1000-games samples of other testers).
Right now it scores slightly more vs houdini and slightly less vs komodo but I guess it's just a noise. But it for sure helps vs weaker engines a lot.

I had not expected better results against Houdini and Komodo and against Komodo it is still slightly less, but as you and Ingo and others say, the subresults will likely be well within the statistical margins because of small samplesize of any individual matches. I agree with your assessment. Very nice indeed and thanks for running it Ingo!

I think probably contempt is not hurting analysis function, in my own very limited testing with Kaissa. In the past Stockfish had asymmetrical King Safety also when in analysis mode after Marco had scrapped UCI_AnalyseMode in Stockfish. So at the moment I see no need to reinstate UCI_AnalyseMode support.

[Ovyron]

I think probably contempt is not hurting analysis function

Contempt has never hurt analysis, I think it was Vas that was the first to introduce change of analysis behavior with Contempt setting (before, engines would just have "draw value" being changed), and it was useful in analysis to increase it (very often, Rybka 3 would find the best move in record time with a higher Contempt.) Though engines have used similar concepts before (like Rebel's Anti-GM, now Confusion.)

I have resorted to using exclusively Contempt for analysis for Stockfish, Komodo and Houdini, Contempt is nothing more than asymetrical analysis (as when you turn it on, the engine prefers its own side), engines now seem dumb without it, choosing boring variations that lead nowhere.

[JJJ]

So contempt adds elo against all engines after all. Well this season Stockfish goes to superfinal at full power

[IWB]

I still do not like contempt (reasons above) but as all top 3 Engines decided for it - and obviously with success! - it seems my opinion is of no avail.

Code: Select all

   # PLAYER                        : RATING  ERROR     (%)    D(%)  OppAvg   CFS(next)    POINTS       W       D       L  PLAYED
   1 Stockfish 230118 64 POPCNT    :   3382     11   83.3%    28.8    3073     100        2748.5    2273     951      76    3300
   2 Houdini 6.02                  :   3351     10   80.7%    31.7    3076     100        2661.5    2139    1045     116    3300
   3 Komodo 11.2.2                 :   3324     10   78.2%    34.0    3077     100        2580.5    2019    1123     158    3300
   4 Shredder 13                   :   3126      8   55.3%    50.2    3091     100        1825.0     997    1656     647    3300
   5 Fizbo 2                       :   3097      8   51.5%    40.5    3092     100        1698.0    1029    1338     933    3300
   6 Ginkgo 2.0                    :   3070      8   47.9%    50.0    3094      93        1579.5     754    1651     895    3300
   7 Gull 3                        :   3061      8   46.7%    46.0    3095      91        1540.5     781    1519    1000    3300
   8 Andscacs 0.92                 :   3053      8   45.6%    44.7    3095     100        1504.0     766    1476    1058    3300
   9 Booot 6.2                     :   3031      8   42.7%    48.9    3097      67        1408.5     601    1615    1084    3300
  10 Jonny 8.00                    :   3029      9   42.3%    46.0    3097      90        1396.5     637    1519    1144    3300
  11 Fritz 16                      :   3021      8   41.3%    45.9    3098      79        1361.5     604    1515    1181    3300
  12 Equinox 3.30                  :   3016      8   40.6%    47.1    3098     100        1339.5     563    1553    1184    3300
  13 Chiron 4                      :   2996      8   38.0%    45.4    3099      70        1255.0     506    1498    1296    3300
  14 Critter 1.6a                  :   2993      8   37.6%    44.9    3099     100        1241.0     500    1482    1318    3300
  15 Nirvanachess 2.4              :   2972      8   34.9%    44.2    3101      96        1152.0     422    1460    1418    3300
  16 Hannibal 1.7                  :   2962      9   33.6%    43.2    3101     ---        1108.5     396    1425    1479    3300

No Contempt:

Code: Select all

   # PLAYER                        : RATING  ERROR     (%)    D(%)  OppAvg   CFS(next)    POINTS       W       D       L  PLAYED
   1 Stockfish 130118 64 POPCNT    :   3349     10   81.0%    35.0    3072      66        2672.0    2095    1154      51    3300
   2 Houdini 6.02                  :   3346     10   80.7%    32.1    3072     100        2663.0    2133    1060     107    3300
   3 Komodo 11.2.2                 :   3317     10   78.0%    34.2    3074     100        2575.0    2010    1130     160    3300
   4 Shredder 13                   :   3126      8   55.8%    51.2    3086     100        1840.5     995    1691     614    3300
   5 Fizbo 2                       :   3097      8   51.9%    41.4    3088     100        1711.5    1029    1365     906    3300
   6 Ginkgo 2.0                    :   3068      8   48.0%    50.2    3090      93        1583.5     755    1657     888    3300
   7 Gull 3                        :   3060      8   46.8%    46.4    3091      97        1546.0     780    1532     988    3300
   8 Andscacs 0.92                 :   3049      8   45.3%    44.3    3092     100        1496.5     765    1463    1072    3300
   9 Booot 6.2                     :   3031      8   43.0%    49.6    3093      74        1420.0     601    1638    1061    3300
  10 Jonny 8.00                    :   3028      8   42.5%    46.7    3093      92        1403.0     633    1540    1127    3300
  11 Fritz 16                      :   3019      8   41.4%    46.1    3094      75        1365.0     604    1522    1174    3300
  12 Equinox 3.30                  :   3015      8   40.8%    47.6    3094     100        1346.5     561    1571    1168    3300
  13 Chiron 4                      :   2996      8   38.3%    45.8    3095      66        1262.5     506    1513    1281    3300
  14 Critter 1.6a                  :   2993      8   37.9%    45.5    3095     100        1251.5     501    1501    1298    3300
  15 Nirvanachess 2.4              :   2970      9   35.0%    44.4    3097      95        1154.0     422    1464    1414    3300
  16 Hannibal 1.7                  :   2960      8   33.6%    43.4    3098     ---        1109.5     394    1431    1475    3300

SF8:

Code: Select all

   # PLAYER              : RATING  ERROR     (%)    D(%)  OppAvg   CFS(next)    POINTS       W       D       L  PLAYED
   1 Houdini 6.02        :   3349     10   81.4%    32.4    3068     100        2687.0    2152    1070      78    3300
   2 Komodo 11.2.2       :   3318     10   78.5%    34.8    3070      99        2591.5    2017    1149     134    3300
   3 Stockfish 8         :   3301      9   76.9%    40.3    3071     100        2538.5    1873    1331      96    3300
   4 Shredder 13         :   3124      8   55.8%    51.2    3083     100        1842.5     997    1691     612    3300
   5 Fizbo 2             :   3096      8   52.1%    41.7    3085     100        1719.5    1032    1375     893    3300
   6 Ginkgo 2.0          :   3066      8   47.9%    50.1    3087      81        1582.0     756    1652     892    3300
   7 Gull 3              :   3060      8   47.2%    46.9    3087      95        1557.5     783    1549     968    3300
   8 Andscacs 0.92       :   3050      8   45.8%    45.2    3088     100        1512.5     766    1493    1041    3300
   9 Booot 6.2           :   3031      8   43.2%    49.9    3089      69        1425.5     602    1647    1051    3300
  10 Jonny 8.00          :   3028      8   42.8%    47.0    3090      90        1412.0     637    1550    1113    3300
  11 Fritz 16            :   3020      8   41.7%    46.7    3090      79        1376.5     606    1541    1153    3300
  12 Equinox 3.30        :   3014      8   41.0%    47.9    3091     100        1354.0     563    1582    1155    3300
  13 Chiron 4            :   2994      8   38.3%    45.8    3092      55        1263.5     507    1513    1280    3300
  14 Critter 1.6a        :   2993      8   38.2%    46.1    3092     100        1260.0     500    1520    1280    3300
  15 Nirvanachess 2.4    :   2969      8   35.0%    44.5    3094      90        1156.0     422    1468    1410    3300
  16 Hannibal 1.7        :   2961      8   34.0%    44.1    3094     ---        1121.5     394    1455    1451    3300

[Vizvezdenec]

Well, stockfish contempt behaves in pretty strange way - it for some reason doesn't lose elo vs equal opponents till you set it as 25 or more. But it helps to gain more points vs weaker engines.
Basically C=20 is just more agressive stockfish w/o any real drawbakcs (well, maybe it loses elo vs stronger opponents... But there are none of them for SF ).

[carldaman]

Well, stockfish contempt behaves in pretty strange way - it for some reason doesn't lose elo vs equal opponents till you set it as 25 or more. But it helps to gain more points vs weaker engines.
Basically C=20 is just more agressive stockfish w/o any real drawbakcs (well, maybe it loses elo vs stronger opponents... But there are none of them for SF ).

Stronger opponents can easily be found if you let SF on 1 core play the other top engines on 4 cores, for example.

regards,
CL