Stockfish with and without contempt

[corres]

The new Stockfish uses a modified contempt system so I made a match between two version of
Stockkfish 191102: one version has the default contempt and the other version has no any contempt.
PC: Threadripper 2950X 16 X 4 GHz
OS: Windows 10 64 bits
Hash: 2048 MB
Endgame DB: 6 men syzygy and 5 men Nalimov (only for Fritz chess GUI)
Time: 1 min + 2 sec / move
Result:
Stockfish with Contempt : Stockfish without Contempt = 8 : 5 (87 draw) 2 x 50 = 100 games
from 50 opening positions with alternated colors.
It seems to me the new contempt gives some benefit in exchange for the less stable evaluation.

[Dann Corbit]

Mathematically, it takes a lot of games to know that a change is good.

[MikeB]

The new Stockfish uses a modified contempt system so I made a match between two version of
Stockkfish 191102: one version has the default contempt and the other version has no any contempt.
PC: Threadripper 2950X 16 X 4 GHz
OS: Windows 10 64 bits
Hash: 2048 MB
Endgame DB: 6 men syzygy and 5 men Nalimov (only for Fritz chess GUI)
Time: 1 min + 2 sec / move
Result:
Stockfish with Contempt : Stockfish without Contempt = 8 : 5 (87 draw) 2 x 50 = 100 games
from 50 opening positions with alternated colors.
It seems to me the new contempt gives some benefit in exchange for the less stable evaluation.

I realized after having a discussion with some members of the SF development team, they think of contempt as an extension of the evaluation - not as a "draw score". Back in the day , contempt ws noting more than a draw score. One raise the draw score to ensure that your engine would not take an early draw, or use negative draw score, if one wanted their engine to take draw a draw as early as possible. When I submitted a patch with a different contempt values for white and black, I was reprimanded for making white and black evaluations asymmetrical - it was then I realized they have a completely different mindset about how and what contempt is. They see it as part of the evaluation and why would anybody want to mess with it or make it asymmetrical , whereas for those of us that are long in the tooth and been around for a while, we see contempt as nothing more than revising the draw score. On the other hand, when you really study the Stockfish code - contempt is nothing about a draw score , it's about scoring for the initiative, so in that sense , what they are saying makes sense. What they call contempt, should be labled "initiative", left alone and should be symmetrical. I plan on introducing a new UCI option called "Draw_Score" in the next release of Honey and may remove any option to adjust contempt at all since it is not really 'contempt' or related to drawscore , it is really just a poor label for what the code is actually doing. This may be more of generational difference than anything else. As evidence of this, if you use a negative contempt, SF plays worse and the draw ratio does not go up at all, which is contrary to how a draw score should impact the results. It is not a draw score , it is not how the contempt factor was constructed or used in the early days of computer chess at all.

https://www.chessprogramming.org/Contempt_Factor

Edit: I will not remove contempt, but rename it as initiative or something similar so it's more clear.

[Michel]

If you think of the static eval as an indication of the expected score against an equal opponent then it is logical to add some adjustment if the opponent is not of the same strength. This adjustment should only be made for non-hard scores (e.g. not for DRAWSCORE). This is the philosophy behind SF's contempt implementation and it is sound. It is has been shown in testing to really amplify elo differences. Although I did not originally like it, this also justifies the use of contempt on fishtest (again to amplify elo differences).

I am not a fan of dynamic contempt which is a different animal and which I believe to be just noise. It is supposed to bring a few elo but I could not duplicate this is in private testing. But even if it brings a few elo then I still believe it compensates in some way for a defect in the static eval. It would be better to fix this, instead of messing with the principles of A/B search.

[corres]

...
I realized after having a discussion with some members of the SF development team, they think of contempt as an extension of the evaluation - not as a "draw score". Back in the day , contempt ws noting more than a draw score. One raise the draw score to ensure that your engine would not take an early draw, or use negative draw score, if one wanted their engine to take draw a draw as early as possible. When I submitted a patch with a different contempt values for white and black, I was reprimanded for making white and black evaluations asymmetrical - it was then I realized they have a completely different mindset about how and what contempt is. They see it as part of the evaluation and why would anybody want to mess with it or make it asymmetrical , whereas for those of us that are long in the tooth and been around for a while, we see contempt as nothing more than revising the draw score. On the other hand, when you really study the Stockfish code - contempt is nothing about a draw score , it's about scoring for the initiative, so in that sense , what they are saying makes sense. What they call contempt, should be labled "initiative", left alone and should be symmetrical. I plan on introducing a new UCI option called "Draw_Score" in the next release of Honey and may remove any option to adjust contempt at all since it is not really 'contempt' or related to drawscore , it is really just a poor label for what the code is actually doing. This may be more of generational difference than anything else. As evidence of this, if you use a negative contempt, SF plays worse and the draw ratio does not go up at all, which is contrary to how a draw score should impact the results. It is not a draw score , it is not how the contempt factor was constructed or used in the early days of computer chess at all.
https://www.chessprogramming.org/Contempt_Factor
Edit: I will not remove contempt, but rename it as initiative or something similar so it's more clear.

I think the effect of Static Contempt (that is the "classical contempt") depend on the structure of search and the measure of Static Contempt as well as the concrete value of evaluation itself. If the search do not contain barriers for value given by search the Static Contempt works as an offset for value of evaluation and it has an effect on the tendency of draw only. If there are barriers (pruning based on value, "Singular extension search", search parameters what depend on value, etc ) the Static Contempt has effects on the effectiveness of search too.
Really, the Static Contempt has effect on a kind of initiativeness. But in general being initiative is a method to handle equalized positions.

[corres]

If you think of the static eval as an indication of the expected score against an equal opponent then it is logical to add some adjustment if the opponent is not of the same strength. This adjustment should only be made for non-hard scores (e.g. not for DRAWSCORE). This is the philosophy behind SF's contempt implementation and it is sound. It is has been shown in testing to really amplify elo differences. Although I did not originally like it, this also justifies the use of contempt on fishtest (again to amplify elo differences).
I am not a fan of dynamic contempt which is a different animal and which I believe to be just noise. It is supposed to bring a few elo but I could not duplicate this is in private testing. But even if it brings a few elo then I still believe it compensates in some way for a defect in the static eval. It would be better to fix this, instead of messing with the principles of A/B search.

I agree you.
If we study the new "Dynamic Contempt" of Stockfish:
int dct = ct + (111 - ct / 2) * previousScore / abs(previousScore) + 176;
where dct = "Full Contempt" and ct = "Static Contempt"
obviously it contains only one real dynamic element: previousScore / abs(previousScore) what has only two value +1 or -1 depending of the sign of previousScore.
So if Stockfish stands better the "Full Contempt" behaves like a "Static Contempt" with + sign and if Stockfish stands weaker the "Full Contempt" behaves like a "Static Contempt" with - sign.
Maybe it would be better if we could change differently the "Full Contempt" with + sign and - sign.

[Deberger]

On the other hand, when you really study the Stockfish code - contempt is nothing about a draw score , it's about scoring for the initiative, so in that sense , what they are saying makes sense. What they call contempt, should be labeled "initiative", left alone and should be symmetrical.

Does this mean that people can finally stop advocating Contempt = 0 for analysis mode?

[Leo]

The new Stockfish uses a modified contempt system so I made a match between two version of
Stockkfish 191102: one version has the default contempt and the other version has no any contempt.
PC: Threadripper 2950X 16 X 4 GHz
OS: Windows 10 64 bits
Hash: 2048 MB
Endgame DB: 6 men syzygy and 5 men Nalimov (only for Fritz chess GUI)
Time: 1 min + 2 sec / move
Result:
Stockfish with Contempt : Stockfish without Contempt = 8 : 5 (87 draw) 2 x 50 = 100 games
from 50 opening positions with alternated colors.
It seems to me the new contempt gives some benefit in exchange for the less stable evaluation.

Interesting test.

[Leo]

On the other hand, when you really study the Stockfish code - contempt is nothing about a draw score , it's about scoring for the initiative, so in that sense , what they are saying makes sense. What they call contempt, should be labeled "initiative", left alone and should be symmetrical.

Does this mean that people can finally stop advocating Contempt = 0 for analysis mode?

Yes.

[MikeB]

On the other hand, when you really study the Stockfish code - contempt is nothing about a draw score , it's about scoring for the initiative, so in that sense , what they are saying makes sense. What they call contempt, should be labeled "initiative", left alone and should be symmetrical.

Does this mean that people can finally stop advocating Contempt = 0 for analysis mode?

That is my opinion - and the way contempt is implemented in SF - it is color neutral , meaning the evaluations are totally symmetrical - so there is no reason to have the off black white and on settings. They are there because it was requested by the user community - but by and large most of the developers would never use those settings period.
I have totally flip flopped on the use of contempt once I understood the mechanics on how it was implemented in SF. It’s complicated and some people may still prefer not to use it and that’s fine or use it in a limited way - that’s fine and users can are free to use it anyway they want.

Anecdotally, I introduced Draw Score to Honey last evening and with just using a very minor negative draw score , these are the results far - keep in mind that without draw score - Honey and Stockfish are almost equal - with a slight nod to Stockfish perhaps.

Code: Select all

336 of 600 games completed...
Time control: 60 seconds + 2.0000 seconds
Threads: 1
Hash: 256
Date: 11/06/19 : 12:21:35
Rank Name              Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 Honey-X6a-DR       1461   0.0   20   20   224  117.5  52.5   29   18  177  12.9  79.0  1444 
   2 Stockfish-110619   1446  15.3   20   20   224  110.0  49.1   21   25  178   9.4  79.5  1452 
   3 Honey X6a          1443   3.0   20   20   224  108.5  48.4   18   25  181   8.0  80.8  1454 
---------------------------------------------------------------------------------------------------------

The Honey DR has draw score enabled with a very minor value - of minus 16 (-16). 16 was converted is converted to end game pawn score - so it represents about 1/6 of the value of end game pawn.

Of course this is just a few games and statistically means nothing - if I ran it long enough , it might totally flip flop - but interesting nonetheless.