To analysis.sesse.net owner : disable contempt

[Dann Corbit]

Contempt = both gives the wrong score.

[BeyondCritics]

Let us first gain some intuition what contempt does.
We have static contempt and dynamic contempt, only the former is visible to the user.
Static contempt adds a tiny option value to every middle game part of the score and exactly half of that to the end game part of the score. https://github.com/mcostalba/Stockfish/ ... h.cpp#L344. Thus, this party is discouraged to allow simplifications, since otherwise it can loose up to half of this bonus. If you don't get that, read https://www.chessprogramming.org/Tapered_Eval first.
Dynamic contempt does the same for both players, but the extent depends on the root value of the previous iteration, nearly linear (partial score derivation roughly 11/25) for small values and converging to roughly 3/4 of a middle game pawn for big values. See https://github.com/mcostalba/Stockfish/ ... h.cpp#L396. Thus, the one with an advantage is always discouraged, to go into simplifications. That makes somehow sense to me either. Since if you are at a positional disadvantage, going into simplifications should help more often than not.

The implementation still silently resets user contempt values out of the "allowed" range {100,..100} to the default value, which i consider as unintuitive.
Apart from that i see no real reason to complain.They test their patches rigorously, and that must mean that contempt does no harm against equal opponents. It helps a lot against weaker opponents.
Of course, they can not test how informative the score for humans is. I have skimmed analysis at http://analysis.sesse.net/ the last game 8 and in deed it seems to wiggle a a lot more than it should in the opening.

[Dann Corbit]

Removal of contempt is only valuable for analysis, not for game play.
When I am analyzing, I want the best possible score, and assuming equal rank to the engine that is analyzing.
If the engine adds a quarter pawn, then it is a quarter pawn lie.
I do not want my analyzing engine to lie.

When it is playing a game of chess against someone else, I could care less what it is doing with the score. I can lie all it likes.

[Sesse]

Hi,

First of all: If you wish to get in contact with me, Talkchess or any other web forum is a suboptimal way. There's contact information on my web page (linked from the bottom of analysis.sesse.net)—I only saw this thread because someone picked it up and sent me a message on Reddit (!), which I also don't typically poll, but happened to visit the other day.

As for the request itself, I immediately dismissed it out of hand; surely, I hadn't set up contempt! But no, seemingly in April, Stockfish changed this to be default (and 21 centipawns at that) also in analysis mode—I usually read the commit logs when upgrading, but somehow, this one slipped by. So indeed, running with the default setting causes this rollercoaster problem, especially in the opening.

I always interpreted contempt as the players' natural dislike for a draw, ie., contempt of 21 would mean I would rather have a -0.20 with play on the board than a three-fold repetition (or stalemate etc.) on the board. Normally I would assume that this simply means that a repetition is scored at -0.21 instead of the usual 0.00. However, this isn't how Stockfish has chosen to implement it—instead, it just adds 0.21 for the side-to-move (at the root; it doesn't switch throughout the search tree), or half that in the endgame. For match play, these are obviously equivalent (and almost all engine testing is done using match play, of course), but for analysis, it causes this rollercoaster effect that I had observed but assumed was due to horizon effects.

The Stockfish team does an amazing job, and for free at that, but I'm not all that happy at this particular decision. If the score were subtracted away again in the final result, it would be different, but I assume it would need to traverse the PV again to figure out whether to subtract 0.21, 0.115 or something in-between. Of course, any kind of contempt will cause some form of oscillation, but it can't be much worse than between 0.21 and 0.00 even in the extreme case, so I'm not that worried. In any case, I've set Analysis Contempt to “Off” now, at the risk of more boring repetition lines being showed; thanks for alerting me to this. Maybe I'll write a patch at some point to try to subtract it back, which I guess is easier than changing how a draw is scored? I haven't looked deeply into it.

As for dynamic contempt, it feels natural to me that the side currently ahead is less interested in a draw, and I can't see that it would cause any oscillations (at most, the side in the lead would get slightly exaggerated scores, which is mostly just a reinterpretation of what 1.00 means), so I'm not patching it out, at least not for the time being.

[syzygy]

But I guess it's hopeless to convince people to fix this for Stockfish 10?

Yes, it is hopeless. People perceive it as some sort of magic and they like to believe in magic.

It's very annoying because of course every application designed around Stockfish inevitably gets it wrong.

In practice, it may be.
In theory, the application developer can either set Contempt to zero or Analysis Contempt to Off. Or just use Cfish

[syzygy]

Apart from that i see no real reason to complain.They test their patches rigorously, and that must mean that contempt does no harm against equal opponents. It helps a lot against weaker opponents.
Of course, they can not test how informative the score for humans is. I have skimmed analysis at http://analysis.sesse.net/ the last game 8 and in deed it seems to wiggle a a lot more than it should in the opening.

The complaint is about the default settings for analysis.

Why does the wiggling come as a surprise to you? It is the direct result of the contempt implementation (and of leaving it on during analysis).

[BeyondCritics]

Apart from that i see no real reason to complain.They test their patches rigorously, and that must mean that contempt does no harm against equal opponents. It helps a lot against weaker opponents.
Of course, they can not test how informative the score for humans is. I have skimmed analysis at http://analysis.sesse.net/ the last game 8 and in deed it seems to wiggle a a lot more than it should in the opening.

The complaint is about the default settings for analysis.

Why does the wiggling come as a surprise to you? It is the direct result of the contempt implementation (and of leaving it on during analysis).

From where do you have that i am surpised from that? I just confirmed it, as an empirical fact. And score is not everything to me.
If you are analyzing with contempt, you put pressure on moves, that allow simplification. That should lead to a lower score where stockfish is deprived of its defences and to a more reliable or even better score where stockfish finds good moves earlier or at all. The reverse for the other side. There are positions known, where simplifications are really needed for one side. In such a scenarion contempt should hurt analysis.
Testing shows surprisingly, that overall the results cancel each other.
Therefore my very wild up front expectation would be, that the scores should have a higher variance, but should still be unbiased.
You as an experienced chess programmer, may have a deeper understanding, so please explain.

[syzygy]

Apart from that i see no real reason to complain.They test their patches rigorously, and that must mean that contempt does no harm against equal opponents. It helps a lot against weaker opponents.
Of course, they can not test how informative the score for humans is. I have skimmed analysis at http://analysis.sesse.net/ the last game 8 and in deed it seems to wiggle a a lot more than it should in the opening.

The complaint is about the default settings for analysis.

Why does the wiggling come as a surprise to you? It is the direct result of the contempt implementation (and of leaving it on during analysis).

From where do you have that i am surpised from that? I just confirmed it, as an empirical fact. And score is not everything to me.

But the explanation that you then give is way too complicated.

The explanation for the wiggling is very simple: if white is to move at the root, white is given a contempt bonus (and, equivalently, black is given a contempt penalty) throughout the tree, so white's score is too high. If black is to move at the root, black is given a contempt bonus (and, equivalently, white is given a contempt penalty) throughout the tree, so black's score is too high (white's score is too low).

So if contempt is +0.20 and the objective score is about +1.00 for white, you'll see about +1.20 if white is to move at the root and about +0.80 if black is to move at the root. Each time Carlsen or Caruana plays a move, the side to move at the root switches and the score wiggles.

What you seem to be thinking is that the bonus/penalty is applied in each internal node of the tree with a sign that depends on whether white or black is to move at that node. But that is not the case: the sign depends only on whether white or black happens to be to move at the root.

Testing shows surprisingly, that overall the results cancel each other.

This has nothing to do with testing. When test games are played, it is either always white to move at the root (if the engine has white) or always black to move at the root (if the engine has black). There is no wiggling during testing!

[BeyondCritics]

The explanation for the wiggling is very simple: if white is to move at the root, white is given a contempt bonus (and, equivalently, black is given a contempt penalty) throughout the tree, so white's score is too high. If black is to move at the root, black is given a contempt bonus (and, equivalently, white is given a contempt penalty) throughout the tree, so black's score is too high (white's score is too low).

...

So if contempt is +0.20 and the objective score is about +1.00 for white, you'll see about +1.20 if white is to move at the root and about +0.80 if black is to move at the root. Each time Carlsen or Caruana plays a move, the side to move at the root switches and the score wiggles.

This is all wrong and you haven't understood contempt correctly. Contempt unevenly shifts the weights to the "mg" and "eg" components of the static evaluation of all leave nodes. Doing this you get typically a completely different search and even if the same node value would make it to the top, you can not simply subtract contempt to get the normal score. Please read one of my further postings, where i refer to the relevant source code lines in stockfish.

[syzygy]

The explanation for the wiggling is very simple: if white is to move at the root, white is given a contempt bonus (and, equivalently, black is given a contempt penalty) throughout the tree, so white's score is too high. If black is to move at the root, black is given a contempt bonus (and, equivalently, white is given a contempt penalty) throughout the tree, so black's score is too high (white's score is too low).

...

So if contempt is +0.20 and the objective score is about +1.00 for white, you'll see about +1.20 if white is to move at the root and about +0.80 if black is to move at the root. Each time Carlsen or Caruana plays a move, the side to move at the root switches and the score wiggles.

This is all wrong and you haven't understood contempt correctly.

This is an approximation, which is why I wrote "about".

If white is to move at the root, all scores throughout the tree are increased for white by "about" the contempt value. Therefore, the reported score is skewed in white's favour by "about" the contempt value.

If black is to move at the root, the same applies with colors reversed. So the reported score is skewed in black's favour by "about" the contempt value.

This trivially explains the wiggling that is obvious for all to see.

So now please read again what I wrote. And maybe you should not assume that I don't know how this stuff works.