Page 2 of 4
Re: Stockfish with and without contempt
Posted: Wed Nov 06, 2019 8:27 pm
by Michel
That is my opinion - and the way contempt is implemented in SF - it is color neutral , meaning the evaluations are totally symmetrical - so there is no reason to have the off black white and on settings. They are there because it was requested by the user community - but by and large most of the developers would never use those settings period.
The options were introduced because in back and forth analysis you do not want the evaluation to depend on the side to move at root. This creates hash table inconsistencies. This was reported by upset users.
Contempt (in analysis) for one side and the opposite contempt for the other side does not suffer from this and may even be helpful if you are looking for a win (positive contempt) or a draw (negative contempt).
In game play, against an equal opponent, any contempt should be detrimental (although testing shows that a little bit of contempt does not hurt).
Re: Stockfish with and without contempt
Posted: Wed Nov 06, 2019 9:12 pm
by MikeB
Michel wrote: ↑Wed Nov 06, 2019 8:27 pm
That is my opinion - and the way contempt is implemented in SF - it is color neutral , meaning the evaluations are totally symmetrical - so there is no reason to have the off black white and on settings. They are there because it was requested by the user community - but by and large most of the developers would never use those settings period.
The options were introduced because in back and forth analysis you do not want the evaluation to depend on the side to move at root. This creates hash table inconsistencies. This was reported by upset users.
Contempt (in analysis) for one side and the opposite contempt for the other side does not suffer from this and may even be helpful if you are looking for a win (positive contempt) or a draw (negative contempt).
In game play, against an equal opponent, any contempt should be detrimental (although testing shows that a little bit of contempt does not hurt).
I agree that is how historically contempt would be expected to work , which is more inline with the draw score method , but that is not how contempt works in SF. Currently, the defaults contempt setting for SF are the best settings when going against an engine similar to SF in strength and the SF development team have repeatedly demonstrated that through their testing.
This is how one would expect contempt or draw score function to work - note the higher draws and , the fewer wins and fewer losses - in his case it it appears to be testing out slightly higher - but roughly equal.
The DR version has a draw score component included.
Code: Select all
Time to complete: 06:13:07
100 rounds and 600 games completed...
Time control: 60 + 2.0000 seconds
Threads = 1
Hash = 256
Date: 11/06/19 : 15:03:53
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
---------------------------------------------------------------------------------------------------------
1 Honey-X6a-DR 1454 0.0 16 16 400 203.0 50.7 45 39 316 11.2 79.0 1448
2 Honey X6a 1452 1.2 16 16 400 202.0 50.5 47 43 310 11.8 77.5 1449
3 Stockfish-110619 1444 8.1 16 16 400 195.0 48.8 42 52 306 10.5 76.5 1453
---------------------------------------------------------------------------------------------------------
[\code]
Re: Stockfish with and without contempt
Posted: Wed Nov 06, 2019 9:39 pm
by Dann Corbit
Ideally, nonzero contempt should mean to play more aggressively, but not to change the score at all.
If it does that, then it is not problematic.
But if there is any bias introduced, it is very, very bad for my purposes.
One might ask the question, "Why not just always play aggressively?"
Sure, it might make you lose against a better player. But it also might make you win against them.
*shrug*
Re: Stockfish with and without contempt
Posted: Wed Nov 06, 2019 11:24 pm
by MikeB
Dann Corbit wrote: ↑Wed Nov 06, 2019 9:39 pm
Ideally, nonzero contempt should mean to play more aggressively, but not to change the score at all.
If it does that, then it is not problematic.
But if there is any bias introduced, it is very, very bad for my purposes.
<snip>
Right before the pv is spitted out , we can deduct the contempt component of the total score. No different than how I do score percentage - wait until the very end. . I might add that into tonight to see what it looks like. Might be interesting - so you still get best move , just unpolluted evaluation. And some people will say the total score with contempt is correct - but the point is we can give people a choice.
Re: Stockfish with and without contempt
Posted: Wed Nov 06, 2019 11:28 pm
by jhellis3
I really don't think it can be quite as bad as you make it out to be (if used reasonably).
The eval of any engine is already uncertain (other than hard cutoffs). So if one has uncertain nodes that eval as draw, they could just as well not be. Which is why SF's evaluate.cpp has received many patches over the years. Thus even if there is bias in the search, I would submit that it is indistinguishable from some other "pure" search with slightly different parameters or potentially even the noise of a multi-thread search with various hash sizes.
The question is where does one draw the line... Is 1 cp really going to matter... no. 10cp, not really.... 20cp, perhaps but still not a huge issue.... 50cp+, ok there might be a problem. So it seems to me your real problem with SF currently is the dynamic contempt it applies because it pushes the fairly reasonable 24 cp base contempt into dubious territory.
You should probably just disable dynamic contempt entirely, reduce base a bit if you feel 24 is too high and be done with it. That will likely give you the results you are after much sooner than a perfect eval for SF or any other engine comes into existence.
Re: Stockfish with and without contempt
Posted: Wed Nov 06, 2019 11:40 pm
by Dann Corbit
jhellis3 wrote: ↑Wed Nov 06, 2019 11:28 pm
It seems to me that you should just disable dynamic contempt entirely, reduce base a bit if you feel 24 is too high and be done with it. That will likely give you the results you are after much sooner than a perfect eval for SF or any other engine comes into existence.
That is exactly what I do, every time there is a new release.
I do rue the fact that it may play more passively in equal positions, but I need the correct score.
Re: Stockfish with and without contempt
Posted: Wed Nov 06, 2019 11:48 pm
by jhellis3
If you create a git branch with the change, you can just pull master into it and it will auto merge as long as no changes have been made to the contempt code upstream.
but I need the correct score.
Well, the chances you aren't getting
that are about the same regardless.
Re: Stockfish with and without contempt
Posted: Wed Nov 06, 2019 11:55 pm
by Dann Corbit
jhellis3 wrote: ↑Wed Nov 06, 2019 11:48 pm
If you create a git branch with the change, you can just pull master into it and it will auto merge as long as no changes have been made to the contempt code upstream.
but I need the correct score.
Well, the chances you aren't getting
that are about the same regardless.
Let's call it an unbiased score then.
Re: Stockfish with and without contempt
Posted: Thu Nov 07, 2019 2:12 am
by corres
corres wrote: ↑Wed Nov 06, 2019 12:43 pm
...
If we study the new "Dynamic Contempt" of Stockfish:
int dct = ct + (111 - ct / 2) * previousScore / abs(previousScore) + 176;
where dct = "Full Contempt" and ct = "Static Contempt"
obviously it contains only one real dynamic element: previousScore / abs(previousScore) what has only two value +1 or -1 depending of the sign of previousScore.
So if Stockfish stands better the "Full Contempt" behaves like a "Static Contempt" with + sign and if Stockfish stands weaker the "Full Contempt" behaves like a "Static Contempt" with - sign.
Maybe it would be better if we could change differently the "Full Contempt" with + sign and - sign.
Sorry, but I made an error writing down the "Full Contempt" formula of Stockfish.
The exact is:
int dct = ct + (111 - ct / 2) * previousScore / (abs(previousScore) + 176);
So the dynamic component is not +1 or -1 but it may change between (near) +1 and (near) -1 depending on the value of previousScore: As the previousScore is higher as the dynamic factor is nearer to +1 or -1.
When the previousScore is 0 the "Full Contempt" = "Static Contempt".
If the previousScore is a positive value the (dynamic component ) * (111 - ct / 2) is added to the "Static Contempt" - that is the "Full Contempt" enhanced and Stockfish will be more aggressive.
If the previousScore is a negative value the (dynamic component) * (111 - ct / 2) is subtracted from the "Static Contempt" - that is the "Full Contempt" decreased and Stockfish will be less aggressive.
Re: Stockfish with and without contempt
Posted: Thu Nov 07, 2019 2:25 am
by corres
Dann Corbit wrote: ↑Wed Nov 06, 2019 9:39 pm
Ideally, nonzero contempt should mean to play more aggressively, but not to change the score at all.
If it does that, then it is not problematic.
But if there is any bias introduced, it is very, very bad for my purposes.
...
I agree you.
For analysis purpose it is more important the exact evaluation than the aggressiveness with uncertain result. If the evaluation of Stockfish is stable one can choose more easily the right continuation.
In engine-engine matches it is not a viewpoint, obviously.