Maybe an reaction to contempt from the Komodo team!

[Frank Quisinsky]

Hi,

I wrote a lot to test-run results from Stockfish, but never I got an answer. Main problem in Stockfish test-runs are the extremly high draw quote after opening book moves. After my calculation SF lost around 7.5 - 12.0 Elo without contempt vs. weaker engines ... in my FCP Rating List with the opponents I am using. Can be see around start of Mai if I start again an experiment with SF 7.

I think ...
Engines with such an high level Komodo or Stockfish have should get an auto contempt. I am not the maestro in programming questions but I think the idea I wrote before in different messages can not be bad.

Listen:
If after the first 10 own calculated moves (after opening book moves) an engine have many ponder hits the opponent should have around the same level. In this case engine can set automatically contempt to 0.

If after opening book moves lesser ponder hits ... contempt should go high. Should be clear that engines is weaker ... in case we compare with Stockfish or Komodo.

Should work in around 75-90% of all cases (not absolutely sure).

Example in my bad English:

10 book moves, engine started with move 11 ...

Move 11 - 20 ... 4 ponder hits ... contempt should be 10
Move 11 - 20 ... 6 ponder hits ... contempt should be 5
Move 11 - 20 ... 8 ponder hits ... contempt should be 0

Move 21 - 30 ... 18 ponder hits ... contempt should be 10
Move 21 - 30 ... 22 ponder hits ... contempt should be 5
Move 21 - 30 ... 26 ponder hits ... contempt should be 0

The same for move number ...
31 - 40 and so one ...

Means engine should have an own contempt configuration table and can switched automatically to a higher or lesser contempt factor.

So we don't need any contempt configuration because engine do it for us.

And the contempt problem is solved for extremly stronger engines.
Auto Contempt can be set on / off ... in UCI configuration and the engine test World should be OK for the next years ...



Best
Frank

I am sure during the programming the idea can improved a lot. Best ideas programmers have during the programming I think. In my opinion modern engines should set important parameters itself. Engines get more intelligence ... and I will see it in the analyze Windows.

Engine give me the Information:
Contempt changed to 10 before the analyses of the next moves started. Such Information I will see ... engines switched the opinion to parameters itself during the analyses.

[Dirt]

If after the first 10 own calculated moves (after opening book moves) an engine have many ponder hits the opponent should have around the same level. In this case engine can set automatically contempt to 0.

If you can identify an opponent is weak after only ten moves you should almost always win anyway. An ELO rating would be useful, but then it's advantageous to lie about who you are.

[IWB]

Hi,

Main problem in Stockfish test-runs are the extremly high draw quote after opening book moves.
After my calculation SF lost around 7.5 - 12.0 Elo without contempt vs. weaker engines ... in my FCP Rating List with the opponents I am using. Can be see around start of Mai if I start again an experiment with SF 7.

At least in my test SF7 has a similar draw rate as Houdini and is overall below average ... the "problem" can't be that pressing.

I think ...
Engines with such an high level Komodo or Stockfish have should get an auto contempt. I am not the maestro in programming questions but I think the idea I wrote before in different messages can not be bad.

Listen:
If after the first 10 own calculated moves (after opening book moves) an engine have many ponder hits the opponent should have around the same level. In this case engine can set automatically contempt to 0.

If after opening book moves lesser ponder hits ... contempt should go high. Should be clear that engines is weaker ... in case we compare with Stockfish or Komodo.

Should work in around 75-90% of all cases (not absolutely sure).

We disscussed this in the german fora already but for whatever reason you have to post it here again.
A low expected move hit rate (you do not ponder) can be a weaker opponent, or a stronger one! And if you expect to be the best engine it "fires back" as soon as you aren't the best one anymore (Houdini) if you adjust automaticaly for "beeing better" without checking the Eval. Besides that, hit rates during the first 10 moves after very short opning lines can be influenced by forced lines as well.

Your whole concept doesn't work without an eval and the higher the miss rate is the bigger the eval difference should be. If you have any positive or negative eval after a few moves (10 own moves is what you ask for) you don't need the contempt anymore.

Ingo

[yurikvelo]

Means engine should have an own contempt configuration table and can switched automatically to a higher or lesser contempt factor.

For SF to implement anything - it must pass Fishtest.

So you have to modify Fishtest to play not only self-test, but also vs 3rd party

[mjlef]

Frank,

I have thought a bit about how to try and identify an opponent. One thing human players generally know is who they are playing. In the UCI specification (available here: http://www.shredderchess.com/download.html), there is a UCI_Opponent command, with a format like this:

setoption name UCI_Opponent value none none computer Shredder

some of the none can be elo ratings, for example.

But it seems that almost no GUIs support it. So how to guess who your opponent is becomes a kind of puzzle. If you knew say how a program sets it basic search time based on the time input to it, you could time the opponent and make some reasonable guess, since each engine is give the current time and increment for each color. If you know a program's "easy mode" always responded with some specific fraction of its search time, you could use that. But there probably would be too much overlapping of the time control code for that to work well.

So saying this, I have never tried to implement anything. We depend on the human operator to set things like Contempt to appropriate values for a given opponent.

I appreciate you sharing your ideas.

Mark

[IWB]

I have thought a bit about how to try and identify an opponent. One thing human players generally know is who they are playing. In the UCI specification (available here: http://www.shredderchess.com/download.html), there is a UCI_Opponent command, with a format like this:

setoption name UCI_Opponent value none none computer Shredder

some of the none can be elo ratings, for example.

But it seems that almost no GUIs support it. So how to guess who your opponent is becomes a kind of puzzle. ...

Opponents Elo doesn't help either as you do not know where your engine is located in that range. CEGT is using different Elo than CCRL than IPON than ... . (I used 0 as a baseline for a while).
Assuming you start a tourney you get an Elo number without knowing all the others. Is that the No. 1 of your opponents or the number 25, and with the next game you have that problem again?

So, 'we depend on the human operator to set things like Contempt to appropriate values for a given opponent.' (That was a quote )

Ingo

[Hugo]

In general, a have a "mind" problem when running Komodo vs Stockfish.
With default contempt Komodo is playing not that good vs Stockfish than with contempt zero.
But changing parameters for a match seems unnatural to me. It is a mental conflict to me. One side wants to play Komodo as ist best strength, but the other side feels it is a little unfair to change parameters just in one engine and leave Stockfish as default.

Here are 2 results from a match:
100 games, engines 8cpu ponder off, 30+15 Timer
The match that won Komodo was with table memory 256, and contempt 0


regards, Clemens Keck

[JJJ]

In general, a have a "mind" problem when running Komodo vs Stockfish.
With default contempt Komodo is playing not that good vs Stockfish than with contempt zero.
But changing parameters for a match seems unnatural to me. It is a mental conflict to me. One side wants to play Komodo as ist best strength, but the other side feels it is a little unfair to change parameters just in one engine and leave Stockfish as default.

Here are 2 results from a match:
100 games, engines 8cpu ponder off, 30+15 Timer
The match that won Komodo was with table memory 256, and contempt 0


regards, Clemens Keck

Interesting to see you got the same result than Graham in his tournament.
So for now, Komodo and Stockfish seems totally equal.

[Frank Quisinsky]

Hi Jean,

if you compare Komodo 9.42 and Stockfish 7 in a match directly. A completly other Situation if we are looking on results vs. all the other opponents.

Stockfish draw Quote is 4% higher. Komodo won much more games in late middlegame and plays more aggressive vs. weaker engine.

With more and more engine in a Rating List ... with the view from Komodo and Stockfish that all others are clearly weaker ... the Elo difference is bigger and bigger.

I think the different between Komodo 9.42 and Stockfish 7 is here around 40-50 Elo ... can be see if my Rating List is complete (have a look in Agenda ... different matches missing ... stronger vs. weaker). End of April I have it.

End of April I will start a short Experiment ... Stockfish 7 with Contempt 7 and you can see that the different is maybe 20-30 Elo and not 40-50 Elo.

Best
Frank

PS: From my view the result Komodo - Stockfish allone is totoly uninteresting because nothing we can see how strong the engines are. We Need a lot of opponents and all this is indeed hard to see because in the most of Rating tables weaker opponents are missing ... for both engines. With 20 opponents the result is an other as with 40 or 60 opponents.

[JJJ]

Hi, that is interesting to hear. I will watch your rating list as I often do.
Thx.
Also, my name is Jean-Baptiste, not Jean