LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

[Hai]

Both 8 GB RAM and all 3-6 piece Syzygybases.
The 20 possible opening moves = 40 games.

LC0 vs Stockfish with only 3 cores (5000 kn/s):
Contempt +25
+17 =20 -3
Winning percentage = 67.5%
=127 elo difference

Contempt -25
+9 =30 -1
Winning percentage = 60%
=70 elo difference

Contempt -50
+10 =27 -3
Winning percentage = 58.75%
=61 elo difference

Contempt -75
+6 =34 -0
Winning percentage = 57.5%
=53 elo difference -> with 3 cores vs 2x RTX 2080 Ti

Doubling cpu cores from 3 to 6 cores should give Stockfish with contempt -75 easily an 53 elo improvement, which means the elo difference is 0.
I don't know how good the elo scaling of Stockfish is with a few cores, but I expect it to be much better than with many cores.

Contempt -75 instead of +25 leads to a 75 elo improvement.
Comparing Stockfish lost only 3 wins using contempt -75 but LC0 lost 11 wins. That means LC0 lost nearly 4x more of his wins than Stockfish.

It would be interesting to test Stockfish up to contempt -200 and learn more about the new playing style/improvements.
And also to have the option to display always contempt 0 in the evaluation, while using contempt -75, would be interesting.

Feel free to do lots of your own different contempt tests.

Stockfish could have won the TCEC easily.

[Branko Radovanovic]

Maybe I'm misunderstanding something here, but what you're saying is:

• You have a setup in which SF is clearly weaker than its opponent
• In this setup, negative contempt yields better match results than positive contempt

But this is precisely what lowering contempt is supposed to do in this situation. On the other hand, if you added enough cores to make SF stronger than LC0, then the setting of -75 would actually become detrimental: what helps you when you are weaker, hurts you when you are stronger.

[Raphexon]

Maybe I'm misunderstanding something here, but what you're saying is:

• You have a setup in which SF is clearly weaker than its opponent
• In this setup, negative contempt yields better match results than positive contempt

But this is precisely what lowering contempt is supposed to do in this situation. On the other hand, if you added enough cores to make SF stronger than LC0, then the setting of -75 would actually become detrimental: what helps you when you are weaker, hurts you when you are stronger.

I think SF benefits from low contempt against Leela even if hardware differences cause SF to be stronger.

Low contempt "forces" SF to trade much sooner which in turn will cause the game to be tactical at a much earlier state too.

[corres]

Some notes:
1,40 games are too few to measure Elo.
2,Basically the contempt has effect on the style of an engine and it has smaller effect on Elo.
3,The chess power of Leela strongly depends on the used net and the parameters of Leela together with move time.
4,In general against a dual RTX 2080 Ti even 16 cores is few for Stockfish to get equal challenge.

[yanquis1972]

Why on earth did you limit SF to 3 cores?? As pointed out by everyone, it can only be surmised that negative contempt is implemented very well. The strength difference is massive. As for SF winning TCEC with such settings...you can’t win if you don’t...win.

[Leo]

Its worth a try. Its good to experiment.

[Leo]

Why on earth did you limit SF to 3 cores?? As pointed out by everyone, it can only be surmised that negative contempt is implemented very well. The strength difference is massive. As for SF winning TCEC with such settings...you can’t win if you don’t...win.

He might only have a quad core or he was proving a point on how decent a weak hardware setup can play against a powerful LCO.

[ernest]

Some notes:
1, 40 games are too few to measure Elo.

Yes indeed !

The error margins (95%) are :
+17 =20 -3 approx. +/- 70 Elo

+9 =30 -1 approx. +/- 50 Elo

+10 =27 -3 approx. +/- 60 Elo

+6 =34 -0 approx. +/- 40 Elo

[MMarco]

And also to have the option to display always contempt 0 in the evaluation, while using contempt -75, would be interesting.

I like this idea. Or more general, show the "default settings" evaluation, when you modify the settings.

[Hai]

And also to have the option to display always contempt 0 in the evaluation, while using contempt -75, would be interesting.

I like this idea. Or more general, show the "default settings" evaluation, when you modify the settings.

Sounds much better.
Can you write in on Github please. https://github.com/LeelaChessZero/lc0/issues