LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Hai
Posts: 548
Joined: Sun Aug 04, 2013 11:19 am

LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Post by Hai » Tue Apr 28, 2020 9:51 pm

Both 8 GB RAM and all 3-6 piece Syzygybases.
The 20 possible opening moves = 40 games.

LC0 vs Stockfish with only 3 cores (5000 kn/s):
Contempt +25
+17 =20 -3
Winning percentage = 67.5%
=127 elo difference

Contempt -25
+9 =30 -1
Winning percentage = 60%
=70 elo difference

Contempt -50
+10 =27 -3
Winning percentage = 58.75%
=61 elo difference

Contempt -75
+6 =34 -0
Winning percentage = 57.5%
=53 elo difference -> with 3 cores vs 2x RTX 2080 Ti

Doubling cpu cores from 3 to 6 cores should give Stockfish with contempt -75 easily an 53 elo improvement, which means the elo difference is 0.
I don't know how good the elo scaling of Stockfish is with a few cores, but I expect it to be much better than with many cores.

Contempt -75 instead of +25 leads to a 75 elo improvement.
Comparing Stockfish lost only 3 wins using contempt -75 but LC0 lost 11 wins. That means LC0 lost nearly 4x more of his wins than Stockfish.

It would be interesting to test Stockfish up to contempt -200 and learn more about the new playing style/improvements.
And also to have the option to display always contempt 0 in the evaluation, while using contempt -75, would be interesting.

Feel free to do lots of your own different contempt tests.

Stockfish could have won the TCEC easily.

Branko Radovanovic
Posts: 66
Joined: Sat Sep 13, 2014 2:12 pm

Re: LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Post by Branko Radovanovic » Tue Apr 28, 2020 11:24 pm

Maybe I'm misunderstanding something here, but what you're saying is:
  • You have a setup in which SF is clearly weaker than its opponent
  • In this setup, negative contempt yields better match results than positive contempt
But this is precisely what lowering contempt is supposed to do in this situation. On the other hand, if you added enough cores to make SF stronger than LC0, then the setting of -75 would actually become detrimental: what helps you when you are weaker, hurts you when you are stronger.

Raphexon
Posts: 327
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Post by Raphexon » Wed Apr 29, 2020 6:13 am

Branko Radovanovic wrote:
Tue Apr 28, 2020 11:24 pm
Maybe I'm misunderstanding something here, but what you're saying is:
  • You have a setup in which SF is clearly weaker than its opponent
  • In this setup, negative contempt yields better match results than positive contempt
But this is precisely what lowering contempt is supposed to do in this situation. On the other hand, if you added enough cores to make SF stronger than LC0, then the setting of -75 would actually become detrimental: what helps you when you are weaker, hurts you when you are stronger.
I think SF benefits from low contempt against Leela even if hardware differences cause SF to be stronger.

Low contempt "forces" SF to trade much sooner which in turn will cause the game to be tactical at a much earlier state too.

corres
Posts: 3482
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Post by corres » Wed Apr 29, 2020 8:35 am

Some notes:
1,40 games are too few to measure Elo.
2,Basically the contempt has effect on the style of an engine and it has smaller effect on Elo.
3,The chess power of Leela strongly depends on the used net and the parameters of Leela together with move time.
4,In general against a dual RTX 2080 Ti even 16 cores is few for Stockfish to get equal challenge.

yanquis1972
Posts: 1766
Joined: Tue Jun 02, 2009 10:14 pm

Re: LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Post by yanquis1972 » Wed Apr 29, 2020 9:18 pm

Why on earth did you limit SF to 3 cores?? As pointed out by everyone, it can only be surmised that negative contempt is implemented very well. The strength difference is massive. As for SF winning TCEC with such settings...you can’t win if you don’t...win.

Leo
Posts: 969
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Post by Leo » Thu Apr 30, 2020 12:07 am

Its worth a try. Its good to experiment.
Advanced Micro Devices fan.

Leo
Posts: 969
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Post by Leo » Thu Apr 30, 2020 12:09 am

yanquis1972 wrote:
Wed Apr 29, 2020 9:18 pm
Why on earth did you limit SF to 3 cores?? As pointed out by everyone, it can only be surmised that negative contempt is implemented very well. The strength difference is massive. As for SF winning TCEC with such settings...you can’t win if you don’t...win.
He might only have a quad core or he was proving a point on how decent a weak hardware setup can play against a powerful LCO.
Advanced Micro Devices fan.

ernest
Posts: 1912
Joined: Wed Mar 08, 2006 7:30 pm

Re: LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Post by ernest » Thu Apr 30, 2020 12:44 am

corres wrote:
Wed Apr 29, 2020 8:35 am
Some notes:
1, 40 games are too few to measure Elo.
Yes indeed !

The error margins (95%) are :
+17 =20 -3 approx. +/- 70 Elo

+9 =30 -1 approx. +/- 50 Elo

+10 =27 -3 approx. +/- 60 Elo

+6 =34 -0 approx. +/- 40 Elo

MMarco
Posts: 132
Joined: Sat Apr 11, 2020 11:09 pm
Full name: Marc-O Moisan-Plante

Re: LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Post by MMarco » Thu Apr 30, 2020 1:22 am

Hai wrote:
Tue Apr 28, 2020 9:51 pm
And also to have the option to display always contempt 0 in the evaluation, while using contempt -75, would be interesting.
I like this idea. Or more general, show the "default settings" evaluation, when you modify the settings.

Hai
Posts: 548
Joined: Sun Aug 04, 2013 11:19 am

Re: LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

Post by Hai » Tue May 05, 2020 7:08 pm

MMarco wrote:
Thu Apr 30, 2020 1:22 am
Hai wrote:
Tue Apr 28, 2020 9:51 pm
And also to have the option to display always contempt 0 in the evaluation, while using contempt -75, would be interesting.
I like this idea. Or more general, show the "default settings" evaluation, when you modify the settings.
Sounds much better.
Can you write in on Github please. https://github.com/LeelaChessZero/lc0/issues

Post Reply