LC0 30x384 (3350) + 2x RTX 2080 Ti vs Stockfish Dev 22.04.2020 + 6 cores + Contempt -75 should be a fair play/ratio

[AdminX]

Saw this post last night and decided to try, I don't have a six core system. I have a quad core box with hyperthreading. I used the -75 comtempt with Stockfish 11BMI vs LC0 63426 using 1 RTX2070 Super. I ran a 40 game test which finished this morning. Time control was 4'2 Blitz, Stockfish 11 using 3 Threads, and LC0 with 2 Threads. Hash was set to 8192 (32 GB Total System Memory) for each in the Fritz 17 GUI using Perfect 2019 Opening Book, with 3-6 piece SYZYGY.

Code: Select all

HAKIN94LIFE, Blitz 4.0min+2.0sec  0                                         
1   Stockfish 11 64 BMI2           +1/=38/-1 50.00%   20.0/40  400.00
2   Lc0 v0.25.1+git.69105b4-Net60  +1/=38/-1 50.00%   20.0/40  400.00

Very good results

What do you think about the result of a match on your machine between RTX 2070 Super used Leela ver.0.25.1 with 63426net and Stockfish 11 with default (+24) contempt?

I will give it a test later today and see how it goes. Currrenyly watching todays Nations Cup.

[AdminX]

Saw this post last night and decided to try, I don't have a six core system. I have a quad core box with hyperthreading. I used the -75 comtempt with Stockfish 11BMI vs LC0 63426 using 1 RTX2070 Super. I ran a 40 game test which finished this morning. Time control was 4'2 Blitz, Stockfish 11 using 3 Threads, and LC0 with 2 Threads. Hash was set to 8192 (32 GB Total System Memory) for each in the Fritz 17 GUI using Perfect 2019 Opening Book, with 3-6 piece SYZYGY.

Code: Select all

HAKIN94LIFE, Blitz 4.0min+2.0sec  0                                         
1   Stockfish 11 64 BMI2           +1/=38/-1 50.00%   20.0/40  400.00
2   Lc0 v0.25.1+git.69105b4-Net60  +1/=38/-1 50.00%   20.0/40  400.00

Very good results

What do you think about the result of a match on your machine between RTX 2070 Super used Leela ver.0.25.1 with 63426net and Stockfish 11 with default (+24) contempt?

Here are the results, same settings as before but with the default Stockfish Contempt of +24

Code: Select all

HAKIN94LIFE, Blitz 4.0min+2.0sec  0

                                         
1   Lc0 v0.25.1+git.69105b4-Net60   +70  +8/=32/-0 60.00%   24.0/40
2   Stockfish 11 64 BMI2            -70  +0/=32/-8 40.00%   16.0/40

[corres]

I used the first 40 games from my xkTRIAD-SV-T60-3350_Stockfish200502 test as base-test.
In this test Stockfish parameters were Contempt = +24 (default), Hash = 2 GB, Threads = 16 x 4000 MHz.
Re-run these 40 games with Stockfish parameters Contempt = -75, Hash = 2 GB, Threads = 6 x 4000 Mhz.
In both cases lc0 ver.0.25 run on my TRIAD: 1 x RTX 2080 Ti + 2 x RTX 2060 (~2 x RTX 2080 Ti).
Lc0 parameters were default (kiudee) and Threads = 6, Backend = Multiplexing, BackendOptions = (backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1),(backend=cudnn-fp16,gpu=2), NNCachesize = 20000000, PrefetchBatch = 64.
Results
1, Base-test Contempt = +24 and threads for Stockfish = 16 x 4000 MHz the all cores of Threadripper 2950x
no HT
xkTRIAD : Stockfish200502 = 5 : 3

2, Re-run Contempt = -75 and Threads for Stockfish = 6(!) x 4000 MHz from the 16 cores of Threadripper 2950x no HT
xkTriad : Stockfish200502 = 9 : 1

Games can be downloaded from
wikisend.com
FileID: 777090
Passw: leela

No comment

[Hai]

Saw this post last night and decided to try, I don't have a six core system. I have a quad core box with hyperthreading. I used the -75 comtempt with Stockfish 11BMI vs LC0 63426 using 1 RTX2070 Super. I ran a 40 game test which finished this morning. Time control was 4'2 Blitz, Stockfish 11 using 3 Threads, and LC0 with 2 Threads. Hash was set to 8192 (32 GB Total System Memory) for each in the Fritz 17 GUI using Perfect 2019 Opening Book, with 3-6 piece SYZYGY.

Code: Select all

HAKIN94LIFE, Blitz 4.0min+2.0sec  0                                         
1   Stockfish 11 64 BMI2           +1/=38/-1 50.00%   20.0/40  400.00
2   Lc0 v0.25.1+git.69105b4-Net60  +1/=38/-1 50.00%   20.0/40  400.00

Very good results

What do you think about the result of a match on your machine between RTX 2070 Super used Leela ver.0.25.1 with 63426net and Stockfish 11 with default (+24) contempt?

Here are the results, same settings as before but with the default Stockfish Contempt of +24

Code: Select all

HAKIN94LIFE, Blitz 4.0min+2.0sec  0

                                         
1   Lc0 v0.25.1+git.69105b4-Net60   +70  +8/=32/-0 60.00%   24.0/40
2   Stockfish 11 64 BMI2            -70  +0/=32/-8 40.00%   16.0/40

As always terrible results for Stockfish with contempt +24 instead of negative contempt -75 against LC0.
Stockfish with contempt +24 loses 8x more than in your other test with contempt -75, holy shit that's bad.

[corres]

I used the first 40 games from my xkTRIAD-SV-T60-3350_Stockfish200502 test as base-test.
In this test Stockfish parameters were Contempt = +24 (default), Hash = 2 GB, Threads = 16 x 4000 MHz.
Re-run these 40 games with Stockfish parameters Contempt = -75, Hash = 2 GB, Threads = 6 x 4000 Mhz.
In both cases lc0 ver.0.25 run on my TRIAD: 1 x RTX 2080 Ti + 2 x RTX 2060 (~2 x RTX 2080 Ti).
Lc0 parameters were default (kiudee) and Threads = 6, Backend = Multiplexing, BackendOptions = (backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1),(backend=cudnn-fp16,gpu=2), NNCachesize = 20000000, PrefetchBatch = 64.
Results
1, Base-test Contempt = +24 and threads for Stockfish = 16 x 4000 MHz the all cores of Threadripper 2950x
no HT
xkTRIAD : Stockfish200502 = 5 : 3

2, Re-run Contempt = -75 and Threads for Stockfish = 6(!) x 4000 MHz from the 16 cores of Threadripper 2950x no HT
xkTriad : Stockfish200502 = 9 : 1

Instead of comment download the games from
wikisend.com
FileID: 777090
Passw: leela

[OliverBr]

@Hai,
I don't see any mention of CUDA in your post. Is this a matter of course?

PS:
In my System, Leela CUDA with a GTX 1080 Ti vs Stockfish 11 one core, Leela is clearly stronger. Leela typically never loses, except if you let her play an opening which doesn't fit to her.
That is, not because of the calculation power, but because Leela is positionally much better, which doesn't change when you give her more or less calculation power.

Here are the results, same settings as before but with the default Stockfish Contempt of +24

Code: Select all

HAKIN94LIFE, Blitz 4.0min+2.0sec  0

                                         
1   Lc0 v0.25.1+git.69105b4-Net60   +70  +8/=32/-0 60.00%   24.0/40
2   Stockfish 11 64 BMI2            -70  +0/=32/-8 40.00%   16.0/40

As always terrible results for Stockfish with contempt +24 instead of negative contempt -75 against LC0.
Stockfish with contempt +24 loses 8x more than in your other test with contempt -75, holy shit that's bad.
[/quote]

There is nothing unusual with your result. Leela is the by far better strategical player. She may be only mediocre in tactics and trolling in endgame. But, with that positional play it's enough.

[mehmet123]

Both 8 GB RAM and all 3-6 piece Syzygybases.
The 20 possible opening moves = 40 games.

LC0 vs Stockfish with only 3 cores (5000 kn/s):
Contempt +25
+17 =20 -3
Winning percentage = 67.5%
=127 elo difference

Contempt -25
+9 =30 -1
Winning percentage = 60%
=70 elo difference

Contempt -50
+10 =27 -3
Winning percentage = 58.75%
=61 elo difference

Contempt -75
+6 =34 -0
Winning percentage = 57.5%
=53 elo difference -> with 3 cores vs 2x RTX 2080 Ti

Doubling cpu cores from 3 to 6 cores should give Stockfish with contempt -75 easily an 53 elo improvement, which means the elo difference is 0.
I don't know how good the elo scaling of Stockfish is with a few cores, but I expect it to be much better than with many cores.

Contempt -75 instead of +25 leads to a 75 elo improvement.
Comparing Stockfish lost only 3 wins using contempt -75 but LC0 lost 11 wins. That means LC0 lost nearly 4x more of his wins than Stockfish.

It would be interesting to test Stockfish up to contempt -200 and learn more about the new playing style/improvements.
And also to have the option to display always contempt 0 in the evaluation, while using contempt -75, would be interesting.

Feel free to do lots of your own different contempt tests.

Stockfish could have won the TCEC easily.

Can you give information about time control of the matches and the kn/s of Lc0 3350