Tests with SF+NNUE PO 270720 popc 4 cores

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
corres
Posts: 3600
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Tests with SF+NNUE PO 270720 popc 4 cores

Post by corres » Mon Aug 03, 2020 9:31 am

My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)

Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC - 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games

Test 2 : kTRIAD - NNUE with 4 cores
TC - 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games

I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.

MMarco
Posts: 141
Joined: Sat Apr 11, 2020 11:09 pm
Full name: Marc-O Moisan-Plante

Re: Tests with SF+NNUE PO 270720 popc 4 cores

Post by MMarco » Mon Aug 03, 2020 12:34 pm

corres wrote:
Mon Aug 03, 2020 9:31 am
My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)

Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC - 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games

Test 2 : kTRIAD - NNUE with 4 cores
TC - 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games

I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
Interesting.
You you mind redoing this with 16 cores for SFNNue (vs kTriad, 200 games)?

corres
Posts: 3600
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Tests with SF+NNUE PO 270720 popc 4 cores

Post by corres » Mon Aug 03, 2020 7:39 pm

MMarco wrote:
Mon Aug 03, 2020 12:34 pm
corres wrote:
Mon Aug 03, 2020 9:31 am
My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)

Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC - 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games

Test 2 : kTRIAD - NNUE with 4 cores
TC - 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games

I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
Interesting.
You you mind redoing this with 16 cores for SFNNue (vs kTriad, 200 games)?
It is too hot.
Maybe it will be fresher time.

corres
Posts: 3600
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Tests with SF+NNUE PO 270720 popc 4 cores

Post by corres » Wed Aug 05, 2020 9:42 am

corres wrote:
Mon Aug 03, 2020 9:31 am
My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)

Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC = 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games

Test 2 : kTRIAD - NNUE with 4 cores
TC = 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games

I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
(continued)

Test 3 : NNUE with 16 cores
TC = 1 min + 2 sec / move
NNUE-net = SV-2138
HASH = 2 GB
Result
SF+NNUE PO 270720 popc, !6(!) cores : Stockfish200718 popc 16(!) cores = 22(!) : 2 (176 draw) 200 games.
Elo difference ~ 70 for the benefit of SF+NNUE PO 270720 (popc, 16 cores, nn = SV-2138)

Note
During the test my system was its border.
In the 30(!) degrees Celsius room temperature they were (in degrees Celsius)
CPU power 220-240 Watts
CPU temperature ~ 90 (at 95 degrees Celsius thermal trottling),
VRM MOS temperature ~ 100 (CPU voltage generators)
System temperature 40/60
Chipset temperature ~ 80
Power supply voltages were stable at their nominal value.

Nay Lin Tun
Posts: 651
Joined: Mon Jan 16, 2012 5:34 am

Re: Tests with SF+NNUE PO 270720 popc 4 cores

Post by Nay Lin Tun » Wed Aug 05, 2020 10:17 am

Consider updating Leela engine to 26.1! There could be speed up 20-30%.

corres
Posts: 3600
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Tests with SF+NNUE PO 270720 popc 4 cores

Post by corres » Wed Aug 05, 2020 10:38 am

Nay Lin Tun wrote:
Wed Aug 05, 2020 10:17 am
Consider updating Leela engine to 26.1! There could be speed up 20-30%.
No
The speed of Leela ver.25.1 and Leela ver 26.1 are near the same
The difference is between CUDA 10 + cudnn 7.4.2 and CUDA 11 + cudnn 7.6.4.
I used Leela ver.25.1 with CUDA 11 + cudnn 7.6.4 (my own compile).

Raphexon
Posts: 340
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: Tests with SF+NNUE PO 270720 popc 4 cores

Post by Raphexon » Wed Aug 05, 2020 11:02 am

Weak result is beating Leela on 4 cores now...

User avatar
Ovyron
Posts: 4405
Joined: Tue Jul 03, 2007 2:30 am

Re: Tests with SF+NNUE PO 270720 popc 4 cores

Post by Ovyron » Wed Aug 05, 2020 11:49 am

Raphexon wrote:
Wed Aug 05, 2020 11:02 am
Weak result is beating Leela on 4 cores now...
Yeah, I've been waiting for an outcome of A beats B, and B beats C, but C beats A, so it'd tell us a lot about chess and exploiting weaknesses. But if it's just A not beating C as badly as one would expect, that doesn't say much, only that A is still clearly superior to both.

corres
Posts: 3600
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Tests with SF+NNUE PO 270720 popc 4 cores

Post by corres » Wed Aug 05, 2020 8:54 pm

corres wrote:
Wed Aug 05, 2020 9:42 am
corres wrote:
Mon Aug 03, 2020 9:31 am
My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC Leela ver.0.25.1, CUDA11 + cudnn7.6.4
Leela-params:
Kiudee settings, Threads = 6, NNCachesize = 20000000, MiniBatchSize 256, Backend = Multiplexing,
BackendOptions = (backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16, gpu=1),(backend=cudnn-fp16,gpu=2)
Opening positions 100 positions from My Minibook
with altered colors (200 games)

Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC = 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games

Test 2 : kTRIAD - NNUE with 4 cores
TC = 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games

I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
(continued)

Test 3 : NNUE with 16 cores - Stockfish(dev)
TC = 1 min + 2 sec / move
NNUE-net = SV-2138
HASH = 2 GB
Result
SF+NNUE PO 270720 popc, !6(!) cores : Stockfish200718 popc 16(!) cores = 22(!) : 2 (176 draw) 200 games.
Elo difference ~ 70 for the benefit of SF+NNUE PO 270720 (popc, 16 cores, nn = SV-2138)
Test 4 : NNUE with 16 cores - kTRIAD
TC = 1 min + 2 sec / move
NNUE-net = nn-97f742aaefcd.nnue (Stockfish-dev)!
HASH = 2GB
Leela ver.0.25.1 CUDA11 cudnn 7.6.4 params Kiudee
Leela-net = SV-384x30-t60-3010
Result
SF+NNUE PO 270720 popc 16 cores : kTRIAD = 14 : 6 (180 draw) 200 games
Elo difference ~25 Elo for the benefit of SF+NNUE PO 270720 (popc, 16 cores nn = SFdev)

Note
1. I completed the parameters of kTRIAD
2. I used for this test Stockfish-dev NNUE-net what gave > 80 Elo plus to Stockfish master

MMarco
Posts: 141
Joined: Sat Apr 11, 2020 11:09 pm
Full name: Marc-O Moisan-Plante

Re: Tests with SF+NNUE PO 270720 popc 4 cores

Post by MMarco » Thu Aug 06, 2020 2:45 am

Nice test. Thank you for running it, I found it interesting to follow.

Post Reply