Page 1 of 2
Tests with SF+NNUE PO 270720 popc 4 cores
Posted: Mon Aug 03, 2020 11:31 am
by corres
My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)
Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC - 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games
Test 2 : kTRIAD - NNUE with 4 cores
TC - 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games
I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Posted: Mon Aug 03, 2020 2:34 pm
by MMarco
corres wrote: ↑Mon Aug 03, 2020 11:31 am
My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)
Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC - 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games
Test 2 : kTRIAD - NNUE with 4 cores
TC - 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games
I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
Interesting.
You you mind redoing this with 16 cores for SFNNue (vs kTriad, 200 games)?
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Posted: Mon Aug 03, 2020 9:39 pm
by corres
MMarco wrote: ↑Mon Aug 03, 2020 2:34 pm
corres wrote: ↑Mon Aug 03, 2020 11:31 am
My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)
Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC - 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games
Test 2 : kTRIAD - NNUE with 4 cores
TC - 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games
I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
Interesting.
You you mind redoing this with 16 cores for SFNNue (vs kTriad, 200 games)?
It is too hot.
Maybe it will be fresher time.
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Posted: Wed Aug 05, 2020 11:42 am
by corres
corres wrote: ↑Mon Aug 03, 2020 11:31 am
My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)
Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC = 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games
Test 2 : kTRIAD - NNUE with 4 cores
TC = 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games
I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
(continued)
Test 3 : NNUE with 16 cores
TC = 1 min + 2 sec / move
NNUE-net = SV-2138
HASH = 2 GB
Result
SF+NNUE PO 270720 popc, !6(!) cores : Stockfish200718 popc 16(!) cores = 22(!) : 2 (176 draw) 200 games.
Elo difference ~ 70 for the benefit of SF+NNUE PO 270720 (popc, 16 cores, nn = SV-2138)
Note
During the test my system was its border.
In the 30(!) degrees Celsius room temperature they were (in degrees Celsius)
CPU power 220-240 Watts
CPU temperature ~ 90 (at 95 degrees Celsius thermal trottling),
VRM MOS temperature ~ 100 (CPU voltage generators)
System temperature 40/60
Chipset temperature ~ 80
Power supply voltages were stable at their nominal value.
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Posted: Wed Aug 05, 2020 12:17 pm
by Nay Lin Tun
Consider updating Leela engine to 26.1! There could be speed up 20-30%.
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Posted: Wed Aug 05, 2020 12:38 pm
by corres
Nay Lin Tun wrote: ↑Wed Aug 05, 2020 12:17 pm
Consider updating Leela engine to 26.1! There could be speed up 20-30%.
No
The speed of Leela ver.25.1 and Leela ver 26.1 are near the same
The difference is between CUDA 10 + cudnn 7.4.2 and CUDA 11 + cudnn 7.6.4.
I used Leela ver.25.1 with CUDA 11 + cudnn 7.6.4 (my own compile).
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Posted: Wed Aug 05, 2020 1:02 pm
by Raphexon
Weak result is beating Leela on 4 cores now...
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Posted: Wed Aug 05, 2020 1:49 pm
by Ovyron
Raphexon wrote: ↑Wed Aug 05, 2020 1:02 pm
Weak result is beating Leela on 4 cores now...
Yeah, I've been waiting for an outcome of A beats B, and B beats C, but C beats A, so it'd tell us a lot about chess and exploiting weaknesses. But if it's just A not beating C as badly as one would expect, that doesn't say much, only that A is still clearly superior to both.
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Posted: Wed Aug 05, 2020 10:54 pm
by corres
corres wrote: ↑Wed Aug 05, 2020 11:42 am
corres wrote: ↑Mon Aug 03, 2020 11:31 am
My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC Leela ver.0.25.1, CUDA11 + cudnn7.6.4
Leela-params:
Kiudee settings, Threads = 6, NNCachesize = 20000000, MiniBatchSize 256, Backend = Multiplexing,
BackendOptions = (backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16, gpu=1),(backend=cudnn-fp16,gpu=2)
Opening positions 100 positions from My Minibook
with altered colors (200 games)
Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC = 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games
Test 2 : kTRIAD - NNUE with 4 cores
TC = 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games
I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
(continued)
Test 3 : NNUE with 16 cores - Stockfish(dev)
TC = 1 min + 2 sec / move
NNUE-net = SV-2138
HASH = 2 GB
Result
SF+NNUE PO 270720 popc, !6(!) cores : Stockfish200718 popc 16(!) cores = 22(!) : 2 (176 draw) 200 games.
Elo difference ~ 70 for the benefit of SF+NNUE PO 270720 (popc, 16 cores, nn = SV-2138)
Test 4 : NNUE with 16 cores - kTRIAD
TC = 1 min + 2 sec / move
NNUE-net = nn-97f742aaefcd.nnue (Stockfish-dev)!
HASH = 2GB
Leela ver.0.25.1 CUDA11 cudnn 7.6.4 params Kiudee
Leela-net = SV-384x30-t60-3010
Result
SF+NNUE PO 270720 popc 16 cores : kTRIAD = 14 : 6 (180 draw) 200 games
Elo difference ~25 Elo for the benefit of SF+NNUE PO 270720 (popc, 16 cores nn = SFdev)
Note
1. I completed the parameters of kTRIAD
2. I used for this test Stockfish-dev NNUE-net what gave > 80 Elo plus to Stockfish master
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Posted: Thu Aug 06, 2020 4:45 am
by MMarco
Nice test. Thank you for running it, I found it interesting to follow.