My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)
Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC - 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games
Test 2 : kTRIAD - NNUE with 4 cores
TC - 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games
I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
Tests with SF+NNUE PO 270720 popc 4 cores
Moderators: hgm, Rebel, chrisw
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
-
- Posts: 195
- Joined: Sun Apr 12, 2020 1:09 am
- Full name: Marc-O Moisan-Plante
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Interesting.corres wrote: ↑Mon Aug 03, 2020 11:31 am My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)
Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC - 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games
Test 2 : kTRIAD - NNUE with 4 cores
TC - 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games
I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
You you mind redoing this with 16 cores for SFNNue (vs kTriad, 200 games)?
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: Tests with SF+NNUE PO 270720 popc 4 cores
It is too hot.MMarco wrote: ↑Mon Aug 03, 2020 2:34 pmInteresting.corres wrote: ↑Mon Aug 03, 2020 11:31 am My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)
Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC - 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games
Test 2 : kTRIAD - NNUE with 4 cores
TC - 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games
I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
You you mind redoing this with 16 cores for SFNNue (vs kTriad, 200 games)?
Maybe it will be fresher time.
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: Tests with SF+NNUE PO 270720 popc 4 cores
(continued)corres wrote: ↑Mon Aug 03, 2020 11:31 am My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC
Opening positions 100 positions from My Minibook
with altered colors (200 games)
Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC = 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games
Test 2 : kTRIAD - NNUE with 4 cores
TC = 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games
I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
Test 3 : NNUE with 16 cores
TC = 1 min + 2 sec / move
NNUE-net = SV-2138
HASH = 2 GB
Result
SF+NNUE PO 270720 popc, !6(!) cores : Stockfish200718 popc 16(!) cores = 22(!) : 2 (176 draw) 200 games.
Elo difference ~ 70 for the benefit of SF+NNUE PO 270720 (popc, 16 cores, nn = SV-2138)
Note
During the test my system was its border.
In the 30(!) degrees Celsius room temperature they were (in degrees Celsius)
CPU power 220-240 Watts
CPU temperature ~ 90 (at 95 degrees Celsius thermal trottling),
VRM MOS temperature ~ 100 (CPU voltage generators)
System temperature 40/60
Chipset temperature ~ 80
Power supply voltages were stable at their nominal value.
-
- Posts: 708
- Joined: Mon Jan 16, 2012 6:34 am
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Consider updating Leela engine to 26.1! There could be speed up 20-30%.
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: Tests with SF+NNUE PO 270720 popc 4 cores
NoNay Lin Tun wrote: ↑Wed Aug 05, 2020 12:17 pm Consider updating Leela engine to 26.1! There could be speed up 20-30%.
The speed of Leela ver.25.1 and Leela ver 26.1 are near the same
The difference is between CUDA 10 + cudnn 7.4.2 and CUDA 11 + cudnn 7.6.4.
I used Leela ver.25.1 with CUDA 11 + cudnn 7.6.4 (my own compile).
-
- Posts: 476
- Joined: Sun Mar 17, 2019 12:00 pm
- Full name: Henk Drost
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Weak result is beating Leela on 4 cores now...
-
- Posts: 4556
- Joined: Tue Jul 03, 2007 4:30 am
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Yeah, I've been waiting for an outcome of A beats B, and B beats C, but C beats A, so it'd tell us a lot about chess and exploiting weaknesses. But if it's just A not beating C as badly as one would expect, that doesn't say much, only that A is still clearly superior to both.
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Test 4 : NNUE with 16 cores - kTRIADcorres wrote: ↑Wed Aug 05, 2020 11:42 am(continued)corres wrote: ↑Mon Aug 03, 2020 11:31 am My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT - Disabled, PB - Disabled
RAM - 32 GB
GUI - Fritz 14
SYZYGY 6 men
Nalimov 5 men (only for GUI)
kTRIAD (NVIDIA) - 1 x RTX 2080 Ti OC Turbo + 2 x RTX 2060 OC Leela ver.0.25.1, CUDA11 + cudnn7.6.4
Leela-params:
Kiudee settings, Threads = 6, NNCachesize = 20000000, MiniBatchSize 256, Backend = Multiplexing,
BackendOptions = (backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16, gpu=1),(backend=cudnn-fp16,gpu=2)
Opening positions 100 positions from My Minibook
with altered colors (200 games)
Test 1 : NNUE with 4 cores - Stockfish with 12 cores
TC = 1 min + 2 sec / move
NNUE-net SV-2138
HASH - 2 GB
Result
SF + NNUE PO 270720 popc with 4 cores : Stockfish200718 popc 12 cores = 19 : 17 (164 draw) 200 games
Test 2 : kTRIAD - NNUE with 4 cores
TC = 1 min + 2 sec / move
NNNUE-net SV-2138
Leela ver-0.25.1, CUDA11, cudnn 7.6.4, parameters = Kiudee
Leela-net SV-384x30-t60-3010
Result
kTRIAD : SF + NNUE PO 270720 = 24 : 5 (171 draw) 200 games
I think the cause of the weak result of NNUE is: Opposite to Stockfis(dev) NNUE can not utilize the the tactical fault of Leela.
For the shake of comparison:
kTriad : Stockfish 200624(with 16 cores) = 21 : 10 (169 draw) 200 games
From my earlier test result.
Test 3 : NNUE with 16 cores - Stockfish(dev)
TC = 1 min + 2 sec / move
NNUE-net = SV-2138
HASH = 2 GB
Result
SF+NNUE PO 270720 popc, !6(!) cores : Stockfish200718 popc 16(!) cores = 22(!) : 2 (176 draw) 200 games.
Elo difference ~ 70 for the benefit of SF+NNUE PO 270720 (popc, 16 cores, nn = SV-2138)
TC = 1 min + 2 sec / move
NNUE-net = nn-97f742aaefcd.nnue (Stockfish-dev)!
HASH = 2GB
Leela ver.0.25.1 CUDA11 cudnn 7.6.4 params Kiudee
Leela-net = SV-384x30-t60-3010
Result
SF+NNUE PO 270720 popc 16 cores : kTRIAD = 14 : 6 (180 draw) 200 games
Elo difference ~25 Elo for the benefit of SF+NNUE PO 270720 (popc, 16 cores nn = SFdev)
Note
1. I completed the parameters of kTRIAD
2. I used for this test Stockfish-dev NNUE-net what gave > 80 Elo plus to Stockfish master
-
- Posts: 195
- Joined: Sun Apr 12, 2020 1:09 am
- Full name: Marc-O Moisan-Plante
Re: Tests with SF+NNUE PO 270720 popc 4 cores
Nice test. Thank you for running it, I found it interesting to follow.