At last I found a normal (256x20) net what can utilize unambiguously well the kiudee settings!corres wrote: ↑Tue Jan 28, 2020 8:42 pmThese are some newer results:corres wrote: ↑Mon Jan 27, 2020 1:02 pm I made some tests with and without kiudee parameters.
Hardware
AMD Threadripper 16 x 4.0 GHz
2 x RTX 2060 OC (~1 x RTX 2080 Ti OC)
Common parameters of the tests
Chess GUI: DeepFritz 14
Hash (for Stockfish): 2048 MB
TC 1 min + 2 sec/move
Openings my TestBook50 with altered colors (2 x 50 = 100 games)
Tablebases: 6 men syzygy + 5 men Nalimov (only for GUI)
Parameters of Leela ver.0.22.0
"def" marked:
threads 4
backend multiplexing
backendoptions (backend=cudnn-fp16,gpu=1),(backend=cudnn-fp16,gpu=2)
NNcacheSize 10000000
MaxPrefetch 64
Others are Leela Defaults
"ccc" marked (kiudee):
threads 4
backend multiplexing
backendoptions (backend=cudnn-fp16,gpu=1),(backend=cudnn-fp16,gpu=2)
NNcacheSize 10000000
CPuct 2.147
CPuctBase 18368
CPuctFactor 2.815
FPUValue 0.443
MaxCollisionEvents 256
MaxPrefetch 64
PolicyTemperature 1.607
Others are Leela Defaults
Results 1.
ccc62078 : defJ13B.2-188 = 11 : 4 (85 draw) 100 games
def62078 : defJ13B.2-188 = 7 : 5 (88 draw) 100 games
(ccc gave +5 points)
Results 2.
ccc62078 : Stockfish191002 = 9 : 8 (83 draw) 100 games
def62078 : Stockfish191002 = 16 : 9 (75 draw) 100 games
(ccc gave -6 points)
Results 3.
ccc62078 : def62078 = 3 : 2 (95 draw) 100 games
(ccc gave +1 point)
From the above results it is obvious 62078net is stronger than J13B.2-188 and Stockfish191002 with and without kiudee parameters. I think the Elo of 62078net is valued down on the list of Leela nets.
On the other hand the effect of kiudee parameters is rather ambiguous so using it as a default settings is risky.
It is proposed to make tests with kiudee on the own system before using for analysis or engine-engine matches.
Note
You can download the test games from
wikisend.com
File ID 266998
Password leela
Results 4.
cccJ13B.2-188 : Stockfish191002 = 12 : 10 (78 draw) 100 games
defJ13B.2-188 : Stockfish191002 = 8 : 12 (80 draw) 100 games
(ccc gave -2 points)
Results 5.
cccT40B.4-160 : Stockfish191002 = 16 : 12 (72 draw) 100 games
defT40B.4-160 : Stockfish191002 = 10 : 10 (80 draw) 100 games
(ccc gave +4 points)
So the the ambiguous effect of kiudee parameters did not change.
This is Sergio 256x20-T40-1541 net.
Results 6.
cccT40-1541 : defT40-1541 = 6 : 0 (94 draw) 100 games
(ccc gave +6 points)
cccT40-1541 : Stockfish191002 = 21 : 7 (72 draw) 100 games
defT40-1541 : Stockfish191002 = 13 : 7 (80 draw) 100 games
(ccc gave +8(!) points)
You can download kiudee_sum.zip from
wikisend.com
File ID 568700
Password leela
It contains .pgn files and my TestBook50.
Note
I use chess engines to make analysis of common chess games and chess positions.
Because of this I assembled my TestBook50 from wide range of chess openings used by human GM.
So my TestBook50 is not an opening set for extravagant tests like "Armageddon" or "spectacle chess" with unbalanced play or to demonstrate the exactness of Elo calculation but a set for watching and studying the behavior of chess engines playing common chess.