TalkChess.com

Posted: **Sat Feb 01, 2020 1:25 pm**

corres wrote: ↑Tue Jan 28, 2020 8:42 pm
corres wrote: ↑Mon Jan 27, 2020 1:02 pm I made some tests with and without kiudee parameters.
Hardware
AMD Threadripper 16 x 4.0 GHz
2 x RTX 2060 OC (~1 x RTX 2080 Ti OC)
Common parameters of the tests
Chess GUI: DeepFritz 14
Hash (for Stockfish): 2048 MB
TC 1 min + 2 sec/move
Openings my TestBook50 with altered colors (2 x 50 = 100 games)
Tablebases: 6 men syzygy + 5 men Nalimov (only for GUI)

Parameters of Leela ver.0.22.0

"def" marked:
threads 4
backend multiplexing
backendoptions (backend=cudnn-fp16,gpu=1),(backend=cudnn-fp16,gpu=2)
NNcacheSize 10000000
MaxPrefetch 64
Others are Leela Defaults

"ccc" marked (kiudee):
threads 4
backend multiplexing
backendoptions (backend=cudnn-fp16,gpu=1),(backend=cudnn-fp16,gpu=2)
NNcacheSize 10000000
CPuct 2.147
CPuctBase 18368
CPuctFactor 2.815
FPUValue 0.443
MaxCollisionEvents 256
MaxPrefetch 64
PolicyTemperature 1.607
Others are Leela Defaults

Results 1.
ccc62078 : defJ13B.2-188 = 11 : 4 (85 draw) 100 games
def62078 : defJ13B.2-188 = 7 : 5 (88 draw) 100 games
(ccc gave +5 points)
Results 2.
ccc62078 : Stockfish191002 = 9 : 8 (83 draw) 100 games
def62078 : Stockfish191002 = 16 : 9 (75 draw) 100 games
(ccc gave -6 points)
Results 3.
ccc62078 : def62078 = 3 : 2 (95 draw) 100 games
(ccc gave +1 point)

From the above results it is obvious 62078net is stronger than J13B.2-188 and Stockfish191002 with and without kiudee parameters. I think the Elo of 62078net is valued down on the list of Leela nets.
On the other hand the effect of kiudee parameters is rather ambiguous so using it as a default settings is risky.
It is proposed to make tests with kiudee on the own system before using for analysis or engine-engine matches.
Note
You can download the test games from
wikisend.com
File ID 266998
Password leela
These are some newer results:
Results 4.
cccJ13B.2-188 : Stockfish191002 = 12 : 10 (78 draw) 100 games
defJ13B.2-188 : Stockfish191002 = 8 : 12 (80 draw) 100 games
(ccc gave -2 points)
Results 5.
cccT40B.4-160 : Stockfish191002 = 16 : 12 (72 draw) 100 games
defT40B.4-160 : Stockfish191002 = 10 : 10 (80 draw) 100 games
(ccc gave +4 points)
So the the ambiguous effect of kiudee parameters did not change.

At last I found a normal (256x20) net what can utilize unambiguously well the kiudee settings!
This is Sergio 256x20-T40-1541 net.
Results 6.
cccT40-1541 : defT40-1541 = 6 : 0 (94 draw) 100 games
(ccc gave +6 points)
cccT40-1541 : Stockfish191002 = 21 : 7 (72 draw) 100 games
defT40-1541 : Stockfish191002 = 13 : 7 (80 draw) 100 games
(ccc gave +8(!) points)

You can download kiudee_sum.zip from
wikisend.com
File ID 568700
Password leela
It contains .pgn files and my TestBook50.

Note
I use chess engines to make analysis of common chess games and chess positions.
Because of this I assembled my TestBook50 from wide range of chess openings used by human GM.
So my TestBook50 is not an opening set for extravagant tests like "Armageddon" or "spectacle chess" with unbalanced play or to demonstrate the exactness of Elo calculation but a set for watching and studying the behavior of chess engines playing common chess.

Posted: **Sat Feb 01, 2020 7:02 pm**

M ANSARI wrote: ↑Sat Feb 01, 2020 8:43 am That is why I still think the perfect solution is an engine that combines A/B with NN. I know that nobody yet is looking at that

It has been looked into and all the ideas that have been tried have produced moves that are weaker than A/B alone or NN alone. I suspect that what is happening is that some people have an average CPU and a superb GPU or a CPU with many cores with an average GPU. For the former the main chess power comes from the NN, and any attempt to improve over this would make it weaker because the suggestions by A/B are poor. For the latter the CPU engine is so strong that the NNs can't provide something of value most of the time (and the time used checking NN's ideas weakens the engine the rest of the game).

Whatever is your forte (CPU or GPU) you'd rather focus on using the strongest thing on it, unless an idea of how to combine them in a way where the strongest thing isn't weakened by the weaker thing is yet to be implemented.

Posted: **Sat Feb 01, 2020 8:02 pm**

Test match...

http://talkchess.com/forum3/viewtopic.php?f=6&t=72973

Posted: **Mon Feb 03, 2020 1:54 am**

jp wrote: ↑Sat Feb 01, 2020 9:47 am
mwyoung wrote: ↑Thu Jan 30, 2020 8:16 pm In each playout, the game is played out to the very end by selecting moves at random.
This is not correct, right? In game-play (not training), most playouts will not end in terminal nodes.

You are correct. My error.

The PUCT used in AGZ and Lc0 replaces rollouts (sampling playouts to a terminal game state) with a neural network that estimates what a rollout would do.
Because it is not a true MCTS.

And this is why a Neural Net with the best estimates is so important. And why size matters. As bigger nets cost NPS, but should have better estimates.

TalkChess.com

Are kiudee parameters the best?

Re: Are kiudee parameters the best?

Re: Are kiudee parameters the best?

Re: Are kiudee parameters the best?

Re: Are kiudee parameters the best?