Some speed tests with Leela and Allie

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
corres
Posts: 1663
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Some speed tests with Leela and Allie

Post by corres » Thu Sep 19, 2019 7:21 pm

The tests are as follows;
Used PC: AMD Ryzen 7 1800x 8x4000 MHz processor and 2 x RTX 2060 OC-ed GPUs
1, Leela ver.0.22 tests
Common parameters:
NNCacheSize 10000000
MiniBatchSize 512
MaxPrefetchBatch 64.
NET 42850
Single GPU with plus parameters Backend cudnn-FP16 and Threads 2
Max speed = 28.4 kn/sec (Depth = 22, Nodes = 7.69 Mn)
DUAL GPU with plus parameters Backend Multiplexing BackendOptions backend=cudnn-FP16,(gpu=0),(gpu=1) and Threads 4
Max speed = 46.0 kn/sec (Depth = 22, Nodes = 7.42 Mn)
NET 320x24 J13B.2-136 (JHorthos)
Single GPU (plus pamaters as above)
Max speed =15.1 kn/sec (Depth = 19, Nodes = 5.47 Mn)
DUAL GPU plus parameters as above
Max speed = 28.6 kn/sec (Depth = 19, Nodes = 5.68 Mn)
2, Allie ver.0.5-dev tests
NET 42850
Single GPU
Parameters: UseFP16 = ON, GPUCores = 1, others are DEFAULT
Max speed = 119 kn/sec (Depth = 15, Nodes = 4.82 Mn)
DUAL GPU
Parameterts: UseFP16 = ON, GPUCores = 2, others are DEFAULT
Nax Speed = 236 kn/sec (Depth = 15, Nodes = 4.79 Mn)
NET 320x24 J13B.2-136 (JHorthos)
Single GPU
Parameters as above
Max speed = 66.5 kn/sec (Depth = 14, Nodes = 6.33 Mn)
DUAL GPU
Parameters as above
Max speed = 135 kn/sec (Depth = 14, Nodes = 6.18 Mn)
As we can see Allie is more faster but its search depth is shallower than Leela have.
Maybe in the good role of Allie the used special NET is an important factor.

Gian-Carlo Pascutto
Posts: 1184
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: Some speed tests with Leela and Allie

Post by Gian-Carlo Pascutto » Thu Sep 19, 2019 7:24 pm

Node speeds reported by lc0 based engines are nonsense because cache hits are counted as full nodes. Likewise, the reported depth for MCTS-based engines is arbitrary.

You can't conclude anything from these numbers.

Nay Lin Tun
Posts: 520
Joined: Mon Jan 16, 2012 5:34 am

Re: Some speed tests with Leela and Allie

Post by Nay Lin Tun » Thu Sep 19, 2019 7:30 pm

corres wrote:
Thu Sep 19, 2019 7:21 pm
The tests are as follows;
Used PC: AMD Ryzen 7 1800x 8x4000 MHz processor and 2 x RTX 2060 OC-ed GPUs
1, Leela ver.0.22 tests
Common parameters:
NNCacheSize 10000000
MiniBatchSize 512
MaxPrefetchBatch 64.
NET 42850
Single GPU with plus parameters Backend cudnn-FP16 and Threads 2
Max speed = 28.4 kn/sec (Depth = 22, Nodes = 7.69 Mn)
DUAL GPU with plus parameters Backend Multiplexing BackendOptions backend=cudnn-FP16,(gpu=0),(gpu=1) and Threads 4
Max speed = 46.0 kn/sec (Depth = 22, Nodes = 7.42 Mn)
NET 320x24 J13B.2-136 (JHorthos)
Single GPU (plus pamaters as above)
Max speed =15.1 kn/sec (Depth = 19, Nodes = 5.47 Mn)
DUAL GPU plus parameters as above
Max speed = 28.6 kn/sec (Depth = 19, Nodes = 5.68 Mn)
2, Allie ver.0.5-dev tests
NET 42850
Single GPU
Parameters: UseFP16 = ON, GPUCores = 1, others are DEFAULT
Max speed = 119 kn/sec (Depth = 15, Nodes = 4.82 Mn)
DUAL GPU
Parameterts: UseFP16 = ON, GPUCores = 2, others are DEFAULT
Nax Speed = 236 kn/sec (Depth = 15, Nodes = 4.79 Mn)
NET 320x24 J13B.2-136 (JHorthos)
Single GPU
Parameters as above
Max speed = 66.5 kn/sec (Depth = 14, Nodes = 6.33 Mn)
DUAL GPU
Parameters as above
Max speed = 135 kn/sec (Depth = 14, Nodes = 6.18 Mn)
As we can see Allie is more faster but its search depth is shallower than Leela have.
Maybe in the good role of Allie the used special NET is an important factor.
Totally irrelevant.
Allie authour explain how he count nodes in Leela discord and Tcec twitch chat.
Leela discord--- Technical channel is the best to discuss NN stuffs.

dkappe
Posts: 261
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Some speed tests with Leela and Allie

Post by dkappe » Thu Sep 19, 2019 8:03 pm

I don’t think he knows who he’s talking to.

corres
Posts: 1663
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Some speed tests with Leela and Allie

Post by corres » Thu Sep 19, 2019 8:32 pm

Gian-Carlo Pascutto wrote:
Thu Sep 19, 2019 7:24 pm
Node speeds reported by lc0 based engines are nonsense because cache hits are counted as full nodes. Likewise, the reported depth for MCTS-based engines is arbitrary.
You can't conclude anything from these numbers.
It is true if you use the the same binary and the same NET for different GPUs you can rank those GPUs based on the yielded speed.
If you use the same NET and the same GPU for different binaries "maybe" you have some guessing about the rank of the binaries. Obviously a lot of game need to decide about the ratings.
Why you think that in the case of given Net and given binary there is no any connection between speed and the chess power?
As if the TCEC and CCC experiments - together with private experiments - can show other...
You would be right if every each binary of Leela would compute the speed with different mode.

Gian-Carlo Pascutto
Posts: 1184
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: Some speed tests with Leela and Allie

Post by Gian-Carlo Pascutto » Thu Sep 19, 2019 8:35 pm

corres wrote:
Thu Sep 19, 2019 8:32 pm
Why you think that in the case of given Net and given binary there is no any connection between speed and the chess power?
Because the speed changes if you change the cache size (reason, see previous post).

(There's still a positive correlation in the general case, but you can come up with pathological examples I'm sure)

If you change the binary, you don't even know if nodes are counted the same way.

corres
Posts: 1663
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Some speed tests with Leela and Allie

Post by corres » Thu Sep 19, 2019 8:54 pm

Gian-Carlo Pascutto wrote:
Thu Sep 19, 2019 8:35 pm
corres wrote:
Thu Sep 19, 2019 8:32 pm
Why you think that in the case of given Net and given binary there is no any connection between speed and the chess power?
Because the speed changes if you change the cache size (reason, see previous post).
(There's still a positive correlation in the general case, but you can come up with pathological examples I'm sure)
If you change the binary, you don't even know if nodes are counted the same way.
Please, read my post again and do not miss the parameters.
Using the same NET and parameters if I change the binaries yielded from the same source the speeds will differ only because of "noise".
If I use different binaries from different sources but the sources are not far from each other
(as Crem stated) then in the case of the same NET the connection between speed and depth may be
rather similar.
I think everybody have right to get such a conclusion from my results what she/he would like.
Last edited by corres on Thu Sep 19, 2019 9:03 pm, edited 1 time in total.

corres
Posts: 1663
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Some speed tests with Leela and Allie

Post by corres » Thu Sep 19, 2019 9:01 pm

...
Totally irrelevant.
Allie authour explain how he count nodes in Leela discord and Tcec twitch chat.
Leela discord--- Technical channel is the best to discuss NN stuffs.
[/quote]

Why you think you are the only one who read my post?
No everybody like just the high-flown articles.

Post Reply