Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Post by zullil »

mwyoung wrote: Sun Oct 06, 2019 7:54 pm
Nordlandia wrote: Sun Oct 06, 2019 7:34 pm Are you sure that 4 threads is better that 2 threads for Leela?

2 is the default and on my system it use 1.5 core.
Yes I am sure. You see the results. You can see the CPU and GPU load on my testing stream. The question you need to ask is why others are not tuning Lc0 to their rig for best performance. I guess because then do not know how, or don't want to take the time.

I have seen some very bad setups for Lc0.

I will tell you. If you are setting Lc0 for maximum NPS. You are on the wrong path. NPS with Lc0 is a false flag.
How did you determine that 4 worker threads are better than 2 for a single GPU system?

https://github.com/LeelaChessZero/lc0/b ... r/FLAGS.md

viewtopic.php?t=70035

https://groups.google.com/forum/#!searc ... IVlc2hBQAJ
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Post by mwyoung »

shrapnel wrote: Sun Oct 06, 2019 8:28 pm
mwyoung wrote: Sun Oct 06, 2019 7:54 pm The question you need to ask is why others are not tuning Lc0 to their rig for best performance. I guess because then do not know how, or don't want to take the time.

I have seen some very bad setups for Lc0.

I will tell you. If you are setting Lc0 for maximum NPS. You are on the wrong path. NPS with Lc0 is a false flag.
Hi
If NPS with Lc0 is a false flag, which agrees with my experience on Infinity Chess, what exactly are you Tuning it to and how ?
Good question!

I tune it by time to depth and GPU loading. I ignore NPS results. And test this over many positions for 5 mins for each position, and you need to use end games positions also, not just middle game positions to achieve the best results when tuning.

And then test against your other setups for improvement.

Each computer and GPU combo will have different settings.

The wild card setting is your cput setting. Tune this to what you think works best. My setting is 3.0, but some insist that others are better like 3.4. And this is dependent on your GPU speed and testing time controls. But tune it to your cput setting....
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
shrapnel
Posts: 1339
Joined: Fri Nov 02, 2012 9:43 am
Location: New Delhi, India

Re: Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Post by shrapnel »

Interesting Idea !
Thank You
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis
User avatar
George Tsavdaris
Posts: 1627
Joined: Thu Mar 09, 2006 12:35 pm

Re: Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Post by George Tsavdaris »

mwyoung wrote: Sun Oct 06, 2019 8:46 pm I will tell you. If you are setting Lc0 for maximum NPS. You are on the wrong path. NPS with Lc0 is a false flag.
I tune it by time to depth and GPU loading. I ignore NPS results.
Tuning Leela according to depth is a very bad idea and it's pointless and dangerous. Dangerous that you will end up with a weaker Leela probably.
Depth is not a good metric at all for MCTS type programs like it is with AB type programs.
N/s is the only easy and sensible metric for performance.
As more N/s, is always better. For Leela that is.

So what you are doing so far with your tests, probably has a somewhat weaker Leela as you didn't try to achieve higher N/s.
Last edited by George Tsavdaris on Sun Oct 06, 2019 11:31 pm, edited 1 time in total.
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Post by mwyoung »

George Tsavdaris wrote: Sun Oct 06, 2019 11:19 pm
mwyoung wrote: Sun Oct 06, 2019 8:46 pm I will tell you. If you are setting Lc0 for maximum NPS. You are on the wrong path. NPS with Lc0 is a false flag.
I tune it by time to depth and GPU loading. I ignore NPS results.
Tuning Leela according to depth is a very bad idea and it's pointless and dangerous. Dangerous that you will end up with a weaker Leela probably.
Depth is not a good metric at all for MCTS type programs like it is with AB type programs.
N/s is the only easy and sensible metric for performance.
As more N/s, is always better.

So what you are doing so far with your tests, probably has a somewhat weaker Leela as you didn't try to achieve higher N/s.
Then don't use GPU loading and TIME TO DEPTH. My results speak for themselves.

You act like lc0 is new, and we have not tried the NPS tuning. Like I said NPS is a false flag.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
User avatar
George Tsavdaris
Posts: 1627
Joined: Thu Mar 09, 2006 12:35 pm

Re: Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Post by George Tsavdaris »

mwyoung wrote: Sun Oct 06, 2019 11:26 pm
George Tsavdaris wrote: Sun Oct 06, 2019 11:19 pm
mwyoung wrote: Sun Oct 06, 2019 8:46 pm I will tell you. If you are setting Lc0 for maximum NPS. You are on the wrong path. NPS with Lc0 is a false flag.
I tune it by time to depth and GPU loading. I ignore NPS results.
Tuning Leela according to depth is a very bad idea and it's pointless and dangerous. Dangerous that you will end up with a weaker Leela probably.
Depth is not a good metric at all for MCTS type programs like it is with AB type programs.
N/s is the only easy and sensible metric for performance.
As more N/s, is always better.

So what you are doing so far with your tests, probably has a somewhat weaker Leela as you didn't try to achieve higher N/s.
Then don't use GPU loading and TIME TO DEPTH. My results speak for themselves.

You act like lc0 is new, and we have not tried the NPS tuning. Like I said NPS is a false flag.
How exactly do they speak if you have nothing to compare?
I bet that using properly set up parameters for obtaining higher N/s would be better for Leela.
No need to assume things anyway, as depth is completely nonsensical metric for Leela. Optimizing for it is useless as i've said.
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Post by mwyoung »

George Tsavdaris wrote: Sun Oct 06, 2019 11:32 pm
mwyoung wrote: Sun Oct 06, 2019 11:26 pm
George Tsavdaris wrote: Sun Oct 06, 2019 11:19 pm
mwyoung wrote: Sun Oct 06, 2019 8:46 pm I will tell you. If you are setting Lc0 for maximum NPS. You are on the wrong path. NPS with Lc0 is a false flag.
I tune it by time to depth and GPU loading. I ignore NPS results.
Tuning Leela according to depth is a very bad idea and it's pointless and dangerous. Dangerous that you will end up with a weaker Leela probably.
Depth is not a good metric at all for MCTS type programs like it is with AB type programs.
N/s is the only easy and sensible metric for performance.
As more N/s, is always better.

So what you are doing so far with your tests, probably has a somewhat weaker Leela as you didn't try to achieve higher N/s.
Then don't use GPU loading and TIME TO DEPTH. My results speak for themselves.

You act like lc0 is new, and we have not tried the NPS tuning. Like I said NPS is a false flag.
How exactly do they speak if you have nothing to compare?
I bet that using properly set up parameters for obtaining higher N/s would be better for Leela.
No need to assume things anyway, as depth is completely nonsensical metric for Leela. Optimizing for it is useless as i've said.
I am good with you are anyone doing want they want to do to setup Lc0. If you think this is wrong, I am good with it. If you think I have not tested other settings you have not been paying attention.

It is to the end user to see if my method is better or worse.

"Whoever has ears, let them hear.”
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
User avatar
Nordlandia
Posts: 2821
Joined: Fri Sep 25, 2015 9:38 pm
Location: Sortland, Norway

Re: Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Post by Nordlandia »

4 threads for Lc0 = exercise in futility.

And so with your ponder match.
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Post by mwyoung »

Nordlandia wrote: Mon Oct 07, 2019 12:36 pm 4 threads for Lc0 = exercise in futility.

And so with your ponder match.
You always say this. But you can never answer why. Why is using 4 threads a exercise in futility?
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
User avatar
Nordlandia
Posts: 2821
Joined: Fri Sep 25, 2015 9:38 pm
Location: Sortland, Norway

Re: Lc0 + 320x24.J13B.2-swa-136000 vs. Stockfish 250919 TC= 30m+30s

Post by Nordlandia »

Because any more than default may hurt raw performance.