Best Nets for Lc0 Page

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
shrapnel
Posts: 1340
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Best Nets for Lc0 Page

Post by shrapnel » Tue Dec 31, 2019 4:31 pm

Just to emphasize the Point, watch FREE Lc0 using FREE Sergio NN THRASH Commercial Fat Fritz with BLACK....
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

Robert Flesher
Posts: 1240
Joined: Tue Aug 18, 2009 1:06 am

Re: Best Nets for Lc0 Page

Post by Robert Flesher » Wed Jan 01, 2020 6:27 pm

Sergio 1705 net is out. Strong!!

User avatar
Laskos
Posts: 10793
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Best Nets for Lc0 Page

Post by Laskos » Wed Jan 01, 2020 6:48 pm

Robert Flesher wrote:
Wed Jan 01, 2020 6:27 pm
Sergio 1705 net is out. Strong!!
I would say that Sergio's 256x20b net 1541 net is the strongest around for my OC-ed RTX 2070 GPU on any reasonable TC, even LTC.

Here is what I get with the BigNets:

Code: Select all

Games at 30'' + 0.3''

SV 384x30b
Rank Name                          Elo     +/-   Games   Score   Draws
   1 1573                           36      19     800   55.1%   38.3%
   2 1538                           26      19     800   53.8%   38.3%
   3 1530                          -62      20     800   41.1%   35.0%
Finished match
And:

Code: Select all

Games at 4 min + 2.4 s

Score of Sergio_384x30_1573 vs Sergio_256x20_1467: 24 - 59 - 117 [0.412]
Elo difference: -61.4 +/- 30.8, LOS: 0.0 %, DrawRatio: 58.5 %

200 of 200 games finished.
In the second one, the time control is already real Blitz. I don't think that the SV BigNet scales so well to catch-up with the 256x20b SV net even to LTC on my GPU, 60 Elo points are too much.

And today came the new Sergio_256x20_1541 net which again improves sensibly on 1467:

Code: Select all

1000 nodes per move:

Score of 1541 vs 1467: 611 - 541 - 848  [0.517] 2000
Elo difference: 12.17 +/- 11.55
Finished match
The real error margins are a factor of 1.4 smaller than the shown ones because a I used unbalanced positions side-and-reversed.

So, all in all, I would take for my RTX GPU Sergio's SV_256x20_1541 at any time control (aside longish analysis, where BigNet 1705 is probably the best).

User avatar
Laskos
Posts: 10793
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Best Nets for Lc0 Page

Post by Laskos » Wed Jan 01, 2020 7:35 pm

Laskos wrote:
Wed Jan 01, 2020 6:48 pm
Robert Flesher wrote:
Wed Jan 01, 2020 6:27 pm
Sergio 1705 net is out. Strong!!
I would say that Sergio's 256x20b net 1541 net is the strongest around for my OC-ed RTX 2070 GPU on any reasonable TC, even LTC.

Here is what I get with the BigNets:

Code: Select all

Games at 30'' + 0.3''

SV 384x30b
Rank Name                          Elo     +/-   Games   Score   Draws
   1 1573                           36      19     800   55.1%   38.3%
   2 1538                           26      19     800   53.8%   38.3%
   3 1530                          -62      20     800   41.1%   35.0%
Finished match
And:

Code: Select all

Games at 4 min + 2.4 s

Score of Sergio_384x30_1573 vs Sergio_256x20_1467: 24 - 59 - 117 [0.412]
Elo difference: -61.4 +/- 30.8, LOS: 0.0 %, DrawRatio: 58.5 %

200 of 200 games finished.
In the second one, the time control is already real Blitz. I don't think that the SV BigNet scales so well to catch-up with the 256x20b SV net even to LTC on my GPU, 60 Elo points are too much.

And today came the new Sergio_256x20_1541 net which again improves sensibly on 1467:

Code: Select all

1000 nodes per move:

Score of 1541 vs 1467: 611 - 541 - 848  [0.517] 2000
Elo difference: 12.17 +/- 11.55
Finished match
The real error margins are a factor of 1.4 smaller than the shown ones because a I used unbalanced positions side-and-reversed.

So, all in all, I would take for my RTX GPU Sergio's SV_256x20_1541 at any time control (aside longish analysis, where BigNet 1705 is probably the best).
Just finished the test of SV 1541 against vanilla T40 net 42850, which ruled supreme for some time in the past.

Code: Select all

Games in 30'' + 0.3''

Score of SV_256x20_1541 vs 42850: 257 - 166 - 377  [0.557] 800
Elo difference: 39.69 +/- 17.51
Finished match
Impressive and consistent. JHorthos best 256x20 nets are some 20 Elo points above vanilla 42850, and Sergio's SV 256x20_1541 is another impressive 30 Elo points above the best JHorthos nets. These improvements do not add orthogonally, so one would expect a total of maybe 40-43 Elo points improvement over the vanilla, consistent with the result. Sergio is doing some amazing stuff there, because his training games are those of the same vanilla T40 run. Maybe he can reach even a total of 50 Elo improvement over the vanilla T40, and I have never expected this.

User avatar
MikeB
Posts: 4393
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Best Nets for Lc0 Page

Post by MikeB » Wed Jan 01, 2020 8:46 pm

The Sergio nets are very very good, On an RTX 2060 Super , the latest Sergio net is at least about equal ( maybe better) to the latest Komodo running on the Threadripper with all 64 threads.
Image

mwyoung
Posts: 2433
Joined: Wed May 12, 2010 8:00 pm

Re: Best Nets for Lc0 Page

Post by mwyoung » Thu Jan 02, 2020 2:41 am

AdminX wrote:
Mon Dec 23, 2019 10:23 pm
"There is no single "best net" for Leela but there a few worth recommending for various purposes. The most important consideration in choosing a net is picking the right size for your hardware and time controls (TC). In general, if you have a weak GPU (or no GPU at all) and you want to only spend milliseconds per move, then you want a smaller net that evaluates positions more quickly, i.e. higher NPS (nodes per second). On the other hand, if you have an RTX card (or several) and you want to run an analysis on a single move for hours at a time, then the quality of the evaluation is more important than the speed and a larger (but slower) net is probably going to work best."

https://github.com/LeelaChessZero/lc0/w ... ts-for-Lc0
The one point you did not mention is the configuration of the Lc0 net in the GUI. The default settings are awful and more so for the bigger nets. In my current testing with the T61xxx nets. With a few simple tweaks this net series is testing 50 to 60 elo better then default settings. And doing this at fast time controls against the Best A/B engines.
Professing themselves to be wise, they became fools,
take on me. Foes 0.

shrapnel
Posts: 1340
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Best Nets for Lc0 Page

Post by shrapnel » Thu Jan 02, 2020 5:52 am

mwyoung wrote:
Thu Jan 02, 2020 2:41 am
With a few simple tweaks this net series is testing 50 to 60 elo better then default settings.
Share your Settings please.
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

corres
Posts: 3452
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Best Nets for Lc0 Page

Post by corres » Thu Jan 02, 2020 7:48 am

mwyoung wrote:
Thu Jan 02, 2020 2:41 am
...
The default settings are awful and more so for the bigger nets. In my current testing with the T61xxx nets. With a few simple tweaks this net series is testing 50 to 60 elo better then default settings. And doing this at fast time controls against the Best A/B engines.
There is only one problem: Using very short time control it is very far from the practical games. A test is useful only if the circumstances of the test are the same as in the case of normal games. It is a more important viewpoint than getting a mathematically exact but practically useless Elo. If somebody stick to gain relative exact Elo one should not regret the time for getting a practically useful result.
It is also an important viewpoint the connection between power of GPU and the dimension of net.
If somebody would like to get the real knowledge what is in a big net that one need a GPU with higher power.
Obviously as the dimension of a net is bigger as it needs more time and/or more faster GPU to scan its content.

mwyoung
Posts: 2433
Joined: Wed May 12, 2010 8:00 pm

Re: Best Nets for Lc0 Page

Post by mwyoung » Thu Jan 02, 2020 12:31 pm

corres wrote:
Thu Jan 02, 2020 7:48 am
mwyoung wrote:
Thu Jan 02, 2020 2:41 am
...
The default settings are awful and more so for the bigger nets. In my current testing with the T61xxx nets. With a few simple tweaks this net series is testing 50 to 60 elo better then default settings. And doing this at fast time controls against the Best A/B engines.
There is only one problem: Using very short time control it is very far from the practical games. A test is useful only if the circumstances of the test are the same as in the case of normal games. It is a more important viewpoint than getting a mathematically exact but practically useless Elo. If somebody stick to gain relative exact Elo one should not regret the time for getting a practically useful result.
It is also an important viewpoint the connection between power of GPU and the dimension of net.
If somebody would like to get the real knowledge what is in a big net that one need a GPU with higher power.
Obviously as the dimension of a net is bigger as it needs more time and/or more faster GPU to scan its content.
I agree, and that is why I test realistic time controls that would be useful. From 5s a move to 5m a move.

I will say a higher power gpu is nice. But if you are running a bad configuration you are wasting a lot of that horse power.
Professing themselves to be wise, they became fools,
take on me. Foes 0.

Robert Flesher
Posts: 1240
Joined: Tue Aug 18, 2009 1:06 am

Re: Best Nets for Lc0 Page

Post by Robert Flesher » Thu Jan 02, 2020 1:29 pm

shrapnel wrote:
Thu Jan 02, 2020 5:52 am
mwyoung wrote:
Thu Jan 02, 2020 2:41 am
With a few simple tweaks this net series is testing 50 to 60 elo better then default settings.
Share your Settings please.
+1

Post Reply