Best Nets for Lc0 Page

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
mwyoung
Posts: 2079
Joined: Wed May 12, 2010 8:00 pm

Re: Best Nets for Lc0 Page

Post by mwyoung » Thu Dec 26, 2019 7:15 pm

Nordlandia wrote:
Wed Dec 25, 2019 1:37 pm
In case of SF vs Leela.

For TC like bullet (1+0 and 1+1. Do it make sense to enable ponder at the cost as less threads available for stockfish. Time is vital in bullet and the result of ponder is somewhat better time management. 60% ponder hit mean 1.6x base time in theory.
Now that CCC seem to think that Ponder on is now OK with Lc0. Glad to see you come around. :D Now it is time to implement Ponder on with Lc0 in the most efficient way. As I have said in the past you should not limit the cores and threads in SF. This is not needed and hurts performance.

Let your computer manage the threads. It is much better at this then a human trying to core cut the CPU.

To do this right give all the logical cores to SF, but run SF at a lower priority. And then run the Lc0 threads at normal priority. Your performance loss will then be limited to the CPU utilization of Lc0 which is very low and will only affect SF.
Professing themselves to be wise, they became fools,
take on me. Foes 0.

corres
Posts: 2659
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Best Nets for Lc0 Page

Post by corres » Thu Dec 26, 2019 8:43 pm

Because the number of cores used by Leela is restricted (2-6 cores) obviously as the full number of cores are higher as the disturbance effect of pondering is smaller on working of engines.
But using ponder has an other effect too: It enhances the effective thinking time. The enhancement is not a constant because it depends on how often the engine "founds out" the move of other engine. If the two engine are very similar to each other the effective thinking time will be near two times of the no ponder time. Opposite to this if the two engine are very far from each other in evaluation using the ponder gives only a small enhancement in effective move time.
So using ponder modifies importantly the results of a match. It is a newer factor what makes more uncertain the ranking line (that is Elo) of engines.

shrapnel
Posts: 1324
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Best Nets for Lc0 Page

Post by shrapnel » Sat Dec 28, 2019 7:27 am

New Sergio NN based on T60 ( 384x30-t60-2-1504.pb)is EVEN STRONGER and its a whopping 158 MB NN :shock: .
Sergio has become very active and his NNs are becoming stronger and stronger with each new Release !
Rating of Lc0 on my Machine on Playchess has reached 2540 from the low 2400s in the last 2-3 days, while using just Sergio NNs.
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

Hai
Posts: 548
Joined: Sun Aug 04, 2013 11:19 am

Re: Best Nets for Lc0 Page

Post by Hai » Sat Dec 28, 2019 9:06 am

shrapnel wrote:
Sat Dec 28, 2019 7:27 am
New Sergio NN based on T60 ( 384x30-t60-2-1504.pb)is EVEN STRONGER and its a whopping 158 MB NN :shock: .
Sergio has become very active and his NNs are becoming stronger and stronger with each new Release !
Rating of Lc0 on my Machine on Playchess has reached 2540 from the low 2400s in the last 2-3 days, while using just Sergio NNs.
Bigger nets are stronger, when they have trained enough. And of course the mb get bigger and bigger.
That's completely normal.
The Sergio nets should be already stronger on 1 rtx gpu then other nets, so you can use 2 rtx gpus for even bigger nets.
Sergio nets need an upgrade to a bigger size of 40x512, maybe someone can do this.
7-men tablebases rescaling should be done too.

Leelenstein would have the strongest nets when upgrading to a size of 40x512.

The LC0 project should also do 7-men tablebases rescaling and train from chess960 start positions.

corres
Posts: 2659
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Best Nets for Lc0 Page

Post by corres » Sat Dec 28, 2019 11:34 am

Hai wrote:
Sat Dec 28, 2019 9:06 am
...
Bigger nets are stronger, when they have trained enough. And of course the mb get bigger and bigger.
That's completely normal.
The Sergio nets should be already stronger on 1 rtx gpu then other nets, so you can use 2 rtx gpus for even bigger nets.
...
Alone the dimension is not a guarantee for being stronger.
It is pity but using a simple RTX card - even if it is a 2080 Ti - jhorthos T40B.4-160 net is stronger than any sergio's big nets. Maybe using LTC they would have the same power.

User avatar
Laskos
Posts: 10198
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Best Nets for Lc0 Page

Post by Laskos » Sat Dec 28, 2019 12:55 pm

corres wrote:
Sat Dec 28, 2019 11:34 am
Hai wrote:
Sat Dec 28, 2019 9:06 am
...
Bigger nets are stronger, when they have trained enough. And of course the mb get bigger and bigger.
That's completely normal.
The Sergio nets should be already stronger on 1 rtx gpu then other nets, so you can use 2 rtx gpus for even bigger nets.
...
Alone the dimension is not a guarantee for being stronger.
It is pity but using a simple RTX card - even if it is a 2080 Ti - jhorthos T40B.4-160 net is stronger than any sergio's big nets. Maybe using LTC they would have the same power.
I have tested Sergio's latest 256x20b and 384x30b nets on RTX 2070 OC-ed GPU. They are very strong, this guy is up to something. Observe that his 256x20b net is smaller and faster than both JHorthos and T40 regular 256x20b nets.

His 384x30-t40-1573 behaves this way:

In 100 games at 60'' + 0.6'' it is weaker than the best net I had so far, jhorthos T40B.4-200, some 50 Elo points weaker in 200 games.
In 40 games at 900'' + 9'' they came completely equal in strength and it is beyond error margins that this Big Net scales better to LTC. If this holds, it would be stronger than T40B.4-200 at tournament time control even on one RTX GPU. This Big Net strangely performs better tactically at 30s/position on Arasan test suite compared to T40B.4-200. And comes about equal positionally (strange behavior of the Big Net, I would have expected the opposite).

The nice surprise came with Sergio's 256x20-t40-1400 net. Aside from very long time controls, this net seems to be the strongest around for RTX GPUs, from bullet TC to rapid TC, at least. In 400 games at 30'' + 0.3'', 256x20-t40-1400 net beat T40B.4-200 by about 20 +/- 7 Elo points (1 standard deviation), which is remarkable. I guess Sergio's nets are now the strongest around for RTX GPUs, with 256x20b nets at shorter and medium TC, and maybe (not sure) the 384x30b 1573 net at very long TC. And the Big Net has yet a lot of learning path to accomplish. Good job of Sergio.

User avatar
AdminX
Posts: 5386
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Re: Best Nets for Lc0 Page

Post by AdminX » Sat Dec 28, 2019 1:22 pm

Laskos wrote:
Sat Dec 28, 2019 12:55 pm
corres wrote:
Sat Dec 28, 2019 11:34 am
Hai wrote:
Sat Dec 28, 2019 9:06 am
...
Bigger nets are stronger, when they have trained enough. And of course the mb get bigger and bigger.
That's completely normal.
The Sergio nets should be already stronger on 1 rtx gpu then other nets, so you can use 2 rtx gpus for even bigger nets.
...
Alone the dimension is not a guarantee for being stronger.
It is pity but using a simple RTX card - even if it is a 2080 Ti - jhorthos T40B.4-160 net is stronger than any sergio's big nets. Maybe using LTC they would have the same power.
I have tested Sergio's latest 256x20b and 384x30b nets on RTX 2070 OC-ed GPU. They are very strong, this guy is up to something. Observe that his 256x20b net is smaller and faster than both JHorthos and T40 regular 256x20b nets.

His 384x30-t40-1573 behaves this way:

In 100 games at 60'' + 0.6'' it is weaker than the best net I had so far, jhorthos T40B.4-200, some 50 Elo points weaker in 200 games.
In 40 games at 900'' + 9'' they came completely equal in strength and it is beyond error margins that this Big Net scales better to LTC. If this holds, it would be stronger than T40B.4-200 at tournament time control even on one RTX GPU. This Big Net strangely performs better tactically at 30s/position on Arasan test suite compared to T40B.4-200. And comes about equal positionally (strange behavior of the Big Net, I would have expected the opposite).

The nice surprise came with Sergio's 256x20-t40-1400 net. Aside from very long time controls, this net seems to be the strongest around for RTX GPUs, from bullet TC to rapid TC, at least. In 400 games at 30'' + 0.3'', 256x20-t40-1400 net beat T40B.4-200 by about 20 +/- 7 Elo points (1 standard deviation), which is remarkable. I guess Sergio's nets are now the strongest around for RTX GPUs, with 256x20b nets at shorter and medium TC, and maybe (not sure) the 384x30b 1573 net at very long TC. And the Big Net has yet a lot of learning path to accomplish. Good job of Sergio.
I definitely agree, I am seeing similar behavior on my end as well.
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

corres
Posts: 2659
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Best Nets for Lc0 Page

Post by corres » Sat Dec 28, 2019 1:58 pm

A note
I wrote about jhorthos T40B.4-160 net.
This was stronger on my system than T40B.4-200.

User avatar
Laskos
Posts: 10198
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Best Nets for Lc0 Page

Post by Laskos » Sat Dec 28, 2019 2:12 pm

corres wrote:
Sat Dec 28, 2019 1:58 pm
A note
I wrote about jhorthos T40B.4-160 net.
This was stronger on my system than T40B.4-200.
I guess any difference between them is within 5 Elo points. I tested them in 2000 games at fixed 1000 nodes, they are that close, with 4-200 coming a couple of points better than 4-160, but well within error margins.

corres
Posts: 2659
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Best Nets for Lc0 Page

Post by corres » Sat Dec 28, 2019 2:21 pm

Laskos wrote:
Sat Dec 28, 2019 2:12 pm
corres wrote:
Sat Dec 28, 2019 1:58 pm
A note
I wrote about jhorthos T40B.4-160 net.
This was stronger on my system than T40B.4-200.
I guess any difference between them is within 5 Elo points. I tested them in 2000 games at fixed 1000 nodes, they are that close, with 4-200 coming a couple of points better than 4-160, but well within error margins.
Tests with fixed nodes are not comparable to tests with fixed time.
As I noted earlier the move/time used by you is too short and disturb the results.
NN engines are non-effective in those short time control.
They need more time to "warm-up".

Post Reply