w49921 net

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Nay Lin Tun
Posts: 576
Joined: Mon Jan 16, 2012 5:34 am

Re: w49921 net

Post by Nay Lin Tun » Wed Nov 27, 2019 2:46 am

shrapnel wrote:
Wed Nov 27, 2019 2:39 am
corres wrote:
Tue Nov 26, 2019 10:45 pm
Why did you choose this net for test and why did you not choose other well-known net like 42850, T40B.4-160 or -260, 40.T8.610, etc? In the latter case we can make comparison more easily to earlier tests.
Hey Corres, I was going to tell you about this NN. Its something like 42850 but much stronger.
Do your Testing with it.

Chad said "Net Id 49921 was from an experimental run 3 that took 42850 and continued training from it with much higher nodes per move. We expected it to become stronger, but it became weaker instead (for various reasons)."

Modern Times
Posts: 2539
Joined: Thu Jun 07, 2012 9:02 pm

Re: w49921 net

Post by Modern Times » Wed Nov 27, 2019 2:58 am

dkappe wrote:
Wed Nov 27, 2019 12:21 am
Some people say it’s 10-20 elo weaker than 42850, some say it’s about even. I can find tests in the leela chess discord on both sides of this issue. In most cases, various late t40 nets and t40 derived nets are within a few points of each other.
It is a really big problem with Lc0 ratings list testing - which net do you choose ? It is a bit of a lottery. I've usually played it safe and chosen one that has been sent to TCEC etc.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

RubiChess
Posts: 167
Joined: Fri Mar 30, 2018 5:20 am

Re: w49921 net

Post by RubiChess » Wed Nov 27, 2019 5:46 am

Nay Lin Tun wrote:
Wed Nov 27, 2019 2:43 am
Modern Times wrote:
Tue Nov 26, 2019 10:02 pm
it is just one of hundreds of Lc0 nets. You'll find it here.

http://lczero.org/networks/
One of the weakest net in T40. (Among full trained net).

None of Leela user use it for analysis.
As long as there is no official statement from the lc0 guys "THIS is our official net. Use THIS for rating lists" you can't blame the testers for taking one of the last nets when they start testing an engine.
Remember that they still use last official SF10 for testing until there is an official SF11 (or an official derivate, but that's another story).

Andreas

Gian-Carlo Pascutto
Posts: 1192
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: w49921 net

Post by Gian-Carlo Pascutto » Wed Nov 27, 2019 7:19 am

RubiChess wrote:
Wed Nov 27, 2019 5:46 am
Remember that they still use last official SF10 for testing
They do not.

corres
Posts: 2592
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: w49921 net

Post by corres » Wed Nov 27, 2019 8:09 am

Nay Lin Tun wrote:
Wed Nov 27, 2019 2:43 am
One of the weakest net in T40. (Among full trained net).
None of Leela user use it for analysis.
I also think about it. But the difference is not too heavy.

corres
Posts: 2592
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: w49921 net

Post by corres » Wed Nov 27, 2019 8:16 am

Modern Times wrote:
Wed Nov 27, 2019 2:58 am
dkappe wrote:
Wed Nov 27, 2019 12:21 am
Some people say it’s 10-20 elo weaker than 42850, some say it’s about even. I can find tests in the leela chess discord on both sides of this issue. In most cases, various late t40 nets and t40 derived nets are within a few points of each other.
It is a really big problem with Lc0 ratings list testing - which net do you choose ? It is a bit of a lottery. I've usually played it safe and chosen one that has been sent to TCEC etc.
I agree. TCEC and CCC matches show real picture about the performance of the used net.
So using 49921 for test is a superfluously extravagant thing.

corres
Posts: 2592
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: w49921 net

Post by corres » Wed Nov 27, 2019 8:32 am

RubiChess wrote:
Wed Nov 27, 2019 5:46 am
...
Remember that they still use last official SF10 for testing until there is an official SF11 (or an official derivate, but that's another story).
Andreas
It is a question if they stick to the original Stockfish binary why use derivative of Stockfish?
Derivatives of Stockfish are the developed versions of Stockfish.
So?
I think in the private life there is some place the liberalism but in technical field we should stick to the well established basements.
I do not like to go thru a liberally planned bridge.
Note
There is one thing a well established investment and another thing is an arbitrary, unestablished but very liberal modification.

pohl4711
Posts: 1239
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: w49921 net

Post by pohl4711 » Wed Nov 27, 2019 9:37 am

corres wrote:
Tue Nov 26, 2019 11:48 pm
Graham Banks wrote:
Tue Nov 26, 2019 10:51 pm
corres wrote:
Tue Nov 26, 2019 10:45 pm
Graham Banks wrote:
Tue Nov 26, 2019 10:01 pm
https://lczero.org/networks/
49921 3 4fd54df8 3355.00 105274 20 256 2019-07-29 17:02:08 +00:00 3355
Thanks.
Why did you choose this net for test and why did you not choose other well-known net like 42850, T40B.4-160 or -260, 40.T8.610, etc? In the latter case we can make comparison more easily to earlier tests.
Sergio will either have had it recommended to him, or chosen it due to its high performance rating there.
Really, performance and behavior of 49921 is rather similar to 42850 and 42872.
Thanks for the answers.
Wait some days for my testing result. The 3000 games testrun of lc0 0.22.0 49921 is running. Then, we will know, how strong this net really is.

https://www.sp-cc.de/lc0-testing.htm

corres
Posts: 2592
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: w49921 net

Post by corres » Wed Nov 27, 2019 11:22 am

pohl4711 wrote:
Wed Nov 27, 2019 9:37 am
corres wrote:
Tue Nov 26, 2019 11:48 pm
...
Really, performance and behavior of 49921 is rather similar to 42850 and 42872.
Wait some days for my testing result. The 3000 games testrun of lc0 0.22.0 49921 is running. Then, we will know, how strong this net really is.
https://www.sp-cc.de/lc0-testing.htm
I have some note about your test.
1, "Leela ratio" would be useful only if the testers would use the same net what was used by A0 team.
So decreasing the speed of GPU is superfluous. But it is very important to use exactly the same hardware and soft-ware to every tests.
2, For tests the usage of Hyper Trading is not a good thing because it distorts the results. This is same in the case of turbo boost of cores. They should turn off during tests.
3, As the TC is shorter as the NN engines lose their chess power relative to AB engines. Your TC is rather short and this give benefit for AB type engines.
Enhancing the number of games it lessen the deviation in Elo but if the TC is far from the practical using of engines the yielded result is no more valuable than tests with more practical TC and fewer games - even if the deviation is higher.

Modern Times
Posts: 2539
Joined: Thu Jun 07, 2012 9:02 pm

Re: w49921 net

Post by Modern Times » Wed Nov 27, 2019 6:18 pm

RubiChess wrote:
Wed Nov 27, 2019 5:46 am

As long as there is no official statement from the lc0 guys "THIS is our official net. Use THIS for rating lists" you can't blame the testers for taking one of the last nets when they start testing an engine.
Exactly right. There are hundreds of nets, it is impossible to know which is the best one to use. You take a gamble and keep your fingers crossed. This sort of criticism in unwarranted. It is too easy for some people to be wise in hindsight.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Post Reply