Chad said "Net Id 49921 was from an experimental run 3 that took 42850 and continued training from it with much higher nodes per move. We expected it to become stronger, but it became weaker instead (for various reasons)."
w49921 net
Moderators: hgm, Rebel, chrisw
-
- Posts: 708
- Joined: Mon Jan 16, 2012 6:34 am
Re: w49921 net
Chad said "Net Id 49921 was from an experimental run 3 that took 42850 and continued training from it with much higher nodes per move. We expected it to become stronger, but it became weaker instead (for various reasons)."
-
- Posts: 3550
- Joined: Thu Jun 07, 2012 11:02 pm
Re: w49921 net
It is a really big problem with Lc0 ratings list testing - which net do you choose ? It is a bit of a lottery. I've usually played it safe and chosen one that has been sent to TCEC etc.
-
- Posts: 587
- Joined: Fri Mar 30, 2018 7:20 am
- Full name: Andreas Matthies
Re: w49921 net
As long as there is no official statement from the lc0 guys "THIS is our official net. Use THIS for rating lists" you can't blame the testers for taking one of the last nets when they start testing an engine.Nay Lin Tun wrote: ↑Wed Nov 27, 2019 3:43 amOne of the weakest net in T40. (Among full trained net).Modern Times wrote: ↑Tue Nov 26, 2019 11:02 pm it is just one of hundreds of Lc0 nets. You'll find it here.
http://lczero.org/networks/
None of Leela user use it for analysis.
Remember that they still use last official SF10 for testing until there is an official SF11 (or an official derivate, but that's another story).
Andreas
-
- Posts: 1243
- Joined: Sat Dec 13, 2008 7:00 pm
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: w49921 net
I also think about it. But the difference is not too heavy.Nay Lin Tun wrote: ↑Wed Nov 27, 2019 3:43 am One of the weakest net in T40. (Among full trained net).
None of Leela user use it for analysis.
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: w49921 net
I agree. TCEC and CCC matches show real picture about the performance of the used net.Modern Times wrote: ↑Wed Nov 27, 2019 3:58 amIt is a really big problem with Lc0 ratings list testing - which net do you choose ? It is a bit of a lottery. I've usually played it safe and chosen one that has been sent to TCEC etc.
So using 49921 for test is a superfluously extravagant thing.
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: w49921 net
It is a question if they stick to the original Stockfish binary why use derivative of Stockfish?
Derivatives of Stockfish are the developed versions of Stockfish.
So?
I think in the private life there is some place the liberalism but in technical field we should stick to the well established basements.
I do not like to go thru a liberally planned bridge.
Note
There is one thing a well established investment and another thing is an arbitrary, unestablished but very liberal modification.
-
- Posts: 2441
- Joined: Sat Sep 03, 2011 7:25 am
- Location: Berlin, Germany
- Full name: Stefan Pohl
Re: w49921 net
Wait some days for my testing result. The 3000 games testrun of lc0 0.22.0 49921 is running. Then, we will know, how strong this net really is.corres wrote: ↑Wed Nov 27, 2019 12:48 amReally, performance and behavior of 49921 is rather similar to 42850 and 42872.Graham Banks wrote: ↑Tue Nov 26, 2019 11:51 pmSergio will either have had it recommended to him, or chosen it due to its high performance rating there.corres wrote: ↑Tue Nov 26, 2019 11:45 pmThanks.Graham Banks wrote: ↑Tue Nov 26, 2019 11:01 pm https://lczero.org/networks/
49921 3 4fd54df8 3355.00 105274 20 256 2019-07-29 17:02:08 +00:00 3355
Why did you choose this net for test and why did you not choose other well-known net like 42850, T40B.4-160 or -260, 40.T8.610, etc? In the latter case we can make comparison more easily to earlier tests.
Thanks for the answers.
https://www.sp-cc.de/lc0-testing.htm
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: w49921 net
I have some note about your test.pohl4711 wrote: ↑Wed Nov 27, 2019 10:37 amWait some days for my testing result. The 3000 games testrun of lc0 0.22.0 49921 is running. Then, we will know, how strong this net really is.
https://www.sp-cc.de/lc0-testing.htm
1, "Leela ratio" would be useful only if the testers would use the same net what was used by A0 team.
So decreasing the speed of GPU is superfluous. But it is very important to use exactly the same hardware and soft-ware to every tests.
2, For tests the usage of Hyper Trading is not a good thing because it distorts the results. This is same in the case of turbo boost of cores. They should turn off during tests.
3, As the TC is shorter as the NN engines lose their chess power relative to AB engines. Your TC is rather short and this give benefit for AB type engines.
Enhancing the number of games it lessen the deviation in Elo but if the TC is far from the practical using of engines the yielded result is no more valuable than tests with more practical TC and fewer games - even if the deviation is higher.
-
- Posts: 3550
- Joined: Thu Jun 07, 2012 11:02 pm
Re: w49921 net
Exactly right. There are hundreds of nets, it is impossible to know which is the best one to use. You take a gamble and keep your fingers crossed. This sort of criticism in unwarranted. It is too easy for some people to be wise in hindsight.