w49921 net

[Nay Lin Tun]

Why did you choose this net for test and why did you not choose other well-known net like 42850, T40B.4-160 or -260, 40.T8.610, etc? In the latter case we can make comparison more easily to earlier tests.

Hey Corres, I was going to tell you about this NN. Its something like 42850 but much stronger.
Do your Testing with it.

Chad said "Net Id 49921 was from an experimental run 3 that took 42850 and continued training from it with much higher nodes per move. We expected it to become stronger, but it became weaker instead (for various reasons)."

[Modern Times]

Some people say it’s 10-20 elo weaker than 42850, some say it’s about even. I can find tests in the leela chess discord on both sides of this issue. In most cases, various late t40 nets and t40 derived nets are within a few points of each other.

It is a really big problem with Lc0 ratings list testing - which net do you choose ? It is a bit of a lottery. I've usually played it safe and chosen one that has been sent to TCEC etc.

[RubiChess]

it is just one of hundreds of Lc0 nets. You'll find it here.

http://lczero.org/networks/

One of the weakest net in T40. (Among full trained net).

None of Leela user use it for analysis.

As long as there is no official statement from the lc0 guys "THIS is our official net. Use THIS for rating lists" you can't blame the testers for taking one of the last nets when they start testing an engine.
Remember that they still use last official SF10 for testing until there is an official SF11 (or an official derivate, but that's another story).

Andreas

[Gian-Carlo Pascutto]

Remember that they still use last official SF10 for testing

They do not.

[corres]

One of the weakest net in T40. (Among full trained net).
None of Leela user use it for analysis.

I also think about it. But the difference is not too heavy.

[corres]

Some people say it’s 10-20 elo weaker than 42850, some say it’s about even. I can find tests in the leela chess discord on both sides of this issue. In most cases, various late t40 nets and t40 derived nets are within a few points of each other.

It is a really big problem with Lc0 ratings list testing - which net do you choose ? It is a bit of a lottery. I've usually played it safe and chosen one that has been sent to TCEC etc.

I agree. TCEC and CCC matches show real picture about the performance of the used net.
So using 49921 for test is a superfluously extravagant thing.

[corres]

...
Remember that they still use last official SF10 for testing until there is an official SF11 (or an official derivate, but that's another story).
Andreas

It is a question if they stick to the original Stockfish binary why use derivative of Stockfish?
Derivatives of Stockfish are the developed versions of Stockfish.
So?
I think in the private life there is some place the liberalism but in technical field we should stick to the well established basements.
I do not like to go thru a liberally planned bridge.
Note
There is one thing a well established investment and another thing is an arbitrary, unestablished but very liberal modification.

[pohl4711]

https://lczero.org/networks/
49921 3 4fd54df8 3355.00 105274 20 256 2019-07-29 17:02:08 +00:00 3355

Thanks.
Why did you choose this net for test and why did you not choose other well-known net like 42850, T40B.4-160 or -260, 40.T8.610, etc? In the latter case we can make comparison more easily to earlier tests.

Sergio will either have had it recommended to him, or chosen it due to its high performance rating there.

Really, performance and behavior of 49921 is rather similar to 42850 and 42872.
Thanks for the answers.

Wait some days for my testing result. The 3000 games testrun of lc0 0.22.0 49921 is running. Then, we will know, how strong this net really is.

https://www.sp-cc.de/lc0-testing.htm

[corres]

...
Really, performance and behavior of 49921 is rather similar to 42850 and 42872.

Wait some days for my testing result. The 3000 games testrun of lc0 0.22.0 49921 is running. Then, we will know, how strong this net really is.
https://www.sp-cc.de/lc0-testing.htm

I have some note about your test.
1, "Leela ratio" would be useful only if the testers would use the same net what was used by A0 team.
So decreasing the speed of GPU is superfluous. But it is very important to use exactly the same hardware and soft-ware to every tests.
2, For tests the usage of Hyper Trading is not a good thing because it distorts the results. This is same in the case of turbo boost of cores. They should turn off during tests.
3, As the TC is shorter as the NN engines lose their chess power relative to AB engines. Your TC is rather short and this give benefit for AB type engines.
Enhancing the number of games it lessen the deviation in Elo but if the TC is far from the practical using of engines the yielded result is no more valuable than tests with more practical TC and fewer games - even if the deviation is higher.

[Modern Times]

As long as there is no official statement from the lc0 guys "THIS is our official net. Use THIS for rating lists" you can't blame the testers for taking one of the last nets when they start testing an engine.

Exactly right. There are hundreds of nets, it is impossible to know which is the best one to use. You take a gamble and keep your fingers crossed. This sort of criticism in unwarranted. It is too easy for some people to be wise in hindsight.