Dann Corbit wrote: ↑Sat Apr 17, 2021 8:47 am
It appears that my point was missed entirely.
If you examine the Elo of the LC0 nets, the larger the net, the larger the Elo. And it isn't close. The big nets CLOBBER the smaller ones.
I think that bigger and bigger nets should be constructed until it is no longer true.
Similarly with SF.
Yes, Albert spent a great deal of money testing out his ideas. But the resources he could purchase are not nearly so large as the free resource of the SF mega-team has at their very fingertips.
And yet this upstart beats them all: http://www.cegt.net/40_40%20Rating%20Li ... liste.html
What did he do?
Well, one OBVIOUS thing is to double the net size.
Now, he also did special analysis to vary the input data, which is not to be discounted.
But why shouldn't the SF team double, quadruple, and so on the size of the net until no more Elo falls out of it?
To me, it seems an utter no-brainer.
2 elo diff +-16 elo
FF2 net is verifiably weaker.
Bigger nets aren't new. They just haven't been proven to be stronger than the default arch yet.
MikeB wrote: ↑Sat Apr 17, 2021 9:18 am
hmmm... I see a difference in style of play and FRC was certainly better, but Elo gain from doubling? - from whatever possible gain was obtained, it does not appear to be worth the extra 20MB of space to store... just my $.02 .. I do like the fact that Albert tried something new and maybe he is on to something, but I am not sure yet.
It's nothing new, everybody understands that a larger network can hold more information. With current computers the storage space is not a problem too, it's all a trade-off between speed and network size. At the point that the Elo gain from increasing the network size becomes less than the Elo loss from the decreased speed you don't want to make it larger anymore.
exactly, FF2 was double the size and how much Elo did we gain, it was disappointing ...
MikeB wrote: ↑Sat Apr 17, 2021 9:18 am
hmmm... I see a difference in style of play and FRC was certainly better, but Elo gain from doubling? - from whatever possible gain was obtained, it does not appear to be worth the extra 20MB of space to store... just my $.02 .. I do like the fact that Albert tried something new and maybe he is on to something, but I am not sure yet.
It's nothing new, everybody understands that a larger network can hold more information. With current computers the storage space is not a problem too, it's all a trade-off between speed and network size. At the point that the Elo gain from increasing the network size becomes less than the Elo loss from the decreased speed you don't want to make it larger anymore.
exactly, FF2 was double the size and how much Elo did we gain, it was disappointing ...
But if "...Elo loss from the decreased speed"...would larger nets still be beneficial in certain areas? Maybe for long analysis or LTC (instead of simple engine vs engine quick play matchups)?
MikeB wrote: ↑Sat Apr 17, 2021 9:18 am
hmmm... I see a difference in style of play and FRC was certainly better, but Elo gain from doubling? - from whatever possible gain was obtained, it does not appear to be worth the extra 20MB of space to store... just my $.02 .. I do like the fact that Albert tried something new and maybe he is on to something, but I am not sure yet.
It's nothing new, everybody understands that a larger network can hold more information. With current computers the storage space is not a problem too, it's all a trade-off between speed and network size. At the point that the Elo gain from increasing the network size becomes less than the Elo loss from the decreased speed you don't want to make it larger anymore.
exactly, FF2 was double the size and how much Elo did we gain, it was disappointing ...
But if "...Elo loss from the decreased speed"...would larger nets still be beneficial in certain areas? Maybe for long analysis or LTC (instead of simple engine vs engine quick play matchups)?
Yes, I can imagine that for analysis without time constraints a larger net could be beneficial. It also depends upon the data used for training, if the information that the net can hold is larger than the information contained in the training data it will probably be counterproductive.
Dann Corbit wrote: ↑Sat Apr 17, 2021 8:47 am
It appears that my point was missed entirely.
If you examine the Elo of the LC0 nets, the larger the net, the larger the Elo. And it isn't close. The big nets CLOBBER the smaller ones.
I think that bigger and bigger nets should be constructed until it is no longer true.
Similarly with SF.
Yes, Albert spent a great deal of money testing out his ideas. But the resources he could purchase are not nearly so large as the free resource of the SF mega-team has at their very fingertips.
And yet this upstart beats them all: http://www.cegt.net/40_40%20Rating%20Li ... liste.html
What did he do?
Well, one OBVIOUS thing is to double the net size.
Now, he also did special analysis to vary the input data, which is not to be discounted.
But why shouldn't the SF team double, quadruple, and so on the size of the net until no more Elo falls out of it?
To me, it seems an utter no-brainer.
2 elo diff +-16 elo
FF2 net is verifiably weaker.
Bigger nets aren't new. They just haven't been proven to be stronger than the default arch yet.
There are a number of experiments conducted on the Rybka forum that show a direct relationship between net size and Elo increase.
Also, when it comes to solving tactical problems bigger nets are the clear winner.
I think we do not know what the best net size is. That is why I suggest trying lots of different sizes, including really large ones.
I guess that the hardware is important too. A cell phone is probably going to want a different net than a 256 core server,
and for LC0, a little card or running on CPU power will be different from two or three of the latest and greatest cards.
When you look at the Elo curves, they seem hardware limited and size limited. So, for instance, after a certain number of data points are fed into it, the strength appears to stop growing. So we have hit some kind of limit. Is the limit due to the size of the net? Is it due to the hardware capability? I guess that it is some of both.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Dann Corbit wrote: ↑Sat Apr 17, 2021 10:58 pm
There are a number of experiments conducted on the Rybka forum that show a direct relationship between net size and Elo increase.
Also, when it comes to solving tactical problems bigger nets are the clear winner.
I think we do not know what the best net size is. That is why I suggest trying lots of different sizes, including really large ones.
Perhaps a commercial engine, like Komodo Dragon, would do well to offer 2 nets with their engine for variety and needs of the end user :
1. Smaller, speedier for 'general game play'/ELO chasing
2. Larger, for LTC's and Serious Analysis
MikeB wrote: ↑Sat Apr 17, 2021 9:18 am
hmmm... I see a difference in style of play and FRC was certainly better, but Elo gain from doubling? - from whatever possible gain was obtained, it does not appear to be worth the extra 20MB of space to store... just my $.02 .. I do like the fact that Albert tried something new and maybe he is on to something, but I am not sure yet.
It's nothing new, everybody understands that a larger network can hold more information. With current computers the storage space is not a problem too, it's all a trade-off between speed and network size. At the point that the Elo gain from increasing the network size becomes less than the Elo loss from the decreased speed you don't want to make it larger anymore.
exactly, FF2 was double the size and how much Elo did we gain, it was disappointing ...
But if "...Elo loss from the decreased speed"...would larger nets still be beneficial in certain areas? Maybe for long analysis or LTC (instead of simple engine vs engine quick play matchups)?
I don't think we really know yet, but yes, that is the hope anyway.
Cornfed wrote: ↑Sat Apr 17, 2021 5:50 pm
But if "...Elo loss from the decreased speed"...would larger nets still be beneficial in certain areas? Maybe for long analysis or LTC (instead of simple engine vs engine quick play matchups)?
Begs the question on hew well FF2 might have done if allowed to compete in TCEC or CCC with monster hardware. I don't follow either of those but I assume it didn't compete under their rules.
