hgm wrote: ↑Tue Feb 16, 2021 11:01 am
towforce wrote: ↑Tue Feb 16, 2021 12:30 amObviously, if a file size is halved, then it cannot store as much data. If there is a linear relationship between the file size and amount of chess knowledge, then the half sized file would only know about half as much about chess.
Not true, because the size of the net only tells you how much knowledge it
could hold. Not how much it actually holds. An untrained net would not hold any chess knowledge, no matter how large it is.
I don't think it hasbeen proven that the full-size net is completely saturated, for the given training.
And size of the net is one thing, but topology is another. It could for instance be that adding an extra layer has much more effect on the amount of knowledge the net can store than doubling the width.
A simplification: there's a new skill, and A and B both want to learn it, and they both have the same size of brain:
A is able to learn it easily, but B isn't, and this is because their previous learning/experience is different. A has previous learning which helps him to learn the new skill, and B does not. Worse still, B has previous learning that draws him away from the required new pattern.
Next, an animal trainer tries to teach an animal the new skill, but is unable to: the skill requires a human brain size, and the animal does not have this.
There's a lot going on with regard to nets and learning - and size is certainly a factor.
