Are neural nets (the weights file) copyrightable?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
hgm
Posts: 27790
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Are neural nets (the weights file) copyrightable?

Post by hgm »

towforce wrote: Sun Feb 28, 2021 6:14 pmPart of SF does shine through in the FF2 weights table: they're using an NN that similar in most important ways to the NN that SF was using.
Similar how? Because it used the same network topology? That sounds like "Your book is a copyright violation of mine, because you printed it on A3 paper, while mine was issued on A4!".
More importantly, the section of the GPL FAQ that I previously linked makes it clear that if you take a GPL bundle and modify, or even "completely replace" a part of it, the new bundle has to be under GPL as well.
Well, as I pointed out that same FAQ mentions that it is perfectly possible to do exactly that, if the bundel is a GPL'ed interpreter plus a GPL'ed program in the language it interprets, and you replace the latter. Only when the latter would call into libraries written in the interpreted language that are also GPL (and not LGPL!), then it would not be allowed. Because you had not replaced the part in the interpreted language entirely.
syzygy
Posts: 5557
Joined: Tue Feb 28, 2012 11:56 pm

Re: Are neural nets (the weights file) copyrightable?

Post by syzygy »

https://web.archive.org/web/20060218021 ... -paper.pdf
A driver loaded as a module into the Linux kernel?

Moglen: No
Stallman: Yes. I think Linus made a mistake when he said he would interpret the GPL so as to regard these as NOT extensions to Linux.

A module written to be plugged into an API defined specifically to support dynamic loading?
Moglen: No.
Stallman: It depends on the detailed circumstances.

A program which uses a library? (i.e., Would it be fair to say that this is generally not a derivative work of that library?)
Moglen: This depends. Code statically linked to other code is a derivative work of the code with which it is linked, as far as we are concerned. But what follows from that depends on the license involved. That's the difference between the GPL and the LGPL, which is designed to permit proprietary code to be linked and distributed with free libraries.
Stallman: I think that depends on the detailed circumstances.

A library linked to a program? (i.e., Is this a derivative work of the program?)
Moglen: Code statically linked to code constitutes a derivative work of the code to which it is linked, without question, regardless of license terms. More specifically, now regarding licensing as well as the status of the work, code that cannot be used at all unless dynamically linked to GPL'd code, and which is distributed along with that GPL'd code, must be distributed under the terms of the GPL. This provides a competitive advantage to free software, requiring those who wish to make unfree software to undertake proprietary reimplementation of feature sets only available in GPL'd libraries, such as GNU readline.
Stallman: That is normally true, but it one needs to be careful drawing conclusions from it. Also, there can be ambiguity in the definition of "library" which can cause confusion about this.

A program running as a process on a Linux system? (A derivative work of the Linux kernel?)
Moglen: Certainly not a derivative work of the kernel, or of anything else, simply by virtue of executability on free software systems, whether the kernel they employ is Linux, the Hurd, or some other kernel.
Stallman: I agree here [that the program running as a process on a Linux system creates a derivative work]--except that it's a misnomer to speak of "a Linux system". Linux is the kernel; the system is really the GNU operating system, modified to use Linux as the kernel.
gonzochess75
Posts: 208
Joined: Mon Dec 10, 2018 3:29 pm
Full name: Adam Treat

Re: Are neural nets (the weights file) copyrightable?

Post by gonzochess75 »

syzygy wrote: Sun Feb 28, 2021 8:28 pm https://web.archive.org/web/20060218021 ... -paper.pdf
A driver loaded as a module into the Linux kernel?

Moglen: No
Stallman: Yes. I think Linus made a mistake when he said he would interpret the GPL so as to regard these as NOT extensions to Linux.

A module written to be plugged into an API defined specifically to support dynamic loading?
Moglen: No.
Stallman: It depends on the detailed circumstances.

A program which uses a library? (i.e., Would it be fair to say that this is generally not a derivative work of that library?)
Moglen: This depends. Code statically linked to other code is a derivative work of the code with which it is linked, as far as we are concerned. But what follows from that depends on the license involved. That's the difference between the GPL and the LGPL, which is designed to permit proprietary code to be linked and distributed with free libraries.
Stallman: I think that depends on the detailed circumstances.

A library linked to a program? (i.e., Is this a derivative work of the program?)
Moglen: Code statically linked to code constitutes a derivative work of the code to which it is linked, without question, regardless of license terms. More specifically, now regarding licensing as well as the status of the work, code that cannot be used at all unless dynamically linked to GPL'd code, and which is distributed along with that GPL'd code, must be distributed under the terms of the GPL. This provides a competitive advantage to free software, requiring those who wish to make unfree software to undertake proprietary reimplementation of feature sets only available in GPL'd libraries, such as GNU readline.
Stallman: That is normally true, but it one needs to be careful drawing conclusions from it. Also, there can be ambiguity in the definition of "library" which can cause confusion about this.

A program running as a process on a Linux system? (A derivative work of the Linux kernel?)
Moglen: Certainly not a derivative work of the kernel, or of anything else, simply by virtue of executability on free software systems, whether the kernel they employ is Linux, the Hurd, or some other kernel.
Stallman: I agree here [that the program running as a process on a Linux system creates a derivative work]--except that it's a misnomer to speak of "a Linux system". Linux is the kernel; the system is really the GNU operating system, modified to use Linux as the kernel.
You forgot to add the next line. Here you go:
To be fair, my questions came without context or clarification, so a certain amount of divergence could be expected.
Also, this was about GPLv2 not v3 which we've already discovered clarifies the questions about "derivative work" by making it global and providing its own definitions. Looking fwd to you putting skin in the game.
User avatar
towforce
Posts: 11561
Joined: Thu Mar 09, 2006 12:57 am
Location: Birmingham UK

Re: Are neural nets (the weights file) copyrightable?

Post by towforce »

hgm wrote: Sun Feb 28, 2021 6:58 pm
towforce wrote: Sun Feb 28, 2021 6:14 pmPart of SF does shine through in the FF2 weights table: they're using an NN that similar in most important ways to the NN that SF was using.
Similar how? Because it used the same network topology? That sounds like "Your book is a copyright violation of mine, because you printed it on A3 paper, while mine was issued on A4!".
More importantly, the section of the GPL FAQ that I previously linked makes it clear that if you take a GPL bundle and modify, or even "completely replace" a part of it, the new bundle has to be under GPL as well.
Well, as I pointed out that same FAQ mentions that it is perfectly possible to do exactly that, if the bundel is a GPL'ed interpreter plus a GPL'ed program in the language it interprets, and you replace the latter. Only when the latter would call into libraries written in the interpreted language that are also GPL (and not LGPL!), then it would not be allowed. Because you had not replaced the part in the interpreted language entirely.

Describing SF as an interpreter for NN weights is a stretch: if I had to argue the point in a court of law, I'd much rather be making the case that SF is not an interpreter for NN weights. For one thing, SF existed before it had an NN component.
Writing is the antidote to confusion.
It's not "how smart you are", it's "how are you smart".
Your brain doesn't work the way you want, so train it!
User avatar
hgm
Posts: 27790
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Are neural nets (the weights file) copyrightable?

Post by hgm »

Which side you prefer to argue can hardly be called relevant. That there were previous Stockfish versions that did not have a NNUE interpreter in them is not relevant: it has one now, and we are arguing about this version, not the previous one (which is not included as a copy). That it does more than just interpreting the NNUE also doesn't seem very relevant: many interpreters have done that as well. They were part of a larger programming environment, that included for example a text editor, with which you could enter the program that was interpreted. They could even do some pre-compilation of the code, suchs as Java, to transform the Java source to 'byte code', and interpret that. (And a disassembler for the byte code. And a run-time debugger...) I am very confident the case mentioned in the GPL FAQ is intended to cover exactly that situation, when I would package it with nonGPL'ed byte code.
syzygy
Posts: 5557
Joined: Tue Feb 28, 2012 11:56 pm

Re: Are neural nets (the weights file) copyrightable?

Post by syzygy »

hgm wrote: Mon Mar 01, 2021 12:00 am Which side you prefer to argue can hardly be called relevant. That there were previous Stockfish versions that did not have a NNUE interpreter in them is not relevant: it has one now, and we are arguing about this version, not the previous one (which is not included as a copy). That it does more than just interpreting the NNUE also doesn't seem very relevant: many interpreters have done that as well. They were part of a larger programming environment, that included for example a text editor, with which you could enter the program that was interpreted. They could even do some pre-compilation of the code, suchs as Java, to transform the Java source to 'byte code', and interpret that. (And a disassembler for the byte code. And a run-time debugger...) I am very confident the case mentioned in the GPL FAQ is intended to cover exactly that situation, when I would package it with nonGPL'ed byte code.
The point of the FAQ is that the interpreted code is just data to the interpreter.
It is clear that NNUE nets are just data to the NNUE evaluation function.
So whether or not the authors of the FAQ would regard the NNUE evaluation function as an interpreter, it seems they would agree that NNUE nets need not be released under the GPL.

FF2 SF/NNUE SF is a chess engine with a programmable evaluation function. That someone has actually programmed it by specifying a bunch of values does not mean that those values become an integral part of FF2 SF.
Michel
Posts: 2272
Joined: Mon Sep 29, 2008 1:50 am

Re: Are neural nets (the weights file) copyrightable?

Post by Michel »

Let me disagree.

This is just a case of dynamic linking.

A NNUE NN is a library written in a mathematical language which is part of Stockfish (when it runs on the users computer) and which is executed through an interpreter (it is conceivable that in the future there will be CPUs that can execute a NN natively, in fact GPUs already do that in some sense).

Part of SF might have been written in Python. The Python part would still fall under the GPL (if you accept that dynamic linking falls under the GPL) even though it is interpreted.

It might be that a NN is not copyrightable because of the lack of clearly defined source code. Then the GPL has nothing to say about it. But this is not because it is an interpreted library.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
User avatar
hgm
Posts: 27790
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Are neural nets (the weights file) copyrightable?

Post by hgm »

OK, you used that example before, and only now I understand what you meant.

Interpreting is not dynamic linking. Stockfish links to the NNUE interpreter (statically, I guess). Not to the interpreted code. If interpreting was linking, it would never be allowed to bundle a GPL-incompatible Python program with a GPL'ed interpreter. But the GPL FAQ explicitly says you can.

The Stockfish NNUE weight file of course is a part of Stockfish. (Although it seems the Stockfish developers have released in with a different license, albeit a GPL-compatible one.) But that doesn't matter when you don't copy it, but provide your own. Only the copyrights on the part of Stockfish that you want to distribute matters. And the GPL grants you permission to distribute it with GPL-incompatible interpreted code. As long as you don't also include other interpreted code that interacts closely with your own interpreted code, but is under GPL. (That would count as 'linking' at the interpreted level.)
User avatar
towforce
Posts: 11561
Joined: Thu Mar 09, 2006 12:57 am
Location: Birmingham UK

Re: Are neural nets (the weights file) copyrightable?

Post by towforce »

hgm wrote: Mon Mar 01, 2021 12:00 amWhich side you prefer to argue can hardly be called relevant.

Actually, what a court would decide is the ONLY thing that is relevant, and which side I'd prefer to argue goes to the fact that I would want to be on the winning side.

If you went against me in court arguing that a set of NN weights is a "language" to be "interpreted", and that SF was written as an "interpreter" for that language (with odd extra bits that are analogous to a text editor), demolishing your case and making you look like a prat would be ABSOLUTELY DELICIOUS. Life doesn't often throw up opportunities to look so superior so easily.
Writing is the antidote to confusion.
It's not "how smart you are", it's "how are you smart".
Your brain doesn't work the way you want, so train it!
User avatar
hgm
Posts: 27790
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Are neural nets (the weights file) copyrightable?

Post by hgm »

In your dreams anything is possible indeed...