Time to rethink what Albert Silver has done?

[smatovic]

Yo, into wasps' nest, I am not into the details, but I read that Stockfish uses now Lc0 data for training, and doubled their net size. Time to rethink what Albert Silver has done? As far as I got it he used FatFritz 1 (Lc0 derivative) data for training and doubled the net size of the FatFritz 2 (Stockfish derivative) network.

--
Srdja

PS: not interested in discussing the marketing of ChessBase for FF2.

[mclane]

But stockfish is not sold.
Fat fritz was a commercial product.

[smatovic]

But stockfish is not sold.
Fat fritz was a commercial product.

I can blame ChessBase for their commercial practices, I can blame Albert Silver for participating in this, and I can acknowledge what Albert Silver has done on a technical plane?

--
Srdja

[Collingwood]

I can blame ChessBase for their commercial practices, I can blame Albert Silver for participating in this, and I can acknowledge what Albert Silver has done on a technical plane?

He hasn't done anything on a technical plane. He just stole others' technical work. And then he lied about it. And then he doubled down.

[smatovic]

I can blame ChessBase for their commercial practices, I can blame Albert Silver for participating in this, and I can acknowledge what Albert Silver has done on a technical plane?

He hasn't done anything on a technical plane. He just stole others' technical work. And then he lied about it. And then he doubled down.

Hmm, did Albert come up with the idea for training a SF NNUE network with Lc0 data? Did he (or his comrades) implement this on a technical plane via a doubled net size? Does SF 14 use now Lc0 data for training, did SF 14 doubled the net size (with a new nn structure)? Just some questions.

--
Srdja

[AndrewGrant]

There are much more qualified and honest people that hang around in the Stockfish and Leela communities, whose insights would be of significantly greater benefit than anything Albert could offer up. Calling upon him is ... some mixture of consulting the wolf on how to protect the chickens, and voting your local dog catcher into the presidency.

[hgm]

I think the criticism on Albert was not so much for what data he used to train a network, but that the network was sold in combination with an engine that was not his own.

[Madeleine Birchfield]

Yo, into wasps' nest, I am not into the details, but I read that Stockfish uses now Lc0 data for training, and doubled their net size. Time to rethink what Albert Silver has done? As far as I got it he used FatFritz 1 (Lc0 derivative) data for training and doubled the net size of the FatFritz 2 (Stockfish derivative) network.

Fat Fritz 2 using Leela data was never the problem. What was the problem was that Fat Frotz 2 used Stockfish's search and most of Stockfish's handcrafted eval and inference code, making Fat Fritz 2 a Stockfish derivative, and then Albert and Chessbase violated Stockfish's GPL licenses by not providing source code and falsely advertising Fat Fritz 2 as a unique engine.

[Sopel]

Yo, into wasps' nest, I am not into the details, but I read that Stockfish uses now Lc0 data for training, and doubled their net size. Time to rethink what Albert Silver has done? As far as I got it he used FatFritz 1 (Lc0 derivative) data for training and doubled the net size of the FatFritz 2 (Stockfish derivative) network.

--
Srdja

PS: not interested in discussing the marketing of ChessBase for FF2.

1. He used leela (FF1) data because he had nothing else to use with the data being generated by himself. It just so happens that we independently discovered that lc0 data works well in our case. This is quite possibly to the MCTS evaluation being more well-behaved and being a closer approximation of the ideal training target than AB evaluation is.

2. Trying to increase the net size is a no brainer once the training and testing procedure is established and more optimizations to the NNUE inference code is made. This was pretty much known from the beginning, as jjosh was training larger nets well before most people even knew about NNUE. It was just infeasibly slow to train for a long time and it was hard to catch SVs nets even with this change (FF2 net didn't accomplish this too, later vondele's nets with the same size were also ~10 elo below). There is nothing creative here.

[Albert Silver]

Yo, into wasps' nest, I am not into the details, but I read that Stockfish uses now Lc0 data for training, and doubled their net size. Time to rethink what Albert Silver has done? As far as I got it he used FatFritz 1 (Lc0 derivative) data for training and doubled the net size of the FatFritz 2 (Stockfish derivative) network.

--
Srdja

PS: not interested in discussing the marketing of ChessBase for FF2.

1. He used leela (FF1) data because he had

... the idea of converting it into NNUE usable data to train a network. He then rented 10 2080ti GPUs on Vast.ai for several months (at a cost of thousands of dollars) to generate the data needed to do this, since of course the FF1 data he had was completely insufficient as maybe only 300 thousand games were from the final net's full strength, the rest was much weaker, and therefore not of interest. Except to test the concept.

It just so happens that we independently discovered that lc0 data works well in our case.

A miracle!

2. Trying to increase the net size is a no brainer once the training and testing procedure is established and

... demonstrated by someone else first.

This was pretty much known from the beginning, as jjosh was training larger nets well before most people even knew about NNUE.

Quite true, and his result was some 50 Elo weaker. As a result, the popular belief was that it was a dead end as was repeatedly told anyone asking about larger net sizes in the SF Discord. Vondele's attempt to reproduce my result was also much weaker, but of course I was using higher quality data.

Still, as I mentioned elsewhere, I felt it would inspire others to try and of course improve my ideas and the proof is there. I'm genuinely glad.