StockFiNN Release and Scaling

[jjoshua2]

Same conditions. 10' + 10" with the strongest gek nneu I could configure in my testing

Code: Select all

Score of stockfish.nnue.halfkp_256x2-32-32-modern vs stockfish_20070321_x64_modern: 77 - 72 - 171 [0.508]
Elo difference: 5.4 +/- 26.0, LOS: 65.9 %, DrawRatio: 53.4 %

Seems weaker by the entire error bar or more. (Comparing to the 10+10 test I did earlier and L Kauffman's.) The good news is it seem's stronger than CCC where it was 50 or 60 elo weaker than SF-dev, maybe a worse binary or bad scaling with threads.

[Alayan]

Josh isn't doing anything illegal and he's abiding by the software licenses while producing something valuable.

But my personal policy is that I would never pay a cent for any Stockfish derivative. Most of the work come from the original SF devs, plus here significant work by the japanese NNUE authors. Even if somebody adds something on top (training a good net), paying for it while the original devs, without whom nothing would have been possible, have given their time for free feels wrong.

Having myself contributed to multiple open-source project, I know I don't mind having given my time to improve a software I value, but if others contributors got paid for their work and I was not, I would feel slighted and unhappy. It's not about getting paid for work, it's about getting paid for a product whose value largely comes from others' work, who aren't getting paid a dime.

[jjoshua2]

In the modern world more and more what is valuable is not the algorithms or the binaries, but the datasets. Facebook and Google started off with some decent algorithms but the only reason why they are worth anything nowdays is because they have vast quantities of search data or user data accumulated over the years they can do stuff like use a public machine learning algorithm with a few tweaks on billions of samples and have a state of the art result. It will be interesting to see if as machine learning techniques get more advanced and maybe be able to be learn from only millions of samples if value will shift elsewhere...

[dkappe]

Producing training data can take days, months and years. I’ve been producing data and training Bad Gyal for over a year. It’s much more expensive than writing and distributing a0lite (for which Bad Gyal 8 is the main net).

I don’t charge for Bad Gyal, not even to defray my electricity, hardware and connectivity costs. I can’t believe that jjosh is even close to breaking even on LS and StockFiNN. I certainly don’t begrudge him the financial support that his patrons give him (and I actively support some blues artists through Patreon, BTW). And I certainly won’t stand for scolding by legal idiots who have no idea what the GPL actually means.

[jjoshua2]

Yes, the only reason I was able to make StockFinn at all was because of past support from Leelenstein patrons. Generating, correcting, and rescoring, dataset to support state of the art research will always be expensive. And gathering pgns to start with is always time consuming. But the good news is it doesn't really matter what the top games are generated with whether by humans or Stockfish and lc0, it should always be possible to gather enough of them to train a good neural network from them with enough time, whether from generating from scratch or rescoring some PGNs. Current law does not allow copyright of the actual moves played, only of annotation/commentary.

I'm excited to see some other similar projects and I hope there will be a few that are sufficiently different to not have a very high move similarity score to keep things interesting! And I'm happy to help people with projects that go along with this goal!

[jjoshua2]

I think in the future clones will be determined based on architecture and dataset used, not exact learning techniques or search, which only modify the results a bit, and less over time as it all improves. So to generate an engine with a new perspective a new dateset and architecture will generate the most move diversity, rather than a new binary.

[peter]

Just see 0.2 play in TCEC- Bonus already right now.

Good luck!

[carldaman]

Where do you see StockFinn playing? I took a look at TCEC and I see other engines there and no mention of 'FiNN.

[Paloma]

> Where do you see StockFinn playing?

On TCEC:

Code: Select all

AllieStein v0.7_dev2-net_15.0		0,5
StockfishNNUE 20200704-StockFiNN-0.2	0,5
Stockfish 202007032109			0
LCZero v0.26.0-sv-t60-4229-mlh		0
Stoofvlees II a14			0

[Chessqueen]

Where do you see StockFinn playing? I took a look at TCEC and I see other engines there and no mention of 'FiNN.

It is playing verses Stockfish at the moment and versus Alliestein, LCZero https://tcec-chess.com/