In defense of Pedro

[Frank Quisinsky]

Hi Modern Times,

situation is interesting.
With Fire you can't beat Fire ... you need Winter but the programmer still make a break I think.
So you need water!

Three programs with three different styles fighting vs. Fire for the much sought-after rank 3.
I think Koivisto is stronger in version 8.9 I have in my test.
Berserk 9 dev3 is very in the near and SlowChess 2.9 very in the near.
The current Ethereal isn't in my test but Andrew is one of best we have and can beat it too.

If programmers not like the way the Fire programmer walked on the way ...
The answere can be give on the chessboard.

rofChade, Revenge, Seer and RubiChess are also in the near.
Also great chess programs!

To create an own enemy image helps own motivation (for many people very important).

That's all ...
Unfortunately, the Fire programmer give his name ... I can give my name too if other need that, why not!

As to start discussions about work of others with his NN files and the reasons why the persons do that ...
A discussion about own copyright's is much more interesting.

The chess comunnity was in all the years very strong.
To copy and paste questionable copyrights is very weak.

Much more interesting if younger programmers search a better way to protect his own software if they like the way of open-source, github pages and so on. That's my personal opinion!

Best
Frank

Furthermore, I like the work the Fire programmer do all the years ... not a secret!
With the great eval from an older Fire version FEOBOS final results are much more stable!

[expositor]

Months ago I created a programmer poll on this issue and the feedback was disappointing, only 8 programmers responded to the 3 choices.

1. My preference is that I am allowed to use any existing network for my engine.
2. My preference is that I am allowed to use any game collection to create my own network.
3. My preference is that I am in favor that any programmer creates his own network either from human game collections or from his own engine.

I didn't see the poll, but I would have voted for 2. There are many different goals to pursue as a chess programmer: strength, originality, learning, research, and so on. I think it's important we don't make anyone feel unwelcome simply because they don't have the same goals, priorities, or interests that we do; chess programming is a hobby, and hobbies are meant to be enjoyable.

[By that reasoning, I suppose I should vote for 1, but it makes me feel a bit uneasy because it's dangerously close to plagiarism. If you get permission to use another project's network, though (or if the network has a license and you abide by the license), I think it's fine. I'm a huge proponent of libre software, but it's also important to be respectful.]

I do think 3 is something good to strive for. Expositor's network was trained primarily on Lichess data (human games) scored by Stockfish, but I'm currently in the process of bootstrapping from an HCE and self-play.

And of course, tournament organizers can make whatever rules they'd like in re eligibility.

[AndrewGrant]

People are "allowed" to do whatever they want. Just personally, and this is a thought shared by many, its much more interesting to teach your own child how to play chess than it is to just take someone else's kid.

Everyone should strive for #3. Especially if you are starting from nothing. You think Stockfish guys use Leela data because they don't know any better, don't know how to generate there own? No. They use Leela data because despite having the ability to generate extremely high quality data, they know Leela data for them is just a bit better.

I disagree with it, but at its a tactical decision, not an expedient one.

[dkappe]

Everyone should strive for #3. Especially if you are starting from nothing. You think Stockfish guys use Leela data because they don't know any better, don't know how to generate there own? No. They use Leela data because despite having the ability to generate extremely high quality data, they know Leela data for them is just a bit better.

After the original RL effort ran out of gas, the SF project wasn’t able to generate data that moved the needle. Enter Fat Fritz 2 and the collaboration between leela and sf was born in a panic. The original nets combined leela data and some SF data that resembled the dodgy position approach I used for my Ender nets back in 2020.

All that high quality SF data you talk about is generated by nets trained largely on lc0 data. SF hasn’t learned how to do effective RL yet. Some day they will. Maybe they can start over from scratch then with SF11 data.

[expositor]

Maybe the question should be clarified: is the question
  Would I prefer to live in a world where developers only train on self-play data, or am I indifferent?
or is the question
  Is it wrong for developers to train on data that they didn't create or that they created with the help of tools they didn't write?

I suppose I was thinking about the second question, not the first.

Personally, I'd prefer to live in a world where engines were more unique. I'm not sure I'd say developers should avoid training on data they didn't create, though, because I don't believe I should dictate the terms of someone's hobby (e.g. someone may not be interested in neural networks but intensely interested in search techniques that rely on having stable, accurate evaluations). That's assuming harm or damage isn't being done – license violations are illegal and making false claims of originality or ownership (explicitly, implied, or by omission) is immoral, which I think we all agree on.

[expositor]

It might be worth adding: if someone asked me for advice, I would recommend they try to make their engine original, because I expect that would be more satisfying to them. But it's possible they'd not be like me in this regard, in this instance.

And to be clear, I get salty when it feels like someone is getting great results with seemingly little effort – because they studied another engine's source or used a library or used an existing network – when I spent a lot of time trying to figure something out for myself! Conversely, I feel guilty whenever I feel that Expo's performance increased disproportionately to the amount of work I put in.

You could say these are "me" problems; I think they largely are (particularly the first). But they can also be community problems. I was speaking with jtwright about this topic and he made the good point that "getting a head start", so to speak, can contribute to others' imposter syndrome. I think the solution for this is also improved discussion and disclosure – recognizing that we all stand on the shoulders of our peers and predecessors. Unrelatedly, it's also easy to have the false perception that a dev accomplished something with little effort, because most work happens to be invisible. That both discredits the dev and contributes to feeling like you don't belong, are incapable, &c.

Anyway, I'm sorry to have written a wall of text; I'm mulling this over as I write.

[Frank Quisinsky]

Hi Kade,

if all the programmers give such information ... perfect for users and all the others!

Example only:

Code: Select all

NNv1 = own net based on games with own engine
NNv2 = own net based on games with own and other engines
NNv3 = own net based on games from other engines
NNv4 = own net based on games comes from humans
NNv5 = own net based on a mix of humans and engine games
--
NNv6 = net based on an other engine with own material (the programmer have to give an explanation on his github or web site).
NNv7 = net based on an other engine (the programmer have to give an explanation on his github or web site).

NNv = Neural Network variant ... x

A programmer should start such a poll, not an user.
Sure I forget important points.

Can be interesting to see that later in a rating-list:
Example: Wasp 5.54 NNv1 dev - Engine 19.9 NNv5 (fabricated)

x64 or not is today not important for engine names.
POP3 or AVX2 and so one is also not very important for the engine names.
Engine names should be short or tools have problems if engines names are to long.

Best
Frank

[Ovyron]

I think Norm has demonstrated very low sim test results Fire vs SF, demonstrating that just because a similar NNUE is used does not mean the engines become similar even though it was a good Elo gain for him.

Has anyone done a similarity test between SF's current net and the previous one? I'd expect them to return different results. Mainly, the old net used to love her queen, and often she made it to the end of the game. The new net, not so much, and often you can see how she'd rather have a knight and a rook than the queen! Of course it's always justified with better elo, so it's better reaching positions where the queen isn't that useful (at least 1 pawn less useful, so some 8.00 queen).

If the nets are different then all these discussions are moot, anybody could use some SF net with low similarity and any rule for an own net would be unenforceable.