NNUE - only from own engine?

[connor_mcmonigle]

What tournaments, testers, etc. want to see from engines is their doing something at least somewhat unique with respect to their evaluation functions.

Yes. I remember Ted, one of the testers, bemoaning a few weeks ago that everything tasted of fish

A question for the net experts then...

Given that at the moment there are still several flavours of net, generated in different ways using different data, as time goes on and they all improve, given enough time, won't they all converge towards a point of 'perfect' strategic play and agree on every positional evaluation?
Then they won't all taste of 'fish' but of 'perfection'?
Early days yet but could nets (eventually) be the death of computer chess?

I think differing hardware limitations will always enforce some diversity as Amanj mentioned. Additionally, doing something different from others in the hopes of striking 'elo gold' is where a lot of the fun in this originates for most "top" engine developers. Possibly networks could bring about the death of computer chess, but I'm at least tentatively hopeful this won't be the case. I know a number of fellow engine developers who are currently exploring interesting ideas related to using neural networks in chess engines.

[ChickenLogic]

What tournaments, testers, etc. want to see from engines is their doing something at least somewhat unique with respect to their evaluation functions.

Yes. I remember Ted, one of the testers, bemoaning a few weeks ago that everything tasted of fish

A question for the net experts then...

Given that at the moment there are still several flavours of net, generated in different ways using different data, as time goes on and they all improve, given enough time, won't they all converge towards a point of 'perfect' strategic play and agree on every positional evaluation?
Then they won't all taste of 'fish' but of 'perfection'?
Early days yet but could nets (eventually) be the death of computer chess?

I think differing hardware limitations will always enforce some diversity as Amanj mentioned. Additionally, doing something different from others in the hopes of striking 'elo gold' is where a lot of the fun in this originates for most "top" engine developers. Possibly networks could bring about the death of computer chess, but I'm at least tentatively hopeful this won't be the case. I know a number of fellow engine developers who are currently exploring interesting ideas related to using neural networks in chess engines.

We'll know what killed computer chess once it is dead. I still remember people saying the end is near when SF 8 was the latest and greatest. And our knowledge about chess engines does in fact also help computer shogi. One day, we may even be able to use A/B for Go without sacrificing elo compared to 'MCTS' + CNN (I know they don't use elo). And we should not to forget about FRC and Capablanca chess either.

[Steve Maughan]

I haven't given this too much thought but here's my 2 cents.

I would have thought for an engine to be considered "unique" it must have a unique net. I really don't care how this net is derived as long as it's unique. Saying that the net must only be derived from games played by the engine for it to be considered unique, seems to be unreasonably restrictive. Haven't we all used Larry Kaufman's "The Evaluation of Material Imbalances" paper to give us ideas about handling material imbalances? Isn't this analogous to using someone else's data to parameterize and tune an evaluation function? Does this invalidate the uniqueness of our engines? I don't believe it does. On a practical level, how would you prove how a net is trained? Would engine authors have to keep and document their training data?

Steve

[mclane]

Take cars. Today the cars look almost the same. Its boring.
Because car A looks and drives like car B.

Computerchess has developed into that direction.
For me only unique engines do interest me.
I am not interested in testing or watching stockfish a versus stockfish clone b or c or d.

Therefore i wish programmers would go individual ways.
No matter how or which elo.
Original or individual is the way that interests me.

The engines i like the most are original.

Mchess, rebell, tiger, colossus, sargon, Philidor, cstal, hiarcs…

[dkappe]

I haven't given this too much thought but here's my 2 cents.

I would have thought for an engine to be considered "unique" it must have a unique net. I really don't care how this net is derived as long as it's unique. Saying that the net must only be derived from games played by the engine for it to be considered unique, seems to be unreasonably restrictive. Haven't we all used Larry Kaufman's "The Evaluation of Material Imbalances" paper to give us ideas about handling material imbalances? Isn't this analogous to using someone else's data to parameterize and tune an evaluation function? Does this invalidate the uniqueness of our engines? I don't believe it does. On a practical level, how would you prove how a net is trained? Would engine authors have to keep and document their training data?

Steve

Where is the fun in training a net the same way everyone else does? Until this latest series of Dragon nets, my favorite net was also my weakest: the Harmon net which was trained on 1m 2300+ FIDE games. But until players take an interest in innovative play rather than elo, we’ll all end up producing engines with ever more similar playing styles.

[carldaman]

I haven't given this too much thought but here's my 2 cents.

I would have thought for an engine to be considered "unique" it must have a unique net. I really don't care how this net is derived as long as it's unique. Saying that the net must only be derived from games played by the engine for it to be considered unique, seems to be unreasonably restrictive. Haven't we all used Larry Kaufman's "The Evaluation of Material Imbalances" paper to give us ideas about handling material imbalances? Isn't this analogous to using someone else's data to parameterize and tune an evaluation function? Does this invalidate the uniqueness of our engines? I don't believe it does. On a practical level, how would you prove how a net is trained? Would engine authors have to keep and document their training data?

Steve

Where is the fun in training a net the same way everyone else does? Until this latest series of Dragon nets, my favorite net was also my weakest: the Harmon net which was trained on 1m 2300+ FIDE games. But until players take an interest in innovative play rather than elo, we’ll all end up producing engines with ever more similar playing styles.

Chess competition could use innovation, too. Someone mentioned FRC and Capablanca chess (though presumably that's best tried on a 10x8 board). There's also odds games, where the engines would have to play well even when down material!

But why should the top engines always go head-to-head? Let them concentrate on facing weaker opponents and then it will be all about winning and who can do the most winning!

[smatovic]

...a lill off-topic, but I can imagine that some day vintage-computer chess programming contests will appear, define a machine to run in an emulator, Atari 800 XE for example, and then squeeze the most out of it

--
Srdja

[Deberger]

I believe most of the major rating list testers (CEGT, CCRL and others) are not interested in SF NNUE unless the net is trained solely on the engine's own games

That's true.

I think the issue is important enough open a discussion.

NNUE eval is the result from :

1. Training software (freely available)
2. Quality of the EPD's (hard own work)
3. NNUE implementation (freely available)

Since the elo is in the EPD (and is the gold digging part) I see no good reason reason to put a limitation on the creative part (the quality of the EPD's). Further it is discrimination to starters who are forced to write a good HCE eval first. Everybody should be free to create his own EPD database as he pleases, train it as he pleases, implement it as he pleases.

I see only one limitation, using an existing NNUE from someone else. It means the creative (and hard) part is skipped. Fire comes to mind, I don't test it.

Lately, nearly all of the progress for SF nets have been made in the trainer - not the data. What you call the 'EPD' part (besides the only EPDs used are the opening books for data generation, EPD itself is not suitable for training)

I believe it's possible to train from an EPD database, if each position includes an evaluation.

Is this what Ed is suggesting?

Has anyone tried this?

[mclane]

...a lill off-topic, but I can imagine that some day vintage-computer chess programming contests will appear, define a machine to run in an emulator, Atari 800 XE for example, and then squeeze the most out of it

--
Srdja

I could imagine a competition who makes the strongest 8 bit chess engine.

And we even have many old engines that give a good start on these machines:
Ed schroeder, the spracklen, dave kittinger, frans morsch, …

all the many dedicated chess computer pioneers made a statement.

The competition of course limits the resources to arround 96 KB rom (bank switching) and 8-64 kb ram.

[matejst]

In the last several years, despite the existence of big cooperative computerchess projects, all the key advances were done by individual programmers: Larry Kaufman, Vaclav Rajlich, Matthew Lai, Peter Osterlund, etc. We see today the same process repeating: individuals exchange ideas, try new things, and show a high level of creativity. Since there is almost no commercial interest in developing a new chess engine, the main motivation of programmers is the TCEC competition.

I honestly believe that TCEC should reward original authors, with clear criteria for participation -- and one of them should be originality. I am not quite sure where the limit -- the so-called "line in the sand" is -- but it should be traced clearly, not to accommodate one participant but for the good of computerchess in general, and it cannot be done ad hoc, like it is today, when there are different rules for some participants, and other rules for the others.

Of course, the TCEC organisers are free to do what they want, and who am I to tell them what it is.