Raphexon wrote: ↑Fri Oct 09, 2020 2:19 pm
Dann Corbit wrote: ↑Fri Oct 09, 2020 12:09 pm
Alayan wrote: ↑Fri Oct 09, 2020 11:47 am
It's not "a new evaluation term". A new evalution term doesn't discard the rest of the eval, it doesn't kill eval originality. It's a drop-in eval replacement.
What is "eval originality"?
Is there some term that chess engine programmers know about but nobody else does?
I guess that every single evaluation term in every chess engine on earth is found in a chess book or in the mind of a GM.
That goes for what the NNs do also, but it may be something that great players do instinctively and they do not know how to voice it.
Of course I might be wrong.
Can some one give me a counter example to show how engines are doing something original that is not found in books or known by GMs?
If you take a term from a book, or from a GM you still have to translate into machine code.
True for every programming endeavor, such as tic-tac-toe as well. This seems to be an argument that although the terms are all well known, it is the implementation details that are original. Therefore if Joe writes a quicksort and Sally writes a quicksort, those quicksorts are original because the implementation details are original. But the implementation details for (for instance) Rybka were very different than the implementation details for Fruit. I would argue that the evaluation function of Rybka was not very original for the most part, even though the implementation details were different. I do not think Rybka was an example of copyright violation, but I do think the evaluation was largely unoriginal, despite the differences in implementation details.
You still have to decide (or tune) the values.
There's a lot of manual labour and tinkering.
Or is the originality contained in a framework that does 10,000 games to find the constant while you are sleeping?
I do not disagree that there is work involved. I find it hard to see the originality in this.
And if programmers just guessed on evaluation terms like they did in the stone ages of chess engines, we would see lots of variation but the engines would lose strength because of this. The idea of running a zillion games to find a number for an evaluation constant was made popular by Vas. Some others might have been doing it, but they did not explain it to the world. So I would say that Vas was original here, and everyone else is simply copying what he did. Highly unoriginal. That is OK, though. There is nothing wrong from learning a good idea and then doing it.
Everybody does this differently.
I was under the impression that they all used one of about three different testing frameworks to drive out the value that they are searching for.
Besides Fishtest, Andrew's framework, and the Texel framework, I suppose there might be others but it looks to me like they all do the same thing.
NNUE disconnects art from eval, instead of painstakingly translating chess terms to machine code you're just feeding data into a calculator that returns a function to best approximate that data.
I agree that term by term evaluation functions written by brute force examination of each constant are very labor intensive to write.
I also agree that not everyone will choose the same collection of items to include in the evaluation function.
I might add the Berliner idea of edge pawns being worth less in the opening and more in the endgame to my engine or not.
Is that what makes it original?
NNUE is really a technique that is a labor saving device. Someone doing Shogi came up with a brilliant way to use the NN to compute an evaluation with much less manual labor than even the frameworks. I think there might be very different ways to write NN tools and they can be utilized in ways we did not imagine yet. And if NNUE is bad for chess, then what about the NN approach in Alpha Zero, LC0, and various other attempts?
While some of the offshoots may be rather copy-like, there are also other attempts that are looking in new directions.
Fat Fritz is looking at different ways to train the network. There are approaches that are trying to vary how it searches. There are different sized networks, from tiny to titanic. All of these approaches produce different results.
I guess if we knew the eval terms that we are missing, then the hand tuned evaluation would be just as strong as the NNUE evaluations.
I look at the NNUE stuff as a new labor saving device, like tablebase files, but one that is much more customizable and trainable than a tablebase. In short, it is a new and revolutionary technique that makes chess programming much easier.
Now that NNUE is known to the world, we are never going to put that genie back into the bottle.
If people want to, they can ban all NN programs from contests and insist on evaluation functions tuned by a million games.
But that is just as excluding as banning NN programs. Not everyone can afford to buy a giant raft of computers or even to rent them.
So I guess what we will be left with in the end is SF verses LC0 for every contest from now to infinity.
After all, they have the most effort into tuning those engines and are therefore the most "this is original due to hard work" engines on the planet.
I want to make something clear. I admire the hard work of people like Andrew. I also admire the hard work of people who write a 1400 Elo chess engine. I am not dismissive of anyone's efforts. I actually have hopes for a new openness, that is spawned by these new techniques.
To me, anyone who learns how to make code play the game of chess and is not just doing some lame copy/paste job is doing something both interesting and admirable.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
