Final Release of Ethereal, V12.75

[Rebel]

An hour after I make this post I'm filled with regret.

[1] Do people really want to see a bunch of NNUE engines, all running the same evaluation function, occupying the top slots of computer chess?
[2] Would people really find it interesting to have Ethereal use whatever "best" network is out there?
[3] Are people still interested when engines replace 50% of what made them unique?
[4] Do people want to continue to have Ethereal, acting as a Stockfish-lite, for new programmers?
[5] Is Chess doomed to go the route of Shogi and Go? We share the Networks, but have our own searches? Is that okay?
[6] Is this chess world okay with me plunking in a Network file and never changing it again?

[7] Maybe the NN guys are currently testing NNUE and will switch

If someone can convince me of all of those things, or at least most of them, then maybe there is still work to be done. The OCD in me

Be happy what you have achieved, go and never look back, it's not worth it.

[Alayan]

It's not "a new evaluation term". A new evalution term doesn't discard the rest of the eval, it doesn't kill eval originality. It's a drop-in eval replacement.

Is there no way for originality in NNUE development?

Someone could attempt to have significantly different input features (so far no one has). How much originality a specific architecture leaves when training different nets AND training them for strength is a matter of debate, my main claim in this regard is that it's intrinsically lower than with distinct architectures.

I mostly wanted to emphasize that an additive eval term (add this on top of an existing eval) and a drop-in eval replacement (remove all your current eval) have a very different impact.

[Dann Corbit]

It's not "a new evaluation term". A new evalution term doesn't discard the rest of the eval, it doesn't kill eval originality. It's a drop-in eval replacement.

What is "eval originality"?
Is there some term that chess engine programmers know about but nobody else does?
I guess that every single evaluation term in every chess engine on earth is found in a chess book or in the mind of a GM.
That goes for what the NNs do also, but it may be something that great players do instinctively and they do not know how to voice it.

Of course I might be wrong.

Can some one give me a counter example to show how engines are doing something original that is not found in books or known by GMs?

[Scally]

Hi Andrew,

I’ve just compiled your EtherLito NNUE version for the Raspberry Pi, it’s version Number is 12.69, is that correct or should I change the version to 12.75?

Best wishes on your new endeavours ....


Al.

[alex67a]

Stockfish plays one way, leela another ...
Neural networks are a relatively young field, and I think there is still a lot to discover

[mvanthoor]

A lot of people who have spent a lot of hours toiling on evaluation are keenly disappointed.

...

So stop jealously guarding your evaluation functions, which are made of terms extracted from any decent chess book, and focus on the really interesting part of a chess engine.
Search.

No, search has nothing to do with chess. Bit boards and other newer board representations haven't either. They're just a way to see deeper by cutting the tree or going faster. In the end, you don't need an evaluation at all. If you extrapolate to the logical conclusion, the only thing you need to have is a material count, because if you can see far enough ahead, you can always know who is better in what position.

But that's not how humans play chess.

The complaint most often heard about chess engines (and Go engines even more so) is that they don't play "like a human"; the neural networks do that better. The reason is that they search less and evaluate more.

For me, the evaluation gives the program its personality. A better search just gives it more speed. Have you seen this topic?

http://talkchess.com/forum3/viewtopic.php?f=2&t=75324

I doubt that all of those sacs are due to search speed. That engine KNOWS about open lines and moving pieces with tempo; it actively sacrifices material for huge positional gains. That is not something a search can reveal, unless you can go all the way to the end (which it probably can't).

I'd be thrilled if I can make my engine play like Thinker in the above example.

[JohnW]

Thank you for your efforts, Andy.

If your main goal is to compete with a hand-crafted engine against another hand-crafted engine, as was the tradition from 1968-2018, then it is a good time to quit. With neural network engines, a stronger engine can be created by just throwing more hardware at training the network, and to train it for longer with more games and data. The fun of creating and hand-crafting your engine from scratch, teaching it better chess with each commit, is gone. The creation of a super strong chess engine has been automated.

Fortunately for me, I'm writing my own chess engine with a different goal.

You know of that tradition, where people spend years of effort to learn penmanship, where people use flex-nib (fountain) pens to write beautiful documents, like it's 1850? In current society, there isn't a lot of use for that, but some people spend the effort because of the sheer beauty of it.

I'm writing my chess engine in the same vein: in the tradition of "teaching a computer to play chess", completely by hand, to see the engine get stronger with each commit or added eval term... things *I* taught it. I want to write a massive website / book collecting everything from A-Z that goes into this chess engine, so there's basically a manual called "How to write a classic chess engine", with all the information in one place.

The reason I can do this is because my goal is different than yours; I write this chess engine to be my personal chess computer. (Running on a Raspberry, connected to a DGT board.) That is the primary goal, and making it stronger is just to see how far I can take this.

At some point I may look into a way to write a replacement for PicoChess on the Raspberry Pi... because I detest the fact that it's written in Python.

Good luck with whatever you choose to do. Writing Go programs is (almost) as useless now, if you're doing it for the competition, because they're all based on MCTS and / or neural networks now.

I agree with most of what you said. I just started writing my own chess program just to see how far I can take it but I am intentionally not looking at the publicly available source code of other engines because I want the engine to be uniquely mine. It will probably never be that strong but I will be very proud if I can one day make it strong enough to consistently beat me.

[Raphexon]

It's not "a new evaluation term". A new evalution term doesn't discard the rest of the eval, it doesn't kill eval originality. It's a drop-in eval replacement.

What is "eval originality"?
Is there some term that chess engine programmers know about but nobody else does?
I guess that every single evaluation term in every chess engine on earth is found in a chess book or in the mind of a GM.
That goes for what the NNs do also, but it may be something that great players do instinctively and they do not know how to voice it.

Of course I might be wrong.

Can some one give me a counter example to show how engines are doing something original that is not found in books or known by GMs?

If you take a term from a book, or from a GM you still have to translate into machine code.
You still have to decide (or tune) the values.
There's a lot of manual labour and tinkering.
Everybody does this differently.


NNUE disconnects art from eval, instead of painstakingly translating chess terms to machine code you're just feeding data into a calculator that returns a function to best approximate that data.

[smatovic]

It's not "a new evaluation term". A new evalution term doesn't discard the rest of the eval, it doesn't kill eval originality. It's a drop-in eval replacement.

What is "eval originality"?
Is there some term that chess engine programmers know about but nobody else does?
I guess that every single evaluation term in every chess engine on earth is found in a chess book or in the mind of a GM.
That goes for what the NNs do also, but it may be something that great players do instinctively and they do not know how to voice it.

Of course I might be wrong.

Can some one give me a counter example to show how engines are doing something original that is not found in books or known by GMs?

If you take a term from a book, or from a GM you still have to translate into machine code.
You still have to decide (or tune) the values.
There's a lot of manual labour and tinkering.
Everybody does this differently.

I think all of this also counts for training/implementation of NNs.

NNUE disconnects art from eval, instead of painstakingly translating chess terms to machine code you're just feeding data into a calculator that returns a function to best approximate that data.

Maybe we figure out that copying a network for use is as copying others PSQT, might work, but it is better to tune this against once own engine with unique search and params, or alike.

--
Srdja

[alex67a]

Hi Andrew,

I’ve just compiled your EtherLito NNUE version for the Raspberry Pi, it’s version Number is 12.69, is that correct or should I change the version to 12.75?

Best wishes on your new endeavours ....


Al.

Hi Scally,

can you give to me a link for the executable ?
I want try the engine

thx