A Crossroad in Computer Chess; Or Desperate Flailing for Relevance

[Daniel Shawul]

Now, in recent months, a similar series of developments are happening. NNUEs, or rather, I want to be specific, the King-Piece structures used in Stockfish, are flooding the scene. With a few hours work, an author can copy paste some code from CFish or Stockfish, add the incremental update code, and with some debugging they can begin playing games with Stockfish's Networks. In the last month a half dozen authors have posted their results from using the Stockfish structure/networks -- almost all +200 elo or more.

The less disheartening part is that none of the authors have acted like hooking up Sergio's nets to their engine is their own work.

Minic author was very clear that Minic + Sergio NNUE is not Minic. And is currently working on his own implementation.
Rubi's author seemed disheartened and was clear that Rubi+SVnet is not Rubi.
Orion's author was annoyed by CCRL testing his engine with Sergio's net: http://talkchess.com/forum3/viewtopic.php?f=6&t=75224

Igel's author is also working on his own implementation, and is currently using a net from Dkappe that mimics his Bad Gyal nets.
Pedone's author also seemed to be working on his own implementation.

Why would chess programmers care if their NNUE implementation drives Andrew to retirement?
This guy was bad-mouthing NNs for long and he jumped the wagon once Stockfish implements a version of NN ?

Programmers can do what they want, experiment with any new technology they like, etc without giving a damn about what anyone else thinks/says.
Pretty sure all the above NNUE authors were afraid of the "lynching mob" when they declared "CCRL please don't test my engine+NNUE", really!?
I am sure the testers were like, "Ahh? I wasn't expecting a thank you for my effort but your response is weird. Ok look we are just testers, and we don't judge for originality".

You can train your own NNUE on your time table if you wish to do so, while in the mean time not caring about what any one else says.
Funny thing is that if you take the time to look into the accuser's chess engine, you would find lots of questionable stuff in there.
I mean find one piece-square-table copied from another engine, and he becomes a cloner by his own standards...
All AB engines that used the ZuriChess 750k epd test positions (scored with Stockfish) for tuning are equally a clone of those who trained NNUE
nets with stockfish games.

[Madeleine Birchfield]

I think the largest problem is team size.
Stockfish and LC0 have giant teams.
Teams with highly skilled people who work for nothing.
Teams with sky-high piles of gleaming hardware worth millions of dollars, donating both hardware, time, electricity and effort in testing.

I guess that Andrew is as clever as the most clever person on either team.
But when you have 100 workers, it is hard to compete against that.
And when you have a million dollars of hardware on your team, how will a lone developer compete with that?

It is, in fact, the mega-teams that are squashing the smaller teams.
And the people taking things are taking them from the mega teams because that is the only way they can compete.

I guess that the only small entity who can beat a mega team is one who has a supremely clever idea (like the NNUE thing) and then hides what they did so that nobody can copy it.

If you even say what your approach was, someone else will try it.

The only reason why there are two mega-teams in computer chess is because AlphaZero spurred the development of Leela. Otherwise, there would only have been Stockfish. The die was cast when Marco Costalba and Joona Kiiski retired from Stockfish development in 2011, and the community chose to continue development on Stockfish as a distributed open-source project.

[AndrewGrant]

Why would chess programmers care if their NNUE implementation drives Andrew to retirement?

I'll argue with you another time about your view of Ethereal. Not today, however. Sorry

[jonkr]

I didn't read every reply on this post, but when I restarted work on SlowChess I assumed something like this was inevitable. The old version 15 years ago was in some premiere divisions for free engines and in a respectable spot on some rating lists, then after the release of Fruit, the Fruit variations started appearing (most notable Toga, originally not admitting to be Fruit) and the rating lists started to become filled with all kinds of Toga and Fruit variations.

That said it doesn't particularly bother me, I already acknowledged it was hard to find outside meaning in making a chess program. Someone could just say all the differences and new ideas only make it weaker than the current best, and all the rest is just existing ideas probably better implemented by the current best, and not necessarily be wrong. I suppose I mainly did it because it was an interesting challenge and I liked seeing my program improve relative to older versions of itself.

Copying the NNUE evaluation code (and using a strong net possibly made for Stockfish) obviously takes insanely less time than developing something yourself. But I don't expect users to care necessarily. For the nets themselves there are a lot of weights, and I think it's possible with different nets the playing style could vary a lot.

Using neural nets in general makes the process seem very detached from chess and more straight computer engineering and data management/generation. But neural nets work far better than I expected they would, so that somewhat makes up for it. Developing with neural nets feels to me less human and more reliant on the brute force calculation power of todays machines (despite sometimes a reputation in the opposite direction, I suppose from the name, and for Leela types spending so much computation power in a single node.)

Late endgame neural nets like I posted about in the programming forum are a bit less interesting now that it seems obvious the process should work (just repeat with a new net, it starts very weak and keep running test matches then training. Although as usual I mess something up along the way, the bishop endgame network still had 1 place on data export left hardcoded to ROOK... so I was wondering why it was stuck at -200 elo, now it's up to -25 elo after fixing.) Out of stubbornness I expect to continue to try to accomplish my goal of beating SF11 in a general endgame suite test match, though will probably take over a 1 month to accomplish, and not sure how much overall elo it will gain, or what issues it will have from overfitting or working nicely with rest of program in whole games.

[jp]

Maybe it's just my wishful thinking, but it seems to me that NNs are turning CC from a battle of programmers into a battle of mathematicians

I'd say 80% of NN papers don't have any advanced math in them. It's a bit of a problem for the field - there is a lot of experimentation but less theory about why this works and this doesn't. So it's still possible to make a lot of progress without being a mathematician. A bigger problem is the lack of access to a lot of computational power.

Yes, it's a lot about ad hoc experimentation in a theory vacuum that would horrify a theoretical computer scientist. That is as it's been for many decades, the difference now being that some people have access to huge computer resources they could only dream of 30 years ago.

There are so many papers overall that the 20% with the math are quite enough. I simply have no time to read the rest

I'm pretty sure mmt just made up the exact number 80. Maybe it's really 99, or 99.9.

[matejst]

(...)
I worry about the future of Computer Chess. I see a timeline where a dozen engines use something very similar to Stockfish's methods. They all shoot up in elo. New developers, people working on their own innovations, are disheartened. They ask themselves why they toil away on their new ideas and tweaks, when one can just embrace the NNUE and be on an equal playing field with the top tiers of engines. So they leave. I leave. Alayan leaves. Many others leave. The result is that there are two engines left; Stockfish, and Leela. I don't find that interesting. Maybe others do I suppose.
(...)
I could be all wrong. I could be out of touch. But if I'm not, then the future of computer chess, the future of unique and diverse engines, depends upon all of us, as individuals, to encourage and promote new ideas while discouraging those who take from Stockfish without trying their hand at the problem. I'm already concerned when I see engines with Stockfish nets being placed onto rating lists.

Andy,

While I do not agree with your -- how to put it? I could erroneously choose the wrong word -- bellicosity in arguing, I not only support you here in your line of thoughts, but I also support and admire what you have already done (and, and it is very important, shared) in the world of computer chess. I think you have chosen the right approach: exchange ideas and share resources, then search for one's own solutions. In my experience, it results with not only original, but good results -- and here I do not think about "computer chess", but simply about "chess". I still use Winter and RubiChess for analyzing positions, and, in certain positions, I even prefer Benjamin, Gandalf or Zarkov.

Unfortunately, the last part: "search for one's own solutions" is something just a few try. It is much easier to loot the ideas of good authors, creators. We all think that everybody reads Stockfish's code before writing his own engine, but I have no doubt lots of ideas from Ethereal, Winter, Rubi, and many others open-source engines have been incorporated, in a way or another, in Stockfish, and then further. Recently, Jonathan Rosenthal tried to use NN's as part of the evaluation. One year after, we got NNUE. Although it seemed that the idea was borrowed from shogi, it was already implemented in chess engines, and better still: several nets, for different parts of the eval, were already implemented. I am sure it will be the future of Stockfish,

It was clear to me from the beginning that projects like Stockfish lead to the end of computerchess. Just like the film, there can be only one. The world is turned on his head: anybody writing an engine acknowledges the Stockfish authors, while it should be the opposite. The fact that SF is open-source is the best way to protect it: new developers have to try other ways -- at any level, code or ideas. And further, their enhancements will be appropriated by the SF project, a project that will slowly die when the Andrew Grants, Jonathan Rosenthal, Andreas Matthies, Daniel Jose Queralto retire. I am sure we have it both ways: SF is a repositorium of all the good ideas found in original engines, and then it goes around. That's the reason I think it is OK to borrow from SF, the moment one has one original idea.

It was a long and convoluted post, but I hope I remained clear, at least in small parts.

[syzygy]

Remember, remember, the 5th of December...

Remember, remember, the 5th of December,
the day the earth stood still,
with its AlphaBeta mill,
evaluation terms are no longer required,
neural networks are from now on desired,
handcrafted code ruled once the whole world,
automatically tuned weights from now on behold,
there is no human knowledge in need,
for the neural network to succeed,
where does this journey end - some people ask,
nobody really knows - but that's not the task,
the sky seems the limit,
so let us join the new spirit,
of this glorious journey,
in this new kind of tourney.

--
Srdja

http://talkchess.com/forum3/viewtopic.p ... 3&p=782095



--
Srdja

Nice!

[syzygy]

SUMMARY:
I believe you should continue your work as if Leela on the one hand and NNUE on the other never existed.
IGNORE modern fancy trends because they are just modern fancy trends.

This is the advice that Kodak got when the digital camera arrived. And Nokia and RIM when touchscreens started to take off.

[dkappe]

Why would chess programmers care if their NNUE implementation drives Andrew to retirement?

I'll argue with you another time about your view of Ethereal. Not today, however. Sorry

Wait. I thought you said Daniel felt your constant attacks on him and Scorpio were all in good fun. Maybe he didn’t get the memo.

[JohnWoe]

Once these 3600 Elo over-night geniuses are discovered and hired by Elon Musk. We can finally start populating universum. Instead of playing some 8x8 board game.
Passing Van Allen Radiation belts, travelling astronomical distances to the nearest stars, surviving on hostile planets without oxygen/water/food/etc are soon to be solved.