Moderators: hgm, chrisw, Rebel
Need to compare what with what? The source code? The move output? What do you even mean? Obviously, SF16 is using the network, as seen in my posting.DmitriyFrosty wrote: ↑Fri Jan 26, 2024 1:24 pmYou need compare it to Stockfish with pure nnue evaluation (when he uses one network)
I very much agree with all this. I would have written the same if I could find the appropriate words.mvanthoor wrote: ↑Fri Jan 26, 2024 1:14 pmThis engine posted by Gabor has the same main sources:
But the link to the engine programming discord doesn't work.Gabor Szots wrote: ↑Thu Jan 25, 2024 4:05 pm New engine Molybdenum
rn5f107s2, Germany
https://github.com/rn5f107s2/Molybdenum ... s/tag/v2.0
Even so, I can't believe that there are people who go: "Let's write a chess engine!" Then: "Let's hang around in the Discord and Engine Programming Channels for a few weeks!" Then, 200 commits and a fart later, another 3500 engine drops.
That just feels wrong to me. Either I'm exceedingly stupid because I can't do that, or many others are just (re)writing code without exactly understanding what it does and why. My challenge would then be: if I gave you a text editor, a compiler of your language of choice and your own knowledge and notes (so NOT the source code of your previous engine), could you do it _again_? If not, then... well... you either didn't understand, or you didn't document and forgot.
There are SO many engines that just list a bunch of features and pop out new versions every other day without even listing a changelog. Those are useless to learn from, even if open source. The same goes for "I've been writing a chess engine for the last few weeks and *pop!* here's your new 3500 Elo engine! Enjoy, bye!" And the creator was never seen again. (And may not even have been seen before.)
I ignore all of them. The only new open source engines I follow are the ones for which I can see the beginnings, that document their progress and what they are doing, and why.
Yes completely different, Lets give Stockfish the White pieces, later on Obsidian 10 the White piecesRas wrote: ↑Fri Jan 26, 2024 1:13 pm Are we talking about Obsidian 10 from https://github.com/gab8192/Obsidian ?
As example, I compared one of the most important parts (search.cpp) with SF16. This is pretty different and not a copy. There are ideas from SF, but that's common practice and also acknowledged. Also the engine file sizes are different because Obsidian uses a small net compared to SF16, and that alone makes it very unlikely that they come up with the exakt same PV and eval. I suspect that Chessqueen's test is garbage, probably running two times the same engine, not SF16 and Obsidian 10.
I built both SF16 and Obsidian 10 from source (Linux) and gave the test position after move 14 with depth 20. Both SF16 and Obsidian at 256MB hash, single thread. The output is completely different, they don't even agree on the best move within the given depth. Despite much higher NPS, probably due to the smaller net, Obsidian takes much longer to reach depth 20.
It's not only with regard to ideas of one's own. It's also about understanding, documentation, time, and testing.Gabor Szots wrote: ↑Fri Jan 26, 2024 1:57 pm It's been my opinion for a long time now that if a very strong engine pops up, developed for some time then abandoned, then it is a clone (sorry, a derivative) with nothing original in it. The author simply abandons it because he has implemented everything there was for the taking and has no ideas of his own.
There is quite literally not a single technique in developing an alpha/beta engine that takes weeks to understand.mvanthoor wrote: ↑Fri Jan 26, 2024 2:43 pm - Understanding: sometimes it takes me WEEKS to understand a topic/concept in enough detail to be able to implement it myself, without looking at other people's code, using my own programming knowledge. Sometimes the implementation changes 2-3 times because I either didn't understand something 100% properly, or I found some nifty feature in Rust that makes the implementation better/faster/whatever.
You can, the most time consuming parts of developing a 3500-level chess engine is testing and data generation, by simply skipping the data generation (i.e. by using Lc0 data, as Obsidian does, which is also, annoyingly, much better than self-generated data anyway) you massively cut down on the time and compute required - and what really is imposing self-generated data other than an arbitrary restriction for the sake of "originality", with no real effect on your "understanding".
Code: Select all
A reasonable search feature progression (starting with vanilla TT (sorting TT move first), PVS, QS and aspiration windows which are all pretty fundamental) imo is: NMP, LMR (log formula is most principled ~ there are a number of adjustments you can experiment with), static NMP (aka RFP),
butterfly history heuristic, LMP, futility pruning, CMH+FMH, QS SEE pruning, PVS SEE pruning (captures and quiets), QS delta pruning, history pruning, capture history heuristic, singular extensions, multicut (using singular search result).
(with a healthy amount of parameter tweaking after each addition)
Idk if I'm missing anything major. Those search heuristics constitute the vast majority of the Elo you'll find in any top engine, though the details of the implementation are very important.
This is silly. The second you add any other features on top of your "perfected" current feature set, its no longer perfect. There's an abundance of evidence for this, e.g: https://chess.swehosting.se/test/4440/ and https://chess.swehosting.se/test/4437/ in my own engine.
Have you tried explaining something as "simple" as alpha/beta to someone on the basis of a code snippet and WHY the code snippet actually works? Try it. You may find it harder than you think. Many techniques in the chess engine world took YEARS to develop and understand back in the 70's. People have written Ph.D.'s on those techniques. I aim to understand techniques at, or close to that level so I can then explain them in simpler terms, in such a way that they can be even understood by chess players and non-programmers if they so desire. Understanding a technique at the 'just enough to implement' is not good enough.
Of course you can just take shortcuts. You either write a chess engine from scratch and understand every single line you wrote and why it works and you generate your own data and run your own tests, or you just (re-)implement the techniques and be done with it. And as far as I can see, there are lots of people doing the latter, because there's more than one 3400-3500+ engine that's suddenly popped up and has little or no development afterwards.You can, the most time consuming parts of developing a 3500-level chess engine is testing and data generation, by simply skipping the data generation (i.e. by using Lc0 data, as Obsidian does, which is also, annoyingly, much better than self-generated data anyway) you massively cut down on the time and compute required - and what really is imposing self-generated data other than an arbitrary restriction for the sake of "originality", with no real effect on your "understanding".
I found out about that roadmap/progression for myself, because I understand the techniques and what they do. And yes, you can just copy that list, find the relevant CPW page and if needed some code some engine, so you can add many techniques to your engine in a very short time. Where's the fun in that? In that case you're just doing busy-work, IMHO. It doesn't guarantee that you understand them well enough to be able to explain them to others, and THAT is what I am aiming for.As for the search, if you join the various engine-related discord and ask for help, you'll most likely be pointed to the "connorpasta":
With perfect, I mean bug-free. As I said, I have assisted more than one engine to boost its strength by +200 and once even +500 points by eliminating bugs in existing features.This is silly. The second you add any other features on top of your "perfected" current feature set, its no longer perfect. There's an abundance of evidence for this, e.g: https://chess.swehosting.se/test/4440/ and https://chess.swehosting.se/test/4437/ in my own engine.
I think I agree with this sentiment despite disagreeing with much of what you've said (i.e, many top engine devs understand alpha-beta pruning/PVS, MTD(f), and the many associated search heuristics at a very deep level as a deep, intuitive understanding is really what's required to push the envelope). I also take issue with the notion that all search heuristics were developed by researchers as this is just patently false. Many of the heuristics you'll find in top engines originated a random developer following their intuition and trying a random idea that popped into their head - the process is much more organic than you seem to suggest.