Imposter syndrome and writing chess engines

[algerbrex]

Imposter syndrome -> reverse Dunning-Kruger effect?

Yes, that's a very accurate way of putting it I think

wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
I like the Christian saying,"Judge them by their fruits", or alike.

I like that as well good point. I'll keep that in mind.

[Ovyron]

Someone who could create a project, like a chess engine, by themselves with no outside help. Of course, I realize this is unrealistic

It can be done, my first chess engine was done in the Game Maker software completely from Scratch without even knowing what other people had done before or how they did it. Unfortunately, it was based on collisions, and there was a bug where, when moving a King it very rarely and randomly would miss landing on the end square, and would move forever out of the board and beyond! That was the end of it as the program would never do anything else until the king landed on that square, which wasn't going to happen... I never could figure it out.

I guess the hardest part is forgetting what you know, because as soon as someone gives you an idea and you can't forget it, the program can't be 100% yours.

I remember that Bob Hyatt mentioned that he'd instruct his students to create a chess program, and then they'd go and invent their own 100% unique chess engine, and he'd never see anyone creating the same code for it. All original and all their own, wished I was given that task when I studied programming, nowadays it's better to admit we're standing over the shoulders of giants, and enjoy what we can out of this hobby.

[tcusr]

With that said, all of the code in Blunder was written by hand, never copy and pasted, and there are areas that I wholly wrote myself. I've worked hard to debug issues, test the engine pretty consistently, and implement features in a way that works in my unique codebase. Nevertheless, I can't help but feel I haven't actually done anything worth being proud of or calling my own.

Compare yourself with others that modify a few numbers from search.cpp or evaluate.cpp of stockfish and feel entitled to change the program's name, put themselves as authors, and sometimes even close the source and sell the engine. And then they feel proud of calling that their own. And they get tested a lot by others that want to know if their modifications are better than default stockfish.

If people can do so little and feel proud and something as their own, what stops you, if you did so much more?

All very true. I fully admit this is more of a peculiar issue with myself than an actual problem.

I think I've always just had this desire, even when I first started programming, to be a "real programmer." Someone who could create a project, like a chess engine, by themselves with no outside help. Of course, I realize this is unrealistic, especially for someone like myself who has only been programming now for 5-6 years. But my mind acts oddly sometimes I suppose.

about real programmers https://www.ee.ryerson.ca/~elf/hack/realmen.html

[jtwright]

@algerbrex

I actually want to say I've been quite impressed by your work on and documentation of your learning regarding Blunder. I've only been chess programming for about a year, year-and-a-half, but in that time I think I've seen a lot of different chess programmers with different goals and processes, and one thing became really clear:

Engine Elo =/= Chess Programmer knowledge/smartness/goodness

Consider Ed Schroeder, developer of Rebel/ProDeo and absolute veteran of chess programming. No one would dispute his incredible depth of knowledge and thoughtful approach. He's a legend essentially and has been programming chess since long before I was born, but until the release of Rebel 14, ProDeo had lower Elo than several "new engines", but wasn't due to a lack of knowledge. Ed (to my knowledge) had certain design principals for his work that meant he didn't necessarily follow a lot of the more stockfish-like patterns of modern chess engines. I'm sure if he had decided that maximizing Elo was his main goal, he would have a very different engine.

In a similar vein to different goals, there's also different journeys. I think that for many of the 3000+ Elo engines that are only ~1 year old, it's not so much a difference in programmer ability or smarts that separate it from the, say 2600s, but just how they pursue the goal of more Elo.

I think the core of that super-rapid growth (and I include myself and Mantissa here as well) is idea-sharing. In this era, this can be through reading code or discussing ideas in Discord or elsewhere, but basically I think a lot of key ideas are at this point shared and ubiquitous among the top engines. I'm not decrying this practice, I think it makes chess programming stronger as a whole, and also I follow this practice. I can't really put words into other people's mouths, so I'll just explain how things went for me for the specifics:

I love reading code, I love learning things. When I exhausted most of what CPW had to say I started reading the forums and other engine source. I would never directly copy-paste something from another engine, but once you've read an interesting idea you can't unlearn it. I would see something I thought was really fascinating in another engine, try to learn entirely how and why it worked, write ideas down and talk about them with a friend of mine who also was writing an engine, until I felt like I understood the idea well enough to try to implement it myself. Rinse and repeat over time. Basically fast-tracking learning the last couple of decades of chess programming advancements. I don't really consider myself smarter or harder working than any other chess programmer, possibly just better informed on how SF/Ethereal/Koi/etc. work. I imagine the process was similar in many other rapidly rising stars, as similar ideas show up in most of the open source top engines (not throwing shade, a good idea is a good idea and I do this too), but I can't say for sure.

Also, NNUE is a huge Elo boost even with a simple net, so that's another factor. Between NNUE and learning from other engines, Mantissa went from 2600 -> 3100 in the space of a few months. But that's just how my own process happened to shake out. I'm very much standing on the shoulders of giants, as I think a lot of chess programmers are.

There are also other factors I could bring up too like better testing capabilities (e.g. OpenBench/Fishtest scale testing capabilities), but I'll stop here for now.

Anyway, that's a meandering post, but I just want to reassure you that you are most definitely legit. Take the journey at your own pace, discover things the way that feels best to you. Chess programming is a hobby for many of us after all, it should be fun. If you want to maximize Elo quickly, there are ways to do that, but if that's not your goal, or it's secondary to other goals, that doesn't make your work any less legitimate or your skills any less real.

[algerbrex]

@algerbrex

I actually want to say I've been quite impressed by your work on and documentation of your learning regarding Blunder. I've only been chess programming for about a year, year-and-a-half, but in that time I think I've seen a lot of different chess programmers with different goals and processes, and one thing became really clear:

...

Anyway, that's a meandering post, but I just want to reassure you that you are most definitely legit. Take the journey at your own pace, discover things the way that feels best to you. Chess programming is a hobby for many of us after all, it should be fun. If you want to maximize Elo quickly, there are ways to do that, but if that's not your goal, or it's secondary to other goals, that doesn't make your work any less legitimate or your skills any less real.

Hi Jeremy, apologizes for missing this message.

But thanks for the encouragement and insight. I enjoyed reading through your post, so don't be too hard on yourself with the meandering

I completely agree with you. Ed, among many of the other people here, is someone I respect immensely. As you said, he was working on chess engines long, long before I was born. And you noting his approach is a great point. Part of the reason I respect him as much as I do is because of his willingness to do things his own way and stick to his own principles. I've tried to emulate that mindset in my own work on Blunder.

The more I've been involved in chess programming, the more I realize I enjoy it for the community almost as much as the actual programming and chess engine theory. I've been programming for about 7 years now, starting when I was 13. And while I'm by no means a veteran, the chess programming community has probably been the most welcoming and open community I've been a part of. As you said, the willingness to share ideas is and knowledge incredible to me. You'll even have people like the late Don Dailey coming on here and sharing ideas that were being used in Komodo, whose code is of course proprietary.

I love reading code, I love learning things. When I exhausted most of what CPW had to say I started reading the forums and other engine source. I would never directly copy-paste something from another engine, but once you've read an interesting idea you can't unlearn it. I would see something I thought was really fascinating in another engine, try to learn entirely how and why it worked, write ideas down and talk about them with a friend of mine who also was writing an engine, until I felt like I understood the idea well enough to try to implement it myself. Rinse and repeat over time. Basically fast-tracking learning the last couple of decades of chess programming advancements. I don't really consider myself smarter or harder working than any other chess programmer, possibly just better informed on how SF/Ethereal/Koi/etc. work. I imagine the process was similar in many other rapidly rising stars, as similar ideas show up in most of the open source top engines (not throwing shade, a good idea is a good idea and I do this too), but I can't say for sure.

Our approaches sound very similar I do what you describe quite a lot as well, often seeing an interesting idea in another engine, not even one that's necessarily at the top of the CCRL, and try to look into understanding the idea well enough to use it myself. I also look at other engines' source code as a sanity check for trying out a feature to make sure I understand the idea. And finally, I test the hell out of it myself, on my own time, to ensure it's a good idea and implementation for me. So good to know I'm not alone in this, and I would completely agree that this doesn't trivialize either of our work.

And that's part of the reason why I try to write clean, well-documented code for Blunder. I want other people to do what I do. I want people to read Blunder's source code, see an interesting idea, and try it for themselves. I love to see when people use Blunder as inspiration for their own engines. Just a couple of days ago I came across and new up and coming engine that actually credited Blunder as one of the main resources it used, which made me incredibly happy.

Engine Elo =/= Chess Programmer knowledge/smartness/goodness

Couldn't agree more. The more I work on and read chess engine, the more I realize it's a science (not exact though!) and an art.

As I work on the release of Blunder 8.0.0 (which is coming along quite well) my primary goal is still Elo gain. But this goal is now balanced with other goals, like originality, and understanding. Part of the reason I've slowed down between Blunder releases is that, like you, I want to understand every big idea that goes into Blunder. I want to take the time to understand why an idea works, what are it's alternatives, what are it's downsides and benefits, etc. Because at the end of the day I want have a strong chess engine, and know how it works, inside out. That to me is much more satisfying then cloning Stockfish and making a few tweaks here and there. And there have also been times when I've lost Elo in favor or originality or aesthetics. I remember I once tuned some PSQT values that gave Blunder +35 Elo in testing, but made the play style so ugly and atrocious, I scrapped them, and end up settling for some values that only gave +10-15 Elo.

So I appreciate your reminder about the aspects of chess programming that I think are often easy to forgot. We haven't had a chance to talk much here, but I'd like to commend you as well for your work on Mantissa. I remember coming across your engine last year and being impressed with the progress you made in such a short amount of time

[lithander]

As I work on the release of Blunder 8.0.0 (which is coming along quite well) my primary goal is still Elo gain. But this goal is now balanced with other goals, like originality, and understanding. Part of the reason I've slowed down between Blunder releases is that, like you, I want to understand every big idea that goes into Blunder. I want to take the time to understand why an idea works, what are it's alternatives, what are it's downsides and benefits, etc. Because at the end of the day I want have a strong chess engine, and know how it works, inside out.

I have similar goals. But maybe my reasons to have these goals may be a little different. At least I've never felt conflicted about my engine being my own brain-child. I don't feel like I have to prove anything to anyone.

Chess programming is surprisingly fun and addictive. But why? This community is cool! I missed phpBB boards, haha. And taking part in tournaments is cool and climbing the Elo ladder is cool. But that's not my main source of motivation. My motivation is that programming a chess engine provides me with an endless source of interesting puzzles that I'm genuinely curious about. I could just look at other engine's source code and bootstrap my progression but than I'd deny myself the joy of working on these fine puzzles.

Of course I'm not working in isolation and all the high level ideas I'm working on I've picked up from the community. I'm not even a good chess player. But I try to stop at the point where I know that people are having a pawn structure term in their evaluation. Or that they use a hash table to avoid recomputing these pawn structure terms. That's usually enough knowledge to get started. Now the next question is not how these features are best implemented (which would mean research the wiki/forum/engines) but the next question is what process should I follow to arrive at a good implementation myself. I'll have to walk the path to know where it will lead and how much Elo this will mean. I love this sense of discovery! (Rediscovery of course, but it doesn't make the puzzle less interesting to know that others can solve it too)

In the end if someone (or yourself) is asking you questions about your code you won't have to try to explain how the code is working but you will think about your process and how that lead you to write the code you ended up committing. Implementing a tuner based on gradient descent was a big step forward for me in that regard. Not because my new tables were so much better than the old ones. But because, once you really understand that part, you have a powerful new development tool: Why limit your tuner to PSQTs? You can now come up with any feature and see how that feature correlates with winning the game. Are doubled pawns important for a pawn structure and in what way? I don't need to consult a wiki for that or look at source code. I can write code to pull the knowledge out of a bunch of EPDs. Look for patterns and convert them into new code. Get rid of the noise and make it pure. Then run a test and see the Elo go up. How great is that? (most people will just shrug here, but those people are probably not programming since they are 13^^)

I'm sure you can do that too if you're willing to walk the extra mile and think about the process first and not right away about the implementation. Sometimes your implementation is not going to be the best one, but it's your personal solution to a problem and it's a solution none-the-less, making your engine better. What else could you want?

In an actual paid position often you don't have the time - you need to be efficient and take all the shortcuts. But in a sparetime project no one's pressuring you. You can take all the extra time and in the end it will only mean you had more fun and the feeling of accomplishment is bigger.

[algerbrex]

I'm sure you can do that too if you're willing to walk the extra mile and think about the process first and not right away about the implementation. Sometimes your implementation is not going to be the best one, but it's your personal solution to a problem and it's a solution none-the-less, making your engine better. What else could you want?

Thanks, I'm sure I would be able to as well. And the more I work on Blunder, the more I realize I should start trying this more instead. Because of the peculiarities of a particular chess engine, it's often not even possible to just use someone else's approach anyway. For example, I remember with LMR in Blunder I couldn't get it working and trying out the pseudo-code others provided didn't help either. It wasn't until I sat down and really took the time to think through the theory behind LMR, and then actually write out my own approach, did I get an Elo gain. So the more I work, the more I'm starting to agree with your sentiment why not just start from the basic idea and figure things out for me from the start?

In fact, this was actually going to be my approach to adding a neural network to Blunder. Of course, I could start by looking at what other people have written and implemented, and I could read everything on the wiki, and jump straight to using the best, most modern designs for my architecture. But I've already decided I'm not.

Blunder 8.0.0 most likely won't have a neural network, but I'm hoping 9.0.0 will. And for that first attempt, I'm going to be trying to work from the ground up. Figuring out the architecture, the inputs, outputs, layers, training positions, computing, etc. I'm currently going through and teaching myself the basics of machine learning via this online book: http://neuralnetworksanddeeplearning.com/about.html. As a very rough first attempt, to get an idea of the problem, I'm probably going to prototype something in Python and then port the weights and biases over into Go.

I know the first attempt will likely be terrible, and I'm fully aware of the need to train on millions of positions to get very good results. And I know at some point I'll move up to learning about modern nets and their integration into chess engines. But like you said this is all part of the process of discovery I want this first attempt to be as simple and straightforward as possible (e.g. 786 (12 x 64) inputs, not many layers, output a centipawn score, etc.) You could even call it, minimal The goal here will be to learn as much as possible, not have Blunder's strength jump from 2600 to 3000+ overnight.

So your approach is a very valuable one I think, and one that I've taken to heart more and blended with my original approach.

In an actual paid position often you don't have the time - you need to be efficient and take all the shortcuts. But in a sparetime project no one's pressuring you. You can take all the extra time and in the end it will only mean you had more fun and the feeling of accomplishment is bigger.

Also good for me to keep in mind. I'd be lying if I denied a little friendly competition between others on here was originally my motivation for working on Blunder round the clock. But now I enjoy working on Blunder much more because I'm no longer putting so much pressure on myself to make progress. Now I'm ok with taking months between version releases because I want to work on getting the best quality possible and learning and improving my programming skills. And when I am out of uni and working at a job, I'll have the skills I need to work under a time crunch.

[mvanthoor]

This is a bit of an odd topic for me to bring up, admittedly. But I'm curious what others' views are.

Most have heard of imposter syndrome, particularly when it comes to programming knowledge and abilities. And while I experience this from time to time, I mostly experience this when working on Blunder.

Often I'll implement a new feature and get it working well and feel some excitement, but this excitement is usually dampened by feeling like I don't really understand what I'm doing, and if every bit of the source code for Blunder were deleted from the face of the earth, I'd never be able to re-write it properly.

Now, I fully acknowledge and will continue to acknowledge for as long as I keep working on Blunder that I didn't get to where I am by myself, as nice as that might be to say. I've had inspiration and help from countless people here, and across different websites, forms, severs, and the like. And many people's codebases and engines have given me brilliant ideas.

With that said, all of the code in Blunder was written by hand, never copy and pasted, and there are areas that I wholly wrote myself. I've worked hard to debug issues, test the engine pretty consistently, and implement features in a way that works in my unique codebase. Nevertheless, I can't help but feel I haven't actually done anything worth being proud of or calling my own.

I'm curious how others generally feel about this topic. Just sharing my two cents.

Interesting topic. I thought about this myself a lot.

This is the key part:

Often I'll implement a new feature and get it working well and feel some excitement, but this excitement is usually dampened by feeling like I don't really understand what I'm doing, and if every bit of the source code for Blunder were deleted from the face of the earth, I'd never be able to re-write it properly.

This (and a general lack of time in the last year) is the main reason why Rustic's development moves so incredibly slowly, and why I try to maintain a description of each feature on a website / in a book. I refuse to just look at feature X, see 10 lines of code, rewrite them to adapt them to my engine's setup, score 100 ELO points, and go "Cool! NEXT!". I want to understand EXACTLY what happens, how, and why this makes the engine stronger. If I understand it, I should be able to write my own explanation for my website / book.

When my website is "finished" (i.e., it contains everything that is in Rustic Alpha 3), then I SHOULD be able to rewrite Rustic in any programming language I ever want to without even having to use any other resources.

If there are people who say "I wrote this chess engine without ever looking at any resources, websites, articles or video's" then I just don't believe them. Classical alpha/beta chess engines have been in development for about 70 years now. Nobody is able to come up with everything on their own. Yes; you will probably be able to write a trivial chess engine on an 8x8 array-based board that uses min-max or alpha-beta with just your computer science university courses to draw upon.

I did it in the 90's when I was 15 years old, just using two library books; one on Borland Pascal, and one on board game algorithms. The engine worked. It was also horribly slow and somewhat buggy. From that point onward I didn't know where to go, so I left it at that and promised that "someday" I'd write a "real" chess engine. (Well; "someday" became about 25 years.)

After an 8x8 array-based board and min-max or alpha-beta, the knowledge becomes so specialized that basically no one in university teaches it (not as far as I know), so you HAVE to look at books, articles, video's of people who came before, read blog posts, and maybe even read source code to be able to progress. You just cannot discover 70 years of computer science / chess engine writing on your own.

I watched a few documentaries on Bruce Lee in my time. He didn't become so good at martial arts by just training by himself. He got his knowledge from MANY different sources and understood it, making it his own.

That's the most important part: it doesn't matter where you get your knowledge from, but you HAVE to understand what you are doing. In that case you're learning and getting better; if you just rewrite or even copy-paste code without understanding, THEN you're possibly an imposter.... worse, if something DOESN'T work, you won't even be capable to debug the problem or even know that there is something wrong.

[mvanthoor]

All very true. I fully admit this is more of a peculiar issue with myself than an actual problem.

I think I've always just had this desire, even when I first started programming, to be a "real programmer." Someone who could create a project, like a chess engine, by themselves with no outside help. Of course, I realize this is unrealistic...

As I said in my earlier post: you can't. Even if you do it "without help" you'll have to learn the basics on programming and algorithms somewhere. You can't invent 70 years of computer science on my own.

To mention Bruce Lee again: he created Jeet Kune Do. If you REALLY look at it for what it is, then you'd see that is "basically" the core of Kung Fu / Wing Chun, blended and extended with techniques from many other martial arts. Choi Yong-Sool created Hapkido. If you look closely at Hapkido, you'll see that it has some techniques that look a lot like the ones in Aikido and Jiu-Jitsu. That's not strange, because Hapkido and Aikido both have the same ancestor, Daitō-ryū Aiki-jūjutsu, which would later transform into the Jiu-Jitsu we know today. Choi was a Korean guy, so when he developed Hapkido, it was almost inevitable that it got to contain Korean techniques as well. (In this case, Tae-Kyon; the ancestor of Taekwondo... so it's not strange that Hapkido basically contains ALL of Taekwondo's kicks except for the ultra-high-flying ones used mostly for showmanship.)

Everything is linked, in one massive web. Every martial art that is developed uses parts of other martial arts. Everybody h as to start training somewhere, and no-one can just "invent" a new system from scratch without ever looking at anything else; to do so, you'd have to, eventually, invent all of martial arts.

That is a ridiculous proposition.

So it is with chess engines. If you write one on your own without ever looking at ANYTHING else and you succeed, you'll end up in roughly 1957 or so, and be stuck there, or you would have to invent a massive part of computer science all on your own. I'll talk to you again in 2740, and see how far you got...

especially for someone like myself who has only been programming now for 5-6 years. But my mind acts oddly sometimes I suppose.

I think that is because people who do something great are often portrayed as if they did everything by themselves. That warps the perception.

It's just not true. Many people who did things that were truly great had to try often, fail many times, study other people's work for a long time, and even THEN they probably needed help now and then, because no-one can know everything.

[mvanthoor]

When my website is "finished" (i.e., it contains everything that is in Rustic Alpha 3), then I SHOULD be able to rewrite Rustic in any programming language I ever want to without even having to use any other resources.

And THAT is actually my (Rustic's) goal. To create, in the end, a comprehensive resource that can be used to write a chess engine from scratch, in any programming language. And the main user of that resource could be me, 20 years down the line, writing in a programming language that hasn't been invented yet.