New engine announcement: Blunder 1.0 (And some questions concerning improvement)

[algerbrex]

I'm happy to announce after quite a bit of hard work, and much help from people here (both directly and indirectly) and in many different places, I have the first working version of my chess engine Blunder (you can probably guess where the name comes from). Currently, the engine is still very basic, and only implements part of the UCI protocol, but definitely enough of the UCI protocol to play games and provide the GUI with some basic statistics (nps, time spent, score, etc.)

Here is a link to the code I've uploaded on Github so I can get feedback, particularly on the search and evaluation phases, although of course advice is welcome applying to anywhere. The repo does include a working binary ready to respond to the UCI protocol, but it's for Linux machines, so apologies in advance if this is an issue.

Currently, my engine implements the following features (hopefully correctly):

• Negamax
• Alpha-beta pruning
• A transposition table
• Iterative deepening
• Move ordering
• Quiescence search
• A very basic opening book, created myself using the Polyglot tool and a couple thousand grandmaster games.

Feedback concerning these areas would also be greatly appreciated.

Now, here are my questions, which have to do with the next steps I'll be taking:

(1) Currently my engine can usually pretty comfortable search to a depth of 8, not including quiescence search, and a depth of around 11-12 including quiescence search. What should my next steps be to be able to search to much higher depths? Ideally, I'd like to be getting into the 20s at least, but I have no clue how to do that currently, and if I tried that with my current code, we would be here till the end of time. What optimizations do I need to look into using to get from a depth of 8 to a depth of 20+? Making such a jump right now seems nigh impossible to me.

(2) Although I'm not sure what the ELO of my engine is, I estimate it's around ~1400, so not really strong at all. I've gotten this estimate from downloading engines listed around 1600 and having them play against Blunder, and in all of the games I ran, Blunder lost. It wasn't necessarily a massacre I'd say, but Blunder was definitely at a disadvantage during the games. However, from testing against bots online at lichess, bots apparently ranked around ~1400, Blunder repeatedly beats them. So I'm honestly not sure. If anyone could point to any solid engines around this rating, I could use those to try to pinpoint the ELO. I also imagine that I could gain quite a bit of ELO by figuring out how to search deeper, so I suppose these two questions are connected.

(3) In addition to the search phase, are any tips you'd recommend for my evaluation phase? My philosophy when designing Blunder's evaluation was basically to keep evaluation light and use that fact to search very deeply. But of course, this leads back to question (1)

I would also like to add that before I turned to algorithmic speed improvements, per the advice of one of the folks here, I started from scratch with just Negamax and alpha-beta pruning, turned on my profiler, and spent a day or two optimizing the code. So that step has been done, hopefully well enough.

Again, any feedback on the code is welcome!

[Madeleine Birchfield]

(3) In addition to the search phase, are any tips you'd recommend for my evaluation phase? My philosophy when designing Blunder's evaluation was basically to keep evaluation light and use that fact to search very deeply. But of course, this leads back to question (1)

For a strong and light evaluation function, you might want to take a look at PeSTO:

https://www.chessprogramming.org/PeSTO% ... n_Function

Alternatively, if you are willing to have a larger evaluation function, then a small neural network might be good:

https://www.chessprogramming.org/Neural_Networks

[mvanthoor]

...

Congratulations with having your own chess engine

At first, I wouldn't worry too much with regard to search depth. It seems your engine uses bitboards, and you're now at the development level of where I was with Rustic Alpha 1. MinimalChess 0.3 (by Thomas Jahn / Lithander) is another engine that is at a similar point of development as your engine.

If you have nothing but alpha/beta, MVV-LVA move ordering, and Rustic's PST's, you should be able to play at somewhere between 1500 and 1700 Elo, depending on the speed of your engine. MinimalChess 0.3 plays at 1570 Elo (CCRL Blitz), but MinimalChess is somewhat slow. Rustic Alpha 1 plays at 1675 Elo, but that is quite a fast engine. Your Go-based engine should be somewhere in between; I think it should at least be 1500 Elo, more probably 1600 Elo.

You could try using Alpha 1's piece square tables (if you also coded your PST's from the white POV), and see if that improves your playing strength.

Try and see what your results are against Rustic Alpha 1 and MinimalChess 0.3.

Later you can add:
- A transposition table and TT-move sorting (+150 Elo)
- Killer moves (+30 to +50 Elo)
- PVS (Principle Variation Search) (+80 Elo)

At that point you would be at the level of Rustic Alpha 3, which could get you to at least 1800 Elo. If you then add a tapered evaluation (use MMC 0.4.1 PST's for that to start with), that should get you to at least 2000, maybe close to 2100 (or over, if your engine is fast enough). You still won't see huge search depths in the middle game, but the TT will cause you to play much better in the endgame, and you will see depths up to 15-16 ply sometimes, and sometimes the engine announces mate in 12 or so. That's cool to see.

You don't even need any other evaluation terms besides the PST's, nor any pruning except alpha/beta itself, to reach at least the 2000, maybe even 2100 level. After that you can go and look into Null Move, History Heuristics / LMR, other prunings... and then into more evaluation terms.

My only advice is: make sure, as much as you can, that your engine is bug free and you're getting as much as possible with regard to playing strength for each feature you add. A tiny mistake can cost you an increase of 100 Elo (as it cost me, accidentally saving the wrong move into the TT), without even crashing your engine or making it play weird. A mistake can completely negate the feature you just tried to add, and then it only serves to slow down the engine.

[algerbrex]

(3) In addition to the search phase, are any tips you'd recommend for my evaluation phase? My philosophy when designing Blunder's evaluation was basically to keep evaluation light and use that fact to search very deeply. But of course, this leads back to question (1)

For a strong and light evaluation function, you might want to take a look at PeSTO:

https://www.chessprogramming.org/PeSTO% ... n_Function

Alternatively, if you are willing to have a larger evaluation function, then a small neural network might be good:

https://www.chessprogramming.org/Neural_Networks

PeSTO looks very interesting, so I'll definitely check that out. Just need to wrap my head around Texel tunning before I try anything like that haha.

[algerbrex]

...

Congratulations with having your own chess engine

At first, I wouldn't worry too much with regard to search depth. It seems your engine uses bitboards, and you're now at the development level of where I was with Rustic Alpha 1. MinimalChess 0.3 (by Thomas Jahn / Lithander) is another engine that is at a similar point of development as your engine.

If you have nothing but alpha/beta, MVV-LVA move ordering, and Rustic's PST's, you should be able to play at somewhere between 1500 and 1700 Elo, depending on the speed of your engine. MinimalChess 0.3 plays at 1570 Elo (CCRL Blitz), but MinimalChess is somewhat slow. Rustic Alpha 1 plays at 1675 Elo, but that is quite a fast engine. Your Go-based engine should be somewhere in between; I think it should at least be 1500 Elo, more probably 1600 Elo.

You could try using Alpha 1's piece square tables (if you also coded your PST's from the white POV), and see if that improves your playing strength.

Try and see what your results are against Rustic Alpha 1 and MinimalChess 0.3.

Later you can add:
- A transposition table and TT-move sorting (+150 Elo)
- Killer moves (+30 to +50 Elo)
- PVS (Principle Variation Search) (+80 Elo)

At that point you would be at the level of Rustic Alpha 3, which could get you to at least 1800 Elo. If you then add a tapered evaluation (use MMC 0.4.1 PST's for that to start with), that should get you to at least 2000, maybe close to 2100 (or over, if your engine is fast enough). You still won't see huge search depths in the middle game, but the TT will cause you to play much better in the endgame, and you will see depths up to 15-16 ply sometimes, and sometimes the engine announces mate in 12 or so. That's cool to see.

You don't even need any other evaluation terms besides the PST's, nor any pruning except alpha/beta itself, to reach at least the 2000, maybe even 2100 level. After that you can go and look into Null Move, History Heuristics / LMR, other prunings... and then into more evaluation terms.

My only advice is: make sure, as much as you can, that your engine is bug free and you're getting as much as possible with regard to playing strength for each feature you add. A tiny mistake can cost you an increase of 100 Elo (as it cost me, accidentally saving the wrong move into the TT), without even crashing your engine or making it play weird. A mistake can completely negate the feature you just tried to add, and then it only serves to slow down the engine.

Thank you! And Ok, I'll look at some testing with those engines. I'm a bit worried by what you mentioned though, because although my engine actually has a transposition currently, I still feel it's no higher than 1600, probably not 1800. Or maybe I've just played some surprisingly strong bots Hopefully I don't have a bug somewhere, but if my engine is missing +200 Elo, then I probably do, so I'll look into that!

Another possibility is that my piece square tables are quite poor since I tried to design them myself based on my own (admittedly amateur) chess knowledge. I might run some tests where I borrow another engine's piece square tables and see how mine plays.

[mvanthoor]

PeSTO looks very interesting, so I'll definitely check that out. Just need to wrap my head around Texel tunning before I try anything like that haha.

You can just add PeSTO's (or MinimalChess' or Rustic's current dev version) tapered tables to your engine and never bother with Texel Tuning. (But I would look into it, for creating your own dataset and tuning your own tables; I'm in that process at the moment.) PeSTO's and MMC's tables are good to get started with though, to see what's possible.

Without even adding a TT (but with PV-move ordering) an no other evaluation terms or pruning, you should be able to get to 1800 Elo at least, with a tapered evaluation. MinimalChess got to 1885; Rustic got to at around 1865 even without a tapered evaluation.

Having some elo-ranges as targets what is possible can be helpful. Both MinimalChess and Rustic are stronger than many engines that have the same feature set; or, they can compete with some engines that have -more- features.

An old standby engine is TSCP 1.84, rating 1725. It's a pawn trick master. It creates doubled, tripled and even quadrupled pawns for an opponent, and then it will destroy you with a pawn storm and make 3 queens. That's it's trademark. You should be able to get quite close with the most basic engine and good PST's (not tailored for TSCP!), and then you should be able to defeat it comfortably by only adding a TT.

[algerbrex]

PeSTO looks very interesting, so I'll definitely check that out. Just need to wrap my head around Texel tunning before I try anything like that haha.

You can just add PeSTO's (or MinimalChess' or Rustic's current dev version) tapered tables to your engine and never bother with Texel Tuning. (But I would look into it, for creating your own dataset and tuning your own tables; I'm in that process at the moment.) PeSTO's and MMC's tables are good to get started with though, to see what's possible.

Without even adding a TT (but with PV-move ordering) an no other evaluation terms or pruning, you should be able to get to 1800 Elo at least, with a tapered evaluation. (MinimalChess got to 1885.)

I think I'll just do that then, at least at the start. But eventually like you I'd like to be able to create some custom tuning, so I'll keep that on my radar too.

[lithander]

This is the pure PST based eval you're probably interested in: https://github.com/lithander/MinimalChe ... luation.cs

[algerbrex]

This is the pure PST based eval you're probably interested in: https://github.com/lithander/MinimalChe ... luation.cs

Ah yep, thanks. I'll look at that and probably starting playing around with some stuff with Blunder.

[algerbrex]

...

If you have nothing but alpha/beta, MVV-LVA move ordering, and Rustic's PST's, you should be able to play at somewhere between 1500 and 1700 Elo, depending on the speed of your engine. MinimalChess 0.3 plays at 1570 Elo (CCRL Blitz), but MinimalChess is somewhat slow. Rustic Alpha 1 plays at 1675 Elo, but that is quite a fast engine. Your Go-based engine should be somewhere in between; I think it should at least be 1500 Elo, more probably 1600 Elo.

You could try using Alpha 1's piece square tables (if you also coded your PST's from the white POV), and see if that improves your playing strength.

Try and see what your results are against Rustic Alpha 1 and MinimalChess 0.3.

So I tried my engine against MinimalChess 0.3, and it didn't end up doing too great haha. It was winning for a while but gave away the advantage apparently. Of course, I'll run more games, but still with the features I have (if they're implemented correctly, and that's a big if), I agree with your estimate that I should at least be around ~1800 and be able to beat ~1600 rated engines, but of course my engine is playing more around ~1300.

So an Elo gap of about ~500 makes me think there's a bug somewhere in my code. Which is a little frustrating, but just another setback. But a gap of 500-600 Elo seems like quite a lot, so I may just start from the beginning again with negamax, alpha-beta, and piece square tables, and go from there, testing my engines Elo at each stage. Because even with all of the features I have now, my engine is very, very weak.