MinimalChess 0.3 released

[lithander]

I tried to compile the engine (v0.2) on Linux (with MonoDevelop), without success. In case you are interested, here is the output of the compiler:

Code: Select all

[roland@localhost MinimalChessEngine]$ TERM=xterm
[roland@localhost MinimalChessEngine]$ mcs Program.cs
Program.cs(82,50): error CS1644: Feature `declaration expression' cannot be used because it is not part of the C# 6.0 language specification
[...]
Compilation failed: 10 error(s), 0 warnings
[roland@localhost MinimalChessEngine]$ 

It's a bit disappointing when you cannot compile yourself an open source engine, especially when it is a didactic engine, whose interest is in the source code.

Hmm.. I did use a few language features that were not yet included in C# 6.0. The current version is 9.0 and I think I used a few 7.0 features. But since Microsoft open-sourced .Net the newest language features are also available on Linux natively making Mono somewhat obsolete. Or so I thought. Have you tried the compilation instructions for Linux that I included in the github Readme? If not, why? (I'm genuinely curious as I'm not a Linux user myself I may have misjudged the situation)

[Roland Chastain]

Have you tried the compilation instructions for Linux that I included in the github Readme? If not, why? (I'm genuinely curious as I'm not a Linux user myself I may have misjudged the situation)

Yes, I tried but it didn't work. The instructions doesn't seem to be made for my Linux distribution (Mageia). I also tried the instructions given here, with no more success. (The OS complains about a missing library that I cannot find. Problem described here.)

I have another PC with another Linux distribution (Manjaro). I will try tomorrow and see if I am luckier with that one.

[lithander]

Thanks for including MMC in your gauntlets! (the poor thing^^)

This is the current list, with the games of MinimalChess 0.3 included:

Code: Select all

1 Clueless 1.4			1888   63   59   100   69%  1734    9%
2 Wukong JS 1.4			1843   59   57   100   64%  1734   15%
3 Deepov 0.4			1831   58   56   100   63%  1734   15%
4 CDrill Build 4		1805   59   58   100   59%  1734   11%
5 Pigeon 1.5.1			1795   54   53   100   59%  1734   34%
6 Rustic Alpha 2 rc5		1792   23   23   600   57%  1741   20%
7 TSCP 1.81			1743   58   58   100   51%  1734   10%
8 Celestial 1.0			1733   55   55   100   50%  1734   22%
9 FracTal 1.0			1714   52   52   100   47%  1734   39%
10 Shallow Blue 2.0		1713   56   57   100   47%  1734   16%
11 Rustic Alpha 1		1677   23   23   600   42%  1741   19%
12 Mizar 3			1662   57   58   100   41%  1734   13%
13 MinimalChess 0.3		1582   40   41   200   29%  1734   25%

If I understand this correctly the engines never played each other but the ranking is concluded indirectly by them all playing only the two Rustic versions?

This is of course a great way to get a more realistic idea of Rustics strength but the rating might not represent the other engines as accurately. For example MMC 0.3 is doing much better against FracTal 1.0 than the list would suggest:

Code: Select all

Score of MinimalChess 0.3 vs FracTalv1.0: 283 - 313 - 404  [0.485] 1000
Elo difference: -10.4 +/- 16.6, LOS: 11.0 %, DrawRatio: 40.4 %

Note the high draw rate. FractTalv1.0 allows MMC 0.3 to convert many of the games it would otherwise lose into draws by 3-fold repetition. 353 of 1000 games ended by 3-fold repetition!

Did you run a match against Rustic Alpha 1 using Rustic's PST's?

Yes I tried the different PSTs against Rustic.

Code: Select all

Score of MinimalChess 0.3 Carnivor vs Rustic: 217 - 576 - 207  [0.321] 1000
Elo difference: -130.5 +/- 20.1, LOS: 0.0 %, DrawRatio: 20.7 %

Score of MinimalChess 0.3 Simple vs Rustic: 231 - 550 - 219  [0.341] 1000
Elo difference: -114.8 +/- 19.8, LOS: 0.0 %, DrawRatio: 21.9 %

Score of MinimalChess 0.3 Rustic vs Rustic: 273 - 505 - 222  [0.384] 1000
Elo difference: -82.1 +/- 19.4, LOS: 0.0 %, DrawRatio: 22.2 %

Score of MinimalChess 0.3 Sunfish vs Rustic: 321 - 488 - 191  [0.416] 1000
Elo difference: -58.6 +/- 19.6, LOS: 0.0 %, DrawRatio: 19.1 %

Texel-tuned Sunfish does best. But keep in mind that it wasn't tuned for play in MMC but Sunfish. It's likely that if you tune PSTs yourself for your own engine the results are even better!

Also, for the Rustic PSTs I had to write my own King table because Rustics original data uses different tables for mid&endgame. My engine doesn't support that atm. And you know how little I know about chess so that table might be flawed.

Congratulations. If you wanted to write a minimal chess engine, you did it.

Thanks! Yes... I actually went farther than what I had in mind for my little "Christmas Break" project. But it's addicting stuff. Now that I have this platform that I understand very well and control there are so many little things I still want to try. For example the next obvious step regarding the evaluation would be to support the PeSTO evaluation.

- Include a few PST sets
- Include a PST-set that has all zero's, effectively disabling it
- Make a UCI-option to disable QSearch

I don't know if disabling QSearch should be an option. If you want to weaken it beyond what material-only does you can limit the search depth or node count using the UCI time controls already.

That seems a perfect opportunity to call it MinimalChess 1.0, and call it quits on this engine

Assuming you want to write a bitboard engine and then go all-out with adding features. I'd be willing to help with this (with tips and hints, and some testing/profiling, not programming), as I went through that process a year before.

Thanks for the offer! I'll take you up on it for sure! Expect a flood of PMs

But I'm not feeling quite ready to start on a more ambitious new engine. I'm not even sure if Bitboards or Mailbox ala HGM are the way to go or what combination of the two would be best.
In MinimalChess when I explore a Million nodes that means I have literally created a Million instances of the Board-class, via copy-constructor and played a move. In a rewrite I would just have one "fat" Board instance that allows me to Undo moves instead. This allows for lots of little lists and masks and a hashvalue that will be updated incrementally. It's hard to design all this without knowing more about what information I will actually need for my move ordering and the evaluation function. And once you have one kind of information precomputed but not the other kind there's a strong bias to only use the information that is cheaply available. So these early design decisions really matter in the long run.

In other words, before I start from scratch I need a clearer vision of the whole thing. If I do another chess engine I don't want to find out I'm stuck just a few ELO above MMC. MMC is slow by design, my next engine would have no such excuses and I'd have to take the blame for measly performance results myself. My ego is a frail thing, I need to do some studying before I put it at risk like that!

[mvanthoor]

If I understand this correctly the engines never played each other but the ranking is concluded indirectly by them all playing only the two Rustic versions?

Yes.

This is of course a great way to get a more realistic idea of Rustics strength but the rating might not represent the other engines as accurately. For example MMC 0.3 is doing much better against FracTal 1.0 than the list would suggest:

Yes. The list would be somewhat more accurate if every engine played every other engine for 200 games per match, to equalize out quirks like the one you point out.

Because the engine pool is different, you also can't compare this list 1:1 to CCRL. It's just intended to measure the playing strength difference of A1 vs field X, and A2 vs field X, which is +104 Elo +/- 23 for A2. Therefore I can assume that A2 is going to be 1758 - 1804 on CCRL.

Yes I tried the different PSTs against Rustic.

Texel-tuned Sunfish does best. But keep in mind that it wasn't tuned for play in MMC but Sunfish. It's likely that if you tune PSTs yourself for your own engine the results are even better!

I'm impressed with myself with regard to the performance of those hand-written PST's. It seems being a somewhat decent chess playier myself has its advantages at some points in the development process

Alpha 3 will be about move ordering. Then I'll research tuning, and I hope to be able to gain another 50-100 Elo (which seems to be an expected gain when looking into other engines) after tuning. That version will be Rustic 4.(Without the Alpha; at that point, I feel I have achieved what can be done with an engine that has only the basics.)

Also, for the Rustic PSTs I had to write my own King table because Rustics original data uses different tables for mid&endgame. My engine doesn't support that atm. And you know how little I know about chess so that table might be flawed.

Without the endgame table, Rustic can't mate. In the gauntlet, I happen to have seen a game where MMC _also_ can't mate with K+R vs. K. I've seen Rustic mate with K+B+B vs K, but I don't know if it can do this consistently. It can't mate with K+B+N vs K, except if the lone king is near the correct corner already. The engine needs more logic for that (or separate endgame tables).

Thanks! Yes... I actually went farther than what I had in mind for my little "Christmas Break" project. But it's addicting stuff. Now that I have this platform that I understand very well and control there are so many little things I still want to try. For example the next obvious step regarding the evaluation would be to support the PeSTO evaluation.

I assume you mean "Tapered evaluation". That is planned for Rustic 5.

Thanks for the offer! I'll take you up on it for sure! Expect a flood of PMs

OK

But I'm not feeling quite ready to start on a more ambitious new engine. I'm not even sure if Bitboards or Mailbox ala HGM are the way to go or what combination of the two would be best.
In MinimalChess when I explore a Million nodes that means I have literally created a Million instances of the Board-class, via copy-constructor and played a move. In a rewrite I would just have one "fat" Board instance that allows me to Undo moves instead. This allows for lots of little lists and masks and a hashvalue that will be updated incrementally. It's hard to design all this without knowing more about what information I will actually need for my move ordering and the evaluation function. And once you have one kind of information precomputed but not the other kind there's a strong bias to only use the information that is cheaply available. So these early design decisions really matter in the long run.

In other words, before I start from scratch I need a clearer vision of the whole thing. If I do another chess engine I don't want to find out I'm stuck just a few ELO above MMC. MMC is slow by design, my next engine would have no such excuses and I'd have to take the blame for measly performance results myself. My ego is a frail thing, I need to do some studying before I put it at risk like that!

Don't start with all of this. Start with the "Board" struct, with 14 bitboards; 6 for white, 6 for black, one with all the black pieces, and one with all the white pieces. Then write your move generator based on that. After this, you can add things to the Board-struct as you need them.

My entire board struct is just this:

Code: Select all

pub struct Board {
    pub bb_pieces: [[Bitboard; NrOf::PIECE_TYPES]; Sides::BOTH],
    pub bb_side: [Bitboard; Sides::BOTH],
    pub game_state: GameState,
    pub history: History,
    pub piece_list: [Piece; NrOf::SQUARES],
    zr: Arc<ZobristRandoms>,
}

The trick is to not wait with refactoring. As soon as you get something working, refactor it and make it fit into the engine as well as you can. Avoid copy/pasting code, or spaghetti-references. If you refactor a lot, keeping the engine as clean as possible at all times, and only working at ONE function each time, you will avoid a massive amount of bugs and slowdowns.

In the entire development of Rustic, I only had one bug: when I forgot to clear castling rights of a rook that hadn't moved yet. (The author of SebLeagueChess made the exact same mistake, which I found funny. It's an easy one to make, because you never see this over the board.)

Any problems I've been having, only existed in the ONE function I've been writing at each time.

I am convinced that this "go slow and refactor/optimize each function" is one of the reasons why Rustic started out at close to 1700 Elo instead of 1400, what you often see with new engines. No bugs, and high speed. It started out at 1695, but at that time, it hadn't played TSCP yet; which I know would sink it's rating by at least 15 points. Which it did, because TSCP destroyed it, as expected. (And I don't even have all of the move sorting options yet; still need to implement Killers and Heuristics. Maybe I should have done that BEFORE the transposition table. Ah well; the TT works, so it doesn't matter.)

Your engine now starts out at close to 1600, which is a great score for a new engine. The 100 Elo difference between Rustic Alpha 1 and MMC is, as far as I can see, only based on A1's raw speed, and better PST's, as long as you don't use the Sunfish ones. If you do, the difference is only +/- 50 Elo because of MMC's lower speed.

Therefore I'm convinced that, if you write a bitboard engine the same way as you wrote MMC, and use good PST's, you can create a C# engine that starts out at around 1650-1700. Rustic A1 will _maybe_ retain a tiny edge because it will still be faster, but it'll be very slight.

[lithander]

I'm happy to see MinimalChess 0.3 now listed on CCRL with a rating of 1561 which is perfectly in line with my expectations. Thanks a lot for running the tests, Gabor!

Looking at these tables I have one question though: What is the meaning of the last collumn, Perf?

[Ras]

Looking at these tables I have one question though: What is the meaning of the last collumn, Perf?

It's how your engine performs against that specific engine relative to your engine's general Elo. A positive Perf means that your engine does better than expected, and a negative one means your engine scores particularly bad against that engine. It may also mean that your engine just had good or bad luck, given the small number of games against specific engines.

[Gabor Szots]

Thanks a lot for running the tests, Gabor!

Wrong address, Thomas. Those games were actually run by Sergio. He beat me to it but I have just submitted my games for the next update so stay tuned.

[lithander]

It's how your engine performs against that specific engine relative to your engine's general Elo. A positive Perf means that your engine does better than expected, and a negative one means your engine scores particularly bad against that engine. It may also mean that your engine just had good or bad luck, given the small number of games against specific engines.

Ah thanks, that makes sense. In hindsight it seems actually quite obvious... :/

Thanks a lot for running the tests, Gabor!

Wrong address, Thomas. Those games were actually run by Sergio. He beat me to it but I have just submitted my games for the next update so stay tuned.

Okay, Thank you Sergio then!! I really appreciate the work you guys are doing. Chess programming would only be half as fun without it!

[mvanthoor]

I'm happy to see MinimalChess 0.3 now listed on CCRL with a rating of 1561 which is perfectly in line with my expectations. Thanks a lot for running the tests, Gabor!

Congrats with the 500+ Elo improvement. There you go, the engine is close to my guesstimate of about 1582 in my gauntlet (The actual final rating also depends on which engines MMC was tested against, and what opening books were used; i.e. if openings are used that happen to jive well with the PST's, then your results will be better.)

Okay, Thank you Sergio then!! I really appreciate the work you guys are doing. Chess programming would only be half as fun without it!

Second that