MinimalChess 0.4 released

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
lithander
Posts: 915
Joined: Sun Dec 27, 2020 2:40 am
Location: Bremen, Germany
Full name: Thomas Jahn

MinimalChess 0.4 released

Post by lithander »

There is a new release version of MinimalChess!

MinimalChess 0.4

The new version now uses tapered Piece-Square tables to evaluate positions. It took me over two weeks of tuning and testing until I finally managed to find values that could rival PeSTO's famous PSTs in strength. I wrote about it in this thread.

Version 0.4 also introduces a killer heuristic and staged move generation that avoids generating moves which will likely never be played. I have also completely rewritten the time control logic which now allocates the given budget smarter, especially in modes where there's an increment each move.

These changes provide a considerable boost in playing strength to approximately 1900 ELO without making the code any less simple and minimal: Version 0.4 has seen a lot of refactoring and even lost a few lines of code compared to v0.3: It's now down to 625 from previously 641 lines of executable code. (The LOC metric of other engines of comparable strength is usually ranging in the thousands)

You can find a more detailed description, the source code as well as builds for Windows, Mac, Linux and ARM Linux (Rasberry Pi) on Github!
https://github.com/lithander/MinimalChessEngine

Let me know of your impressions should you play the engine! I wonder what experienced chess players think of the playstyle of the new tables. I'm sadly not good enough to judge it myself so please share your verdict here! :)
Minimal Chess (simple, open source, C#) - Youtube & Github
Leorik (competitive, in active development, C#) - Github & Lichess
Mike Sherwin
Posts: 965
Joined: Fri Aug 21, 2020 1:25 am
Location: Planet Earth, Sol system
Full name: Michael J Sherwin

Re: MinimalChess 0.4 released

Post by Mike Sherwin »

lithander wrote: Tue Apr 27, 2021 10:30 am There is a new release version of MinimalChess!

MinimalChess 0.4

The new version now uses tapered Piece-Square tables to evaluate positions. It took me over two weeks of tuning and testing until I finally managed to find values that could rival PeSTO's famous PSTs in strength. I wrote about it in this thread.

Version 0.4 also introduces a killer heuristic and staged move generation that avoids generating moves which will likely never be played. I have also completely rewritten the time control logic which now allocates the given budget smarter, especially in modes where there's an increment each move.

These changes provide a considerable boost in playing strength to approximately 1900 ELO without making the code any less simple and minimal: Version 0.4 has seen a lot of refactoring and even lost a few lines of code compared to v0.3: It's now down to 625 from previously 641 lines of executable code. (The LOC metric of other engines of comparable strength is usually ranging in the thousands)

You can find a more detailed description, the source code as well as builds for Windows, Mac, Linux and ARM Linux (Rasberry Pi) on Github!
https://github.com/lithander/MinimalChessEngine

Let me know of your impressions should you play the engine! I wonder what experienced chess players think of the playstyle of the new tables. I'm sadly not good enough to judge it myself so please share your verdict here! :)
It is difficult to judge playing style when the moves go flying past so quickly. However, my initial impression is that it looks more anti human than human. It does play some post opening positions very well though! Then as the game proceeds it looks more and more mechanical. CAR may not be as strong, nonetheless it plays very human like. Would it be an idea to start with CAR and use your tuning to improve CAR? It might be interesting but I really do not know.
User avatar
Gabor Szots
Posts: 1451
Joined: Sat Jul 21, 2018 7:43 am
Location: Budapest, Hungary
Full name: Gabor Szots

Re: MinimalChess 0.4 released

Post by Gabor Szots »

Excellent work, Thomas. I am looking forward to testing it and, in particular, seeing the new TM in action.
Gabor Szots
CCRL testing group
User avatar
lithander
Posts: 915
Joined: Sun Dec 27, 2020 2:40 am
Location: Bremen, Germany
Full name: Thomas Jahn

Re: MinimalChess 0.4 released

Post by lithander »

Mike Sherwin wrote: Tue Apr 27, 2021 2:53 pm It is difficult to judge playing style when the moves go flying past so quickly. However, my initial impression is that it looks more anti human than human. It does play some post opening positions very well though! Then as the game proceeds it looks more and more mechanical.
Thanks! Yeah, I guess that makes sense: it's probably easier to encode how to develop your pieces from the starting position forward in static PST tables than how to play an endgame so the engine starts to just trade pieces because the PSTs say that it's worth a few centipawns until it hopefully simpliefies the position enough that it sees a way to mate the enemy king. But at least it does that quite well now... with a Rook or Queen it's guaranteed, with two Bishops it takes a bit of luck and more then a few milliseconds time so the necessary depths can be reached.
Mike Sherwin wrote: Tue Apr 27, 2021 2:53 pm CAR may not be as strong, nonetheless it plays very human like. Would it be an idea to start with CAR and use your tuning to improve CAR? It might be interesting but I really do not know.
I guess the tuning would overwrite the personality of CAR pretty quickly to better match the dataset's idea of where pieces should be placed in a winning position.
Gabor Szots wrote: Tue Apr 27, 2021 5:48 pm Excellent work, Thomas. I am looking forward to testing it and, in particular, seeing the new TM in action.
I'm as always eagerly anticipating the new placement on the CCRL! Thanks for taking the time!

The new TM basically allows the engine to spend more of the starting budget early when there is a regular increment because with an increment there's no real risk of ever losing due to running out of time. The previous version would often end games with more time than it started with, wich of course isn't ideal! ;)
Minimal Chess (simple, open source, C#) - Youtube & Github
Leorik (competitive, in active development, C#) - Github & Lichess
User avatar
mvanthoor
Posts: 1784
Joined: Wed Jul 03, 2019 4:42 pm
Location: Netherlands
Full name: Marcel Vanthoor

Re: MinimalChess 0.4 released

Post by mvanthoor »

lithander wrote: Tue Apr 27, 2021 10:30 am There is a new release version of MinimalChess!

MinimalChess 0.4

The new version now uses tapered Piece-Square tables to evaluate positions. It took me over two weeks of tuning and testing until I finally managed to find values that could rival PeSTO's famous PSTs in strength. I wrote about it in this thread.

Version 0.4 also introduces a killer heuristic and staged move generation that avoids generating moves which will likely never be played. I have also completely rewritten the time control logic which now allocates the given budget smarter, especially in modes where there's an increment each move.

These changes provide a considerable boost in playing strength to approximately 1900 ELO without making the code any less simple and minimal: Version 0.4 has seen a lot of refactoring and even lost a few lines of code compared to v0.3: It's now down to 625 from previously 641 lines of executable code. (The LOC metric of other engines of comparable strength is usually ranging in the thousands)

You can find a more detailed description, the source code as well as builds for Windows, Mac, Linux and ARM Linux (Rasberry Pi) on Github!
https://github.com/lithander/MinimalChessEngine

Let me know of your impressions should you play the engine! I wonder what experienced chess players think of the playstyle of the new tables. I'm sadly not good enough to judge it myself so please share your verdict here! :)
Congratulations :) I haven't been able to work on Rustic for a few weeks (read: home improvement project). I only managed to write a build script to automatically build all Rustic versions for the platform it's compiling on :?

How can you write a complete chess engine in under 700 lines of code without using chess libraries? You're right that comparable engines have thousands of lines of code; In case of Rustic, it's about 4500. (This is including the not yet enabled XBoard protocol, and including a gazillion list of constants and tables. I could shorten the code by thousands of lines if I'd put every array constant on a single line, and just use magic numbers and strings instead of constants.)

No transposition table yet, and already 1900 Elo? Nice. That's a bump of ~325 Elo.

You have added functions that Rustic doesn't have yet, so if I'd add ~325 Elo to my engine, it'd be 2140. I could live with that. We'll see. First killer moves, history heuristics, pvs, and aspriation windows. Then some tuning. First I'll have to rerun a few gauntlets so I can recalibrate the Elo calculation in my test procedure to be more CCRL-compatible.
Author of Rustic, an engine written in Rust.
Releases | Code | Docs | Progress | CCRL
User avatar
lithander
Posts: 915
Joined: Sun Dec 27, 2020 2:40 am
Location: Bremen, Germany
Full name: Thomas Jahn

Re: MinimalChess 0.4 released

Post by lithander »

mvanthoor wrote: Wed Apr 28, 2021 3:06 pm How can you write a complete chess engine in under 700 lines of code without using chess libraries? You're right that comparable engines have thousands of lines of code; In case of Rustic, it's about 4500. (This is including the not yet enabled XBoard protocol, and including a gazillion list of constants and tables. I could shorten the code by thousands of lines if I'd put every array constant on a single line, and just use magic numbers and strings instead of constants.)
"Lines of the executable code" is a metric built into Visual Studio that does not stupidly count every line in the source code. Instead it looks at the size of your code after it has been compiled to IL and counts that. This makes the metric does not discourage from writing good, maintainable code. If you shorten your the code by replacing constants with magic numbers etc and make your code ugly and condensed and confusing then LOC metric wouldn't change a bit. It would NOT give you the false impression that you improved the code when in reality you didn't.

If I would count lines of source code instead I'm at ~2000 but that includes empty lines, curly braces, comments, copyright notices... it's absolutely meaningless. So in the C# world when someone says Lines of code (or LOC) he means lines of executable code usually.

Don't compare apples with oranges of course, but for example it's fair to compare my 620 LOC with other C# engines such as MadChess' over 4000 lines of code listed on the MadChess Download page Because Eric does also use the meaningful metric of executable lines of code when he gives these stats. If he would actually count every line of source code I guess his engines would have over 10.000. (Correct me if I'm wrong, Eric)
mvanthoor wrote: Wed Apr 28, 2021 3:06 pm You have added functions that Rustic doesn't have yet, so if I'd add ~325 Elo to my engine, it'd be 2140. I could live with that.
You also have functions that I don't have like TT and bitboards etc! ;) I don't think our engines are so easily comparable anymore... but I'm fairly sure you could gain a lot of ELO by moving to tapered PSTs. I'd feel honored if you chose to borrow mine! MMC used Rustics tables for a while, so it would only be fair!
Minimal Chess (simple, open source, C#) - Youtube & Github
Leorik (competitive, in active development, C#) - Github & Lichess
User avatar
mvanthoor
Posts: 1784
Joined: Wed Jul 03, 2019 4:42 pm
Location: Netherlands
Full name: Marcel Vanthoor

Re: MinimalChess 0.4 released

Post by mvanthoor »

lithander wrote: Wed Apr 28, 2021 4:18 pm "Lines of the executable code" is a metric built into Visual Studio that does not stupidly count every line in the source code. Instead it looks at the size of your code after it has been compiled to IL and counts that. This makes the metric does not discourage from writing good, maintainable code. If you shorten your the code by replacing constants with magic numbers etc and make your code ugly and condensed and confusing then LOC metric wouldn't change a bit. It would NOT give you the false impression that you improved the code when in reality you didn't.

If I would count lines of source code instead I'm at ~2000 but that includes empty lines, curly braces, comments, copyright notices... it's absolutely meaningless. So in the C# world when someone says Lines of code (or LOC) he means lines of executable code usually.

Don't compare apples with oranges of course, but for example it's fair to compare my 620 LOC with other C# engines such as MadChess' over 4000 lines of code listed on the MadChess Download page Because Eric does also use the meaningful metric of executable lines of code when he gives these stats. If he would actually count every line of source code I guess his engines would have over 10.000. (Correct me if I'm wrong, Eric)
LoEC is indeed a much better metric than the old LoC. Many engines with "small source code" are also very hard to maintain because they don't use constants, or write entire "if" or even "switch" statements onto a single line.

Still, I feel that Rustic is somewhat gigantic with around 4500 lines of code (which is everything, except comments and blank lines). But, I do have a penchant for making really tiny steps:

Code: Select all

a = 5
b = 10
c = 7

x = a * b
y = b + c
result = x - y
I could also have written:

Code: Select all

r = (5 * 10) - (10 + 7)
In that case however, you don't know what "5" or "10" is, and you don't know what "5 * 10" represents, and so on. This way of writing code adds a lot of lines; together with the fact that I _hate_ long files. If a file gets over over 250-300 lines, I'm going to look into splitting it up if at all possible.
mvanthoor wrote: Wed Apr 28, 2021 3:06 pm You have added functions that Rustic doesn't have yet, so if I'd add ~325 Elo to my engine, it'd be 2140. I could live with that.
You also have functions that I don't have like TT and bitboards etc! ;) I don't think our engines are so easily comparable anymore... but I'm fairly sure you could gain a lot of ELO by moving to tapered PSTs. I'd feel honored if you chose to borrow mine! MMC used Rustics tables for a while, so it would only be fair!
[/quote]

I may look into MinimalChess's tuner, because I don't have a clue about how to go about writing one. I'll have to research the algorithm first.

With regard to PST's: I still see many engines (BitGenie is the latest I checked, for example, but certainly not the only one) that re-add all the PST values in the evaluation function. That takes a lot of time, because for every move, you are re-adding all the numbers. I add all the values once, at the beginning of the game, and then keep this value incrementally (just like the Zobrist-keys).

With regard to tapering, I think I'm just going to change the current name to "MG_", and add incremental values for "EG_", and another for "game_phase". Then I only need to calculate the final tapered value in the evaluation function. In the beginning, both MG and EG will be the same, leading to the exact same evaluation as if it wasn't tapered. (And thus the engine will become slower because it has to keep more data.) Then I'll tune both MG and EG. That way, I can just start with and tune my own PST's, and then either be amazed (or disappointed) how much strength the tapering and tuning adds.

(I feel it could be less than expected, because my PST's already encode quite a bit of my own positional knowledge. At the moment, Rustic plays a lot like a faster, deeper-thinking version of myself... sometimes that's really eerie to see, if you're watching a game, expect a certain move, and the engine actually plays that move about 75% of the time. Against Alpha 2, my own draw rate is _very_ high :lol: )
Author of Rustic, an engine written in Rust.
Releases | Code | Docs | Progress | CCRL
User avatar
lithander
Posts: 915
Joined: Sun Dec 27, 2020 2:40 am
Location: Bremen, Germany
Full name: Thomas Jahn

MinimalChess 0.4.1 released

Post by lithander »

Gabor Szots wrote: Tue Apr 27, 2021 5:48 pm Excellent work, Thomas. I am looking forward to testing it and, in particular, seeing the new TM in action.
Version 0.4 sadly had a bug where the 'depth' option in the UCI go command was no longer handled correctly. As this was the only way to make the engine play quickly while giving a human player a lot more (or infinite) amount of time I felt like I should fix that immediately and update the release. I now also added support for the 'nodes' constraint.

If you have already begun testing don't worry about it, otherwise please consider downloading and testing version 0.4.1 instead! Thanks!
Minimal Chess (simple, open source, C#) - Youtube & Github
Leorik (competitive, in active development, C#) - Github & Lichess
User avatar
emadsen
Posts: 440
Joined: Thu Apr 26, 2012 1:51 am
Location: Oak Park, IL, USA
Full name: Erik Madsen

Re: MinimalChess 0.4 released

Post by emadsen »

lithander wrote: Wed Apr 28, 2021 4:18 pmIt's fair to compare my 620 LOC with other C# engines such as MadChess' over 4000 lines of code listed on the MadChess Download page Because Eric does also use the meaningful metric of executable lines of code
That's correct Thomas. I use Visual Studio > Analyze > Calculate Code Metrics > For Solution > Lines of Executable Code. If I exclude tuning or test code it's about 4100 lines.
Erik Madsen | My C# chess engine: https://www.madchess.net
User avatar
lithander
Posts: 915
Joined: Sun Dec 27, 2020 2:40 am
Location: Bremen, Germany
Full name: Thomas Jahn

Re: MinimalChess 0.4 released

Post by lithander »

mvanthoor wrote: Wed Apr 28, 2021 5:20 pm I may look into MinimalChess's tuner, because I don't have a clue about how to go about writing one. I'll have to research the algorithm first.
You'll find it in the dev branch. It's simpler than you might think. Gradient descent in it's simplest form is to try adding and subtracting 1 to each table value and to continue to do that until it ceases to improve the eval. I had the basic tuner written in one evening.

But when the PSTs stabilize and neither adding nor subtracting values anywhere gets you any improvement anymore you're sadly still far from done. You're just stuck at one of a billion possible local maxima. You need to realize that what you minimize with the tuner is the mean squared error of your eval. You teach your PST eval to predict as accurately as possible the outcome of a specific position - on average. What that does not automatically mean is that if you play by these PST values your engine will develop pieces in the right order, convert won endgames and just generally play good. It's also prone to overfitting if the dataset is small.

It was a good thing that I had started out with PeSTOs values so that I knew what was theoretically possible. Not getting there with full autotuning (start the tuner wait for 6 hours and get the final results) I started to add a command-line interface to the tuner. Now I could start to come up with "recipes", series of commands that would refine tables in a supervised manner.

Code: Select all

>>1
Loading DATA from 'data/quiet-labeled.epd'
725000 labeled positions loaded!
272601 - 253615 - 198784 [0.513] 725000
-- Operation took 2.7797s

>>restore material
Loading EVAL from 'material'
MeanSquareError(k=102): 0.28284902129905315
-- Operation took 0.1056s

>>compare material material
Comparing 'material' vs 'material'
    136038| +0.423075 | +0.000000 | +0.423075 | 136038
     20448| +0.405576 | +0.000000 | +0.405576 | 20448
     24340| +0.343198 | +0.000000 | +0.343198 | 24340
     23594| +0.338060 | +0.000000 | +0.338060 | 23594
     25449| +0.321714 | +0.000000 | +0.321714 | 25449
     41394| +0.307132 | +0.000000 | +0.307132 | 41394
     44528| +0.264205 | +0.000000 | +0.264205 | 44528
     64713| +0.218483 | +0.000000 | +0.218483 | 64713
     89342| +0.226750 | +0.000000 | +0.226750 | 89342
    255154| +0.218794 | +0.000000 | +0.218794 | 255154
------------------------------------------------------
            +0.282849 | +0.000000 | +0.282849

>>tune mg 5 0 0.1 1 20 (18% pos)
>>tune eg 5 0.9 1 1 20 (36% pos)
>>dump chili001

Comparing 'chilli001' vs 'material'
    136038| +0.380039 | -0.043036 | +0.423075 | 136038
     20448| +0.378834 | -0.026742 | +0.405576 | 20448
     24340| +0.320445 | -0.022754 | +0.343198 | 24340
     23594| +0.321228 | -0.016832 | +0.338060 | 23594
     25449| +0.312176 | -0.009538 | +0.321714 | 25449
     41394| +0.295421 | -0.011712 | +0.307132 | 41394
     44528| +0.253195 | -0.011010 | +0.264205 | 44528
     64713| +0.202319 | -0.016165 | +0.218483 | 64713
     89342| +0.207705 | -0.019045 | +0.226750 | 89342
    255154| +0.193081 | -0.025713 | +0.218794 | 255154
------------------------------------------------------
            +0.258189 | -0.024660 | +0.282849
-- Operation took 0.2394s

>>tune mg 5 0 0.1 0.1 20 (30% pos)
>>tune eg 5 0.9 1 0.1 20 (31% pos)

>>dump base002

Comparing 'chilli002' vs 'chilli001'
    136038| +0.368988 | -0.011051 | +0.380039 | 136038
     20448| +0.365006 | -0.013828 | +0.378834 | 20448
     24340| +0.307752 | -0.012693 | +0.320445 | 24340
     23594| +0.310328 | -0.010900 | +0.321228 | 23594
     25449| +0.301211 | -0.010965 | +0.312176 | 25449
     41394| +0.288955 | -0.006465 | +0.295421 | 41394
     44528| +0.245307 | -0.007887 | +0.253195 | 44528
     64713| +0.199368 | -0.002951 | +0.202319 | 64713
     89342| +0.203121 | -0.004584 | +0.207705 | 89342
    255154| +0.187898 | -0.005183 | +0.193081 | 255154
------------------------------------------------------
            +0.251054 | -0.007135 | +0.258189


>>tune eg 5 0.8 1.0 0.1 10
>>tune mg 5 0 0.2 0.1 10

>>dump chilli003

>>compare base003 base002
Comparing 'chilli003' vs 'chilli002'
    136038| +0.368536 | -0.000452 | +0.368988 | 136038
     20448| +0.357939 | -0.007067 | +0.365006 | 20448
     24340| +0.303871 | -0.003880 | +0.307752 | 24340
     23594| +0.304961 | -0.005367 | +0.310328 | 23594
     25449| +0.296801 | -0.004411 | +0.301211 | 25449
     41394| +0.285261 | -0.003694 | +0.288955 | 41394
     44528| +0.241685 | -0.003622 | +0.245307 | 44528
     64713| +0.197558 | -0.001809 | +0.199368 | 64713
     89342| +0.200568 | -0.002553 | +0.203121 | 89342
    255154| +0.188328 | +0.000430 | +0.187898 | 255154
------------------------------------------------------
            +0.249552 | -0.001502 | +0.251054

>>tune eg 1 0.9 1.0 0.1 5
>>tune eg 1 0.8 1.0 0.1 5
>>tune mg 1 0 0.1 0.1 5
>>tune mg 1 0 0.2 0.1 5
>>dump base004

>>compare chilli004 chilli003
Comparing 'chilli004' vs 'chilli003'
    136038| +0.367441 | -0.001095 | +0.368536 | 136038
     20448| +0.356104 | -0.001835 | +0.357939 | 20448
     24340| +0.302772 | -0.001099 | +0.303871 | 24340
     23594| +0.304067 | -0.000894 | +0.304961 | 23594
     25449| +0.296616 | -0.000185 | +0.296801 | 25449
     41394| +0.284999 | -0.000263 | +0.285261 | 41394
     44528| +0.241373 | -0.000312 | +0.241685 | 44528
     64713| +0.197340 | -0.000218 | +0.197558 | 64713
     89342| +0.200555 | -0.000013 | +0.200568 | 89342
    255154| +0.188081 | -0.000247 | +0.188328 | 255154
------------------------------------------------------
            +0.249080 | -0.000472 | +0.249552

>>tune phase 10 0.05 20
>>dump deep_base005

Comparing 'chilli005' vs 'chilli004'
     98444| +0.399894 | +0.032454 | +0.367441 | 136038
     28559| +0.330323 | -0.025781 | +0.356104 | 20448
     27376| +0.309899 | +0.007126 | +0.302772 | 24340
     28986| +0.307396 | +0.003329 | +0.304067 | 23594
     31239| +0.280778 | -0.015838 | +0.296616 | 25449
     43150| +0.281020 | -0.003978 | +0.284999 | 41394
     54768| +0.236880 | -0.004493 | +0.241373 | 44528
     78449| +0.190641 | -0.006698 | +0.197340 | 64713
    137489| +0.187562 | -0.012992 | +0.200555 | 89342
    196540| +0.202689 | +0.014607 | +0.188081 | 255154
------------------------------------------------------
            +0.249166 | +0.000086 | +0.249080

>>tune eg 1 0.9 1.0 0.1 5
>>tune eg 1 0.7 1.0 0.1 5
>>tune mg 1 0 0.1 0.1 5
>>tune mg 1 0 0.3 0.1 5
>>dump base007

>>compare chilli006 chilli005
Comparing 'chilli006' vs 'chilli005'
     98444| +0.399460 | -0.000434 | +0.399894 | 98444
     28559| +0.329706 | -0.000618 | +0.330323 | 28559
     27376| +0.309042 | -0.000857 | +0.309899 | 27376
     28986| +0.306820 | -0.000576 | +0.307396 | 28986
     31239| +0.280169 | -0.000609 | +0.280778 | 31239
     43150| +0.280314 | -0.000706 | +0.281020 | 43150
     54768| +0.236373 | -0.000506 | +0.236880 | 54768
     78449| +0.189674 | -0.000967 | +0.190641 | 78449
    137489| +0.187562 | -+0.000000 | +0.187562 | 137489
    196540| +0.202950 | +0.000262 | +0.202689 | 196540
------------------------------------------------------
            +0.248887 | -0.000279 | +0.249166

>>tune phase 5 0 20
>>dump chilli007

>>compare chilli007 chilli006
Comparing 'chilli007' vs 'chilli006'
     98444| +0.399672 | +0.000212 | +0.399460 | 98444
     28559| +0.327450 | -0.002255 | +0.329706 | 28559
     27376| +0.304703 | -0.004339 | +0.309042 | 27376
     28986| +0.301152 | -0.005668 | +0.306820 | 28986
     31239| +0.275945 | -0.004224 | +0.280169 | 31239
     43150| +0.274859 | -0.005455 | +0.280314 | 43150
     54768| +0.232293 | -0.004081 | +0.236373 | 54768
     78449| +0.186986 | -0.002689 | +0.189674 | 78449
    137489| +0.186456 | -0.001105 | +0.187562 | 137489
    196540| +0.203213 | +0.000263 | +0.202950 | 196540
------------------------------------------------------
            +0.247192 | -0.001695 | +0.248887
-- Operation took 0.5634s

>>tune phase 5 0 20
Writing EVAL to 'dump'
Tuning step=5, suppression=0, passCap=20 on 725000 positions
#0: 0.2471922726756248 MSE, MG 5290..450 EG
#1: 0.24719145648261692 MSE, 8.161930078742774E-07 delta, 1 inc, 1 dec!
#2: 0.24718859310194624 MSE, 3.6795736785555366E-06 delta, 2 inc, 1 dec!
#3: 0.24718348819158992 MSE, 8.784484034879414E-06 delta, 2 inc, 0 dec!
#4: 0.24718235321621554 MSE, 9.919459409257758E-06 delta, 2 inc, 1 dec!
#5: 0.24718113022982086 MSE, 1.1142445803941081E-05 delta, 1 inc, 1 dec!
#6: 0.24718024674414388 MSE, 1.2025931480919017E-05 delta, 1 inc, 0 dec!
#7: 0.24718024674414388 MSE, 1.2025931480919017E-05 delta, 0 inc, 0 dec!
MG 5255..435 EG!
Done! Eval improved by 1.2025931480919017E-05
-- Operation took 68.7297s

>>tune tables 1 0.01 1 20
>>dump chilli008

Comparing 'chilli008' vs 'chilli007'
    105609| +0.396982 | -0.002690 | +0.399672 | 98444
     28134| +0.309869 | -0.017581 | +0.327450 | 28559
     26512| +0.307327 | +0.002624 | +0.304703 | 27376
     28655| +0.310953 | +0.009801 | +0.301152 | 28986
     35035| +0.274207 | -0.001738 | +0.275945 | 31239
     41092| +0.248645 | -0.026214 | +0.274859 | 43150
     54940| +0.232610 | +0.000317 | +0.232293 | 54768
     92300| +0.198449 | +0.011464 | +0.186986 | 78449
    116183| +0.174541 | -0.011915 | +0.186456 | 137489
    196540| +0.203098 | -0.000115 | +0.203213 | 196540
------------------------------------------------------
            +0.246644 | -0.000548 | +0.247192
-- Operation took 0.201s

>>tune tables 1 0.01 1 20
>>dump chilli009

Comparing 'chilli009' vs 'chilli008'
    105609| +0.396885 | -0.000097 | +0.396982 | 105609
     28134| +0.309842 | -0.000027 | +0.309869 | 28134
     26512| +0.307148 | -0.000179 | +0.307327 | 26512
     28655| +0.310624 | -0.000330 | +0.310953 | 28655
     35035| +0.273864 | -0.000343 | +0.274207 | 35035
     41092| +0.248356 | -0.000288 | +0.248645 | 41092
     54940| +0.232571 | -0.000039 | +0.232610 | 54940
     92300| +0.198357 | -0.000092 | +0.198449 | 92300
    116183| +0.174553 | +0.000012 | +0.174541 | 116183
    196540| +0.203064 | -0.000034 | +0.203098 | 196540
------------------------------------------------------
            +0.246555 | -0.000090 | +0.246644
-- Operation took 0.2699s

>>mix pepper017
[P] EEEEEEEE..X...M..EM....ME..E.E..E.E....EEEE.E..E..E..E.EEEEEEEEE
[N] .M...E..M.......M....M..................................E.......
[B] M.MM.................M..........................................
[R] ...XXX.....XE.EEMEE....E.M..E..EEX.EEEEE.EEM..E.XEXEX.XE.MXM..M.
[Q] .E.EXXXXEXXXXMEMEEEXXEE..EEEEEXE.XEEXXE.E.E.XEEX.EX.X...MEMEEMXX
[K] .EEMM.M..MEM.MM.EM.M.M.M....MM.M...MMMMX.......X..........E...EE
-- Operation took 185.9885s

>>dump chilipepper002

>>tune tables 1 0.01 1 20
[...]
#19: 0.2181464328261707 MSE, 0 inc, 0 dec!
Done! Eval improved by 0.028341865013501527

>>dump chillipepper003

Comparing 'chillipepper003' vs 'chilli009'
    105609| +0.396856 | -0.000028 | +0.396885 | 105609
     28134| +0.309630 | -0.000212 | +0.309842 | 28134
     26512| +0.306764 | -0.000383 | +0.307148 | 26512
     28655| +0.310108 | -0.000515 | +0.310624 | 28655
     35035| +0.273508 | -0.000356 | +0.273864 | 35035
     41092| +0.247912 | -0.000445 | +0.248356 | 41092
     54940| +0.232401 | -0.000170 | +0.232571 | 54940
     92300| +0.198270 | -0.000087 | +0.198357 | 92300
    116183| +0.174543 | -0.000010 | +0.174553 | 116183
    196540| +0.203038 | -0.000026 | +0.203064 | 196540
------------------------------------------------------
            +0.246433 | -0.000122 | +0.246555
That's the recipe I followed to get the final tables that version 0.4 shipped with and that can hold a candle to PeSTO. (or actually in my tests improve upon it by 20 ELO) So the difficulty was not in writing the tuner but figuring out why the tuned values didn't work and improving the recipe and the capabilities of the tuner (lots of commands and command options added later on to address specific needs) until it finally clicked.
mvanthoor wrote: Wed Apr 28, 2021 5:20 pmWith regard to PST's: I still see many engines (BitGenie is the latest I checked, for example, but certainly not the only one) that re-add all the PST values in the evaluation function. That takes a lot of time, because for every move, you are re-adding all the numbers. I add all the values once, at the beginning of the game, and then keep this value incrementally (just like the Zobrist-keys).
I tried a lot of optimizations and features in the dev branch that didn't make it into master branch. Incremental eval was one of the things I tried. If you don't compute the score from scratch in the eval you have to keep track of the midgameScore, the endgameScore and the phase of a board position. You need to implement AddPiece() and RemovePiece() to change these values and call it whenever you change something in the board state. And in my case where I don't have Make/Unmake but depend on a Copy/Make approach instead this means that my copy-constructer has a little more work to do. All this adds up. In code-complexity and also it takes time - still only a fraction of doing eval from scratch but in the end it didn't do me enough good to keep it. If I optimize stuff like this, add piece lists etc... then I can go from 1M nodes per second to 1.7M nps. That's really not worth the effort... or let's say it doesn't fit my vision for minimal chess. I'm making good progress still without turning to desperate measures like micro-optimizations! :)
Minimal Chess (simple, open source, C#) - Youtube & Github
Leorik (competitive, in active development, C#) - Github & Lichess