Pedantic Developer's Log Stardate...

[JoAnnP38]

You use MinimalChess and MadChess already so I understand if you don't want another C# engine. But Leorik 1.0 should otherwise be interesting for you. It doesn't have any of the bugs or issues you list above and it's interesting because it gives you the same results as a min-max algorithm just faster. In other words it never reduces or prunes and thus never misses a thing unless it's beyond the search horizon. Search depth of course ain't great if you have no selectivity whatsoever. But I think this is a rather unique property for an engine. It was 2150 CCRL Blitz Elo last I checked. Could be more now as many engines in that list have seen their Elo go up by ~50 points recently.

Leorik (even 1.0) was too strong for the cohort I was assembling. However, God-willing maybe I can include Leorik in my second-release testing. By yes, I was definitely considering it.

[JoAnnP38]

If you've been following this thread you know that I just previously completed a round-robin tournament against a cohort of 9 other engines at 40+0.4s time control. While I had hoped for more than the 2276 Elo I felt I fairly earned, I was satisfied. My next step was to run a short tourney at much longer time controls (i.e. 40/3:00+6) just to make sure Pedantic didn't exhibit some undesirable characteristics at longer times. Well the results are in and I can happily say that Pedantic weathered the gauntlet. However, most of the other contestants -- NOT SO MUCH!

Code: Select all

Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw
   0 Pedantic 0.1                  218      61     160     117      28      15    124.5   77.8%    9.4%
   1 Horizon 4.4                     9      94      40      15      14      11     20.5   51.2%   27.5%
   2 Glass 1.2                    -117     114      40      12      25       3     13.5   33.8%    7.5%
   3 CeeChess 1.3.2               -564     nan      40       1      38       1      1.5    3.8%    2.5%
   4 GOOB 1.0.0                   -inf     nan      40       0      40       0      0.0    0.0%    0.0%

SPRT: llr 2.89 (98.2%), lbound -2.94, ubound 2.94
Finished match

Nearly all of the engines except Horizon 4.4, suffered from longer time controls. Glass 1.2 lost more than half of its games due to running out of time and both CeeChess and GOOB lost all of their games on time!

When I first started building Pedantic, I had no clue how to control for time in my engine. I have never played a competitive game of chess or even seen a chess clock. As such I didn't feel a bit guilty when I lifted (whole cloth mind you) the TimeControl class from lithander's MinimalChess. Yes, I've made a few tweaks to it along the way, but I never understood until just now what I had stolen. Now, I feel guilty. Perhaps had I banged my head against a wall long enough I would have come up with a time control class that worked and then sputter out just like these other engines have.

I'm going to run another short gauntlet that includes the other higher ranking engines from my earlier hyper-bullet, round-robin and drop out these losers!

[JoAnnP38]

Okay, a second gauntlet was completed with all the stronger engines from the initial round-robin, but it still yielded surprising results. Gaia 3.5, the strongest engine from the hyper-bullet, round-robin could not deal with the longer time controls (i.e. 40/3:00+6) and lost nearly every game due to time. And while the other engines dealt with the time control just fine, Pedantic played much stronger against them! I am going to put together another round-robin with some stronger engines, but longer time controls than the first and then see how Pedantic fares.

Code: Select all

Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw
   0 Pedantic 0.1                   68      49     160      80      49      31     95.5   59.7%   19.4%
   1 AbsoluteZero 2.4.7.2           26      90      40      15      12      13     21.5   53.8%   32.5%
   2 MadChess 2.00                  26      97      40      17      14       9     21.5   53.8%   22.5%
   3 Horizon 4.4                     9      97      40      16      15       9     20.5   51.2%   22.5%
   4 Gaia 3.5                     -636     nan      40       1      39       0      1.0    2.5%    0.0%

SPRT: llr 1.04 (35.2%), lbound -2.94, ubound 2.94
Finished match

[JoAnnP38]

Announcement: Official release of Pedantic 0.1 is now available
I am very happy to announce my first official release of Pedantic 0.1. Pedantic is an UCI engine that I have tested in both Arena and CuteChess. It comes with its own opening book, but of course you don't' have to use it. At time control 40+0.4 Pedantic 0.1 has demonstrated an Elo strength of 2276. If you have need of an engine with this playing strength please give it a try. I would be happy to hear any experiences you have with it especially if you encounter any bugs.

NOTES: While I can compile a version of Pedantic not to use hardware intrinsics, the release binaries currently rely on the following:

• Bmi1.X64.TrailingZeroCount
• Lzcnt.X64.LeadingZeroCount
• Bmi1.X64.ResetLowestSetBit
• Popcnt.X64.PopCount
• Bmi1.X64.BitFieldExtract

Unfortunately, I am just now realizing that I neglected to test a version that doesn't use these intrinsics. I will try to rectify that before the 0.2 release is available.

[lithander]

Congratz on the good progress and the release!

And I was also happy to hear that MinimalChess' time control code is used in your engine and more importantly seems to work well for you. It looks almost trivial but I remember well when I tried optimizing it for all possible tc settings without bloating the code.

[JoAnnP38]

Congratz on the good progress and the release!

And I was also happy to hear that MinimalChess' time control code is used in your engine and more importantly seems to work well for you. It looks almost trivial but I remember well when I tried optimizing it for all possible tc settings without bloating the code.

One of the areas I have outlined for investigation in my next release is improving my time control along these lines -- as a non-chess player, I look at time control as perhaps a game within a game. For instance, it may pay to make judgements on how much time to budget based on how well your opponent is doing on time and whether or not pondering is enabled. If I can assist their demise based on time, why not? Also, I made a small change to your class that ended up ensuring Pedantic never lost on time except in cases where there is no increment (or the increment is smaller than the time required to search 1 ply). It works great, but it may mean I'm taking too little risk with my time to maximize performance. I may also want to dynamically collect statistics on branching factor so that would allow better time budgets based on the phase of the game I'm in. Additionally, I've seen some information online that the odd ply branching factor is generally larger than even ply branch factors due to how alpha-beta works. I don't really know, but hey that's why I'm going to investigate it.

[JoAnnP38]

I'm making good progress on release 0.2 of Pedantic. Some of the features I have completed so far:

• Re-added mop-up evaluation to Pedantic based on the design described in Chess 4.5. This was removed from 0.1 as I didn't have time to test. (+14 Elo)
• Optimized the branching factor used by time controls for predicting time spent in next iteration (+28 Elo)
• Optimized Razoring/LMP pruning (+58 Elo)

I am currently adding SEE to Pedantic, but I'm not having a lot of luck so far. On my first attempt I used it to determine whether or not Quiescence should explore certain captures or whether it can be pruned. Also, I used it to help eliminate checking moves that result in a piece being lost immediately. Unfortunately, this resulted in a serious loss of Elo. I'm going back to square on and going to take this step by step. This should help not hinder the strength of Pedantic but we'll see.

[lithander]

I'm making good progress on release 0.2 of Pedantic. Some of the features I have completed so far:

• I am currently adding SEE to Pedantic, but I'm not having a lot of luck so far. On my first attempt I used it to determine whether or not Quiescence should explore certain captures or whether it can be pruned. Also, I used it to help eliminate checking moves that result in a piece being lost immediately. Unfortunately, this resulted in a serious loss of Elo. I'm going back to square on and going to take this step by step. This should help not hinder the strength of Pedantic but we'll see.

Using SEE in MinimalChess to identify and skip bad captures in Qsearch was worth some Elo. But I never comitted it to the master branch because of the pledge to stay minimal. In Leorik my first attemt to do exactly the same thing that worked in MMC lost Elo. Presumably because Leorik is much faster and playing the bad capture does not take significantly longer then running inefficient SEE on every capture in QSearch.

When I implemented SEE from scratch in a different and much faster way I finally got the Elo gain I was hoping for. I have since found other uses of SEE in the search, so I'd say it's a useful tool to have.

I have compiled a list of 74 testcases from various sources and added some positions of my own to verify correctness of my implementation: https://github.com/lithander/Leorik/blo ... st/see.epd

You could check if your implementation agrees on these positions with Leorik!

[JoAnnP38]

I have compiled a list of 74 testcases from various sources and added some positions of my own to verify correctness of my implementation: https://github.com/lithander/Leorik/blo ... st/see.epd

You could check if your implementation agrees on these positions with Leorik!

Wow, that is super generous of you. I had started putting together my own test suite, but I currently only have 5 positions. Thank you!

This release Pedantic should be strong enough to include both MinimalChess 0.6 and a version of Leorik in my round-robin test tournament to establish an aggregate Elo. If you are interested in the PGN(s) to add to your own training database let me know. I'm guessing it will be a couple of more weeks before I am at that stage based on the content I am planning for this release.

[JoAnnP38]

I am closing in on release 0.2 of Pedantic and I may be able to attain 2500 Elo rating in this release. That would be a pretty good pickup from my first release. I have uncovered a recent bug that I am perplexed about fixing, because every time it happens, I have a good laugh. The bug causes Pedantic to under-promote even when it doesn't need a knight for checkmate. The first time it happened it was when I was testing Pedantic against Leorik and Pendantic underpromoted to a Rook. It didn't affect the outcome of the game because it was a promotion that was intended to force Leorik to capture and then Leorik's rook was captured in exchange. At the time I had a good belly laugh as I imagined Pedantic trolling Leorik. Since then, it has occurred a handful of times. Today, it underpromoted to a bishop that wasn't en prise!! Pedantic went on to win the game, but this bug needs to be stamped out, even though I still laugh at the prospect that Pedantic is taunting the other engine.