patricia devlog

[pohl4711]

Further improvement has been a bit tricky, but today I found a bug in SEE that made it not be able to detect pawn sacrifices. Hopefully that will help increase Patricia's sacrifice count.

The answer.... was yes.
First attempt.

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    301361  29.84%  27.40%  07.52%   62   Patricia  
   2    127198  14.31%  30.03%  19.48%   64   Peacekeeper 1.50  
   3    100220  15.11%  18.40%  17.00%   68   Willow 2.8  
   4     99916  10.18%  18.71%  15.49%   70   Midnight v5  
   5     80646  11.02%  15.92%  17.82%   71   Wahoo v3  
   6     73076  09.62%  16.47%  20.34%   70   Polaris 1.6.1  
   7     54251  03.20%  20.28%  29.45%   66   Akimbo 0.4.1  
-------------------------------------------------------------------

Wow! This is an awesome EAS-Score!!! And all stats look great (sacs, shorts, bad draws..), not only the overall score. I am really impressed.

[Whiskers]

I ran a test at a longer time control, results were pretty similar. Patricia's a menace. How long until the next release? Is 400k EAS possible?

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    307560  36.08%  27.83%  07.62%   62   Patricia  
   2    102447  13.83%  24.85%  22.25%   68   Peacekeeper 1.50  
   3     84922  15.35%  14.11%  18.89%   71   Willow 2.8  
   4     74646  12.96%  13.76%  20.91%   70   Midnight v5  
   5     60945  08.90%  09.22%  18.30%   74   Wahoo v3  
   6     59615  09.80%  12.75%  22.53%   73   Polaris 1.6.1  
   7     41745  03.35%  14.76%  32.76%   68   Akimbo 0.4.1  
-------------------------------------------------------------------

[pohl4711]

The only engine ever better than 300000 EAS was OpenTal, which I tested in my testing-environment for developing own settings for the TheKing-engine (to use them in my TheKing Chesscomputer). OpenTal is very weak (Elo 2300), because it plays insane (sacs, which dont make sense). Stronger engines (superhuman (3000 Elo or better)) never reached 300000 EAS points...
In my old SPCC-ratinglist, Velvet 4.1.0 had the highest EAS-score of 282189 points:
1 282189 35.59% 42.40% 14.89% 64 Velvet 4.1.0 avx2

So, if Patricia can hold 300k EAS-Scoring, this would make Patricia the most aggressive engine on a high Elo-level on the planet.
And playing such aggressive chess makes Patricia perfect for human players and their electronic chessboards. But for this, Patricia would need a skill-parameter for playing weaker and an Android-binary (for ChessForAndroid, Droidfish etc).

But for you as an engine-developer, I recommend to look on the single stats more than on the overall EAS-score:
Looking at the numbers of Velvet 4.1.0 (see above), we see, that Velvet 4.1.0 is on the same level with sacs (35%/36%) as Patricia, but Velvet 4.1.0 is better with short wins (42% to 28% (Patricia)) and weaker with the bad draws (14.89% to 7.62%).
So, what we learn here, is: Room for EAS-score-improvement for Patricia is (perhaps) the number of short wins. But nonetheless, Patricias EAS-stats are already really impressive and definitly worth a release - no doubt about that IMO.

[Whiskers]

The only engine ever better than 300000 EAS was OpenTal, which I tested in my testing-environment for developing own settings for the TheKing-engine (to use them in my TheKing Chesscomputer). OpenTal is very weak (Elo 2300), because it plays insane (sacs, which dont make sense). Stronger engines (superhuman (3000 Elo or better)) never reached 300000 EAS points...
In my old SPCC-ratinglist, Velvet 4.1.0 had the highest EAS-score of 282189 points:
1 282189 35.59% 42.40% 14.89% 64 Velvet 4.1.0 avx2

So, if Patricia can hold 300k EAS-Scoring, this would make Patricia the most aggressive engine on a high Elo-level on the planet.
And playing such aggressive chess makes Patricia perfect for human players and their electronic chessboards. But for this, Patricia would need a skill-parameter for playing weaker and an Android-binary (for ChessForAndroid, Droidfish etc).

But for you as an engine-developer, I recommend to look on the single stats more than on the overall EAS-score:
Looking at the numbers of Velvet 4.1.0 (see above), we see, that Velvet 4.1.0 is on the same level with sacs (35%/36%) as Patricia, but Velvet 4.1.0 is better with short wins (42% to 28% (Patricia)) and weaker with the bad draws (14.89% to 7.62%).
So, what we learn here, is: Room for EAS-score-improvement for Patricia is (perhaps) the number of short wins. But nonetheless, Patricias EAS-stats are already really impressive and definitly worth a release - no doubt about that IMO.

I've been thinking about how to make Patricia win faster. The only thing I can think of right now is scale down the evaluations when there's low material, so that she's motivated to keep attacking rather than win a piece and trade into a safely winning endgame. It seems difficult to force wins quickly against strong opponents, though.

[pohl4711]

The only engine ever better than 300000 EAS was OpenTal, which I tested in my testing-environment for developing own settings for the TheKing-engine (to use them in my TheKing Chesscomputer). OpenTal is very weak (Elo 2300), because it plays insane (sacs, which dont make sense). Stronger engines (superhuman (3000 Elo or better)) never reached 300000 EAS points...
In my old SPCC-ratinglist, Velvet 4.1.0 had the highest EAS-score of 282189 points:
1 282189 35.59% 42.40% 14.89% 64 Velvet 4.1.0 avx2

So, if Patricia can hold 300k EAS-Scoring, this would make Patricia the most aggressive engine on a high Elo-level on the planet.
And playing such aggressive chess makes Patricia perfect for human players and their electronic chessboards. But for this, Patricia would need a skill-parameter for playing weaker and an Android-binary (for ChessForAndroid, Droidfish etc).

But for you as an engine-developer, I recommend to look on the single stats more than on the overall EAS-score:
Looking at the numbers of Velvet 4.1.0 (see above), we see, that Velvet 4.1.0 is on the same level with sacs (35%/36%) as Patricia, but Velvet 4.1.0 is better with short wins (42% to 28% (Patricia)) and weaker with the bad draws (14.89% to 7.62%).
So, what we learn here, is: Room for EAS-score-improvement for Patricia is (perhaps) the number of short wins. But nonetheless, Patricias EAS-stats are already really impressive and definitly worth a release - no doubt about that IMO.

I've been thinking about how to make Patricia win faster. The only thing I can think of right now is scale down the evaluations when there's low material, so that she's motivated to keep attacking rather than win a piece and trade into a safely winning endgame. It seems difficult to force wins quickly against strong opponents, though.

I dont say, that it is easy. I just wanted to point out, that the single-stats of the EAS-Tool can give useful hints to engine developers, where to find potential for making the engine getting a better EAS-score.

[Archimedes]

And playing such aggressive chess makes Patricia perfect for human players and their electronic chessboards. But for this, Patricia would need a skill-parameter for playing weaker and an Android-binary (for ChessForAndroid, Droidfish etc).

As far as the last point is concerned, I can help.

Patricia 1.0 for Android:
https://sourceforge.net/projects/chess- ... p/download

Patricia 1.0 for Android (OEX):
https://sourceforge.net/projects/chess- ... k/download

[Whiskers]

I implemented another couple of features into Patricia:
first, I added a significant amount of material scaling; the less pieces there are on the board, the more the eval is scaled down, until endgame position evaluations are cut in half. This will incentivize Patricia to continue attacking when winning instead of settling for trading down into a winning endgame.

Second, I handed out an explicit bonus for attacking the enemy king. I tried this once before and it didn't work, but this time around it was more effective; probably because I now only evaluate the king safety for the non-root side (i.e. the side Patricia is trying to attack).

As a general note, I've found that making a lot of these bonuses side-specific is a great help overall in the engine. By side-specific I mean that it evaluates all aggression metrics from her point of view and not the opponent's; if it's her turn, she gets a bonus in evaluation, but if it's the opponent's turn, they get a penalty for it instead. This (a) makes sure that a +50 aggression bonus from Patricia's point of view is a -50 aggression penalty from the opponent's point of view, which lets the bonuses propagate back from the leaf nodes; and (b) makes it so that Patricia isn't constantly preoccupied with trying to keep her king safe.

Patricia now clocks in at an EAS score of 340k. I'm honestly quite surprised that the sacrifice rate isn't higher and will have to look at some games in order to determine what's going on, because when I watch her play Patricia throws away pawns every single game, usually within the first few moves. She plays the king's gambit against e5 and 2.b4 against the Sicilian.

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    341284  32.69%  36.90%  06.21%   59   Patricia  
   2    118861  14.10%  29.41%  19.05%   65   Peacekeeper 1.50  
   3     85742  11.47%  14.85%  16.49%   70   Wahoo v3  
   4     83649  15.13%  16.48%  18.10%   70   Willow 2.8  
   5     75075  10.63%  16.65%  19.84%   71   Midnight v5  
   6     72457  10.92%  15.08%  20.35%   70   Polaris 1.6.1  
   7     56303  03.62%  18.30%  25.37%   68   Akimbo 0.4.1  
-------------------------------------------------------------------

[pohl4711]

I implemented another couple of features into Patricia:
first, I added a significant amount of material scaling; the less pieces there are on the board, the more the eval is scaled down, until endgame position evaluations are cut in half. This will incentivize Patricia to continue attacking when winning instead of settling for trading down into a winning endgame.

Second, I handed out an explicit bonus for attacking the enemy king. I tried this once before and it didn't work, but this time around it was more effective; probably because I now only evaluate the king safety for the non-root side (i.e. the side Patricia is trying to attack).

As a general note, I've found that making a lot of these bonuses side-specific is a great help overall in the engine. By side-specific I mean that it evaluates all aggression metrics from her point of view and not the opponent's; if it's her turn, she gets a bonus in evaluation, but if it's the opponent's turn, they get a penalty for it instead. This (a) makes sure that a +50 aggression bonus from Patricia's point of view is a -50 aggression penalty from the opponent's point of view, which lets the bonuses propagate back from the leaf nodes; and (b) makes it so that Patricia isn't constantly preoccupied with trying to keep her king safe.

Patricia now clocks in at an EAS score of 340k. I'm honestly quite surprised that the sacrifice rate isn't higher and will have to look at some games in order to determine what's going on, because when I watch her play Patricia throws away pawns every single game, usually within the first few moves. She plays the king's gambit against e5 and 2.b4 against the Sicilian.

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    341284  32.69%  36.90%  06.21%   59   Patricia  
   2    118861  14.10%  29.41%  19.05%   65   Peacekeeper 1.50  
   3     85742  11.47%  14.85%  16.49%   70   Wahoo v3  
   4     83649  15.13%  16.48%  18.10%   70   Willow 2.8  
   5     75075  10.63%  16.65%  19.84%   71   Midnight v5  
   6     72457  10.92%  15.08%  20.35%   70   Polaris 1.6.1  
   7     56303  03.62%  18.30%  25.37%   68   Akimbo 0.4.1  
-------------------------------------------------------------------

This looks really great. Now, the short wins are looking great, too.
The problem with single pawn sacs is, that there must be a sequence of 12 plies without antoher capture, when a pawn is sacrificed (sacrificing more than one pawn, only 8 plies are the limit), because otherwise the one pawn-sac detection, using pgn-extract, delivers too many "false-sac-alarms". But on the other hand, this can mean, that some one pawn sacs are not detected.
But IMO, this EAS-Scoring of your new version looks really fantastic. If you manage to keep the EAS-Score that high and gain some more Elo with the upcoming new Patricia-versions, all is fine!

[mclane]

This patricia plays really interesting.

[Whiskers]

I trained a new neural network on the entire Willow database - 2.6 billion positions. The result is 170 elo stronger than old Patricia, however I need to retrain the nets to increase aggressiveness again and then do more tuning. This takes a while, I have an old CPU with a lot of threads that's very good for playing games, but not so good for training nets...