Wow! This is an awesome EAS-Score!!! And all stats look great (sacs, shorts, bad draws..), not only the overall score. I am really impressed.Whiskers wrote: ↑Mon Mar 04, 2024 2:12 am
The answer.... was yes.
First attempt.
Code: Select all
Rank EAS-Score sacs shorts draws moves Engine/player ------------------------------------------------------------------- 1 301361 29.84% 27.40% 07.52% 62 Patricia 2 127198 14.31% 30.03% 19.48% 64 Peacekeeper 1.50 3 100220 15.11% 18.40% 17.00% 68 Willow 2.8 4 99916 10.18% 18.71% 15.49% 70 Midnight v5 5 80646 11.02% 15.92% 17.82% 71 Wahoo v3 6 73076 09.62% 16.47% 20.34% 70 Polaris 1.6.1 7 54251 03.20% 20.28% 29.45% 66 Akimbo 0.4.1 -------------------------------------------------------------------
patricia devlog
Moderators: hgm, Rebel, chrisw
-
- Posts: 2557
- Joined: Sat Sep 03, 2011 7:25 am
- Location: Berlin, Germany
- Full name: Stefan Pohl
Re: patricia devlog
-
- Posts: 216
- Joined: Tue Jan 31, 2023 4:34 pm
- Full name: Adam Kulju
Re: patricia devlog
I ran a test at a longer time control, results were pretty similar. Patricia's a menace. How long until the next release? Is 400k EAS possible?
Code: Select all
Rank EAS-Score sacs shorts draws moves Engine/player
-------------------------------------------------------------------
1 307560 36.08% 27.83% 07.62% 62 Patricia
2 102447 13.83% 24.85% 22.25% 68 Peacekeeper 1.50
3 84922 15.35% 14.11% 18.89% 71 Willow 2.8
4 74646 12.96% 13.76% 20.91% 70 Midnight v5
5 60945 08.90% 09.22% 18.30% 74 Wahoo v3
6 59615 09.80% 12.75% 22.53% 73 Polaris 1.6.1
7 41745 03.35% 14.76% 32.76% 68 Akimbo 0.4.1
-------------------------------------------------------------------
go and star https://github.com/Adam-Kulju/Patricia!
-
- Posts: 2557
- Joined: Sat Sep 03, 2011 7:25 am
- Location: Berlin, Germany
- Full name: Stefan Pohl
Re: patricia devlog
The only engine ever better than 300000 EAS was OpenTal, which I tested in my testing-environment for developing own settings for the TheKing-engine (to use them in my TheKing Chesscomputer). OpenTal is very weak (Elo 2300), because it plays insane (sacs, which dont make sense). Stronger engines (superhuman (3000 Elo or better)) never reached 300000 EAS points...
In my old SPCC-ratinglist, Velvet 4.1.0 had the highest EAS-score of 282189 points:
1 282189 35.59% 42.40% 14.89% 64 Velvet 4.1.0 avx2
So, if Patricia can hold 300k EAS-Scoring, this would make Patricia the most aggressive engine on a high Elo-level on the planet.
And playing such aggressive chess makes Patricia perfect for human players and their electronic chessboards. But for this, Patricia would need a skill-parameter for playing weaker and an Android-binary (for ChessForAndroid, Droidfish etc).
But for you as an engine-developer, I recommend to look on the single stats more than on the overall EAS-score:
Looking at the numbers of Velvet 4.1.0 (see above), we see, that Velvet 4.1.0 is on the same level with sacs (35%/36%) as Patricia, but Velvet 4.1.0 is better with short wins (42% to 28% (Patricia)) and weaker with the bad draws (14.89% to 7.62%).
So, what we learn here, is: Room for EAS-score-improvement for Patricia is (perhaps) the number of short wins. But nonetheless, Patricias EAS-stats are already really impressive and definitly worth a release - no doubt about that IMO.
In my old SPCC-ratinglist, Velvet 4.1.0 had the highest EAS-score of 282189 points:
1 282189 35.59% 42.40% 14.89% 64 Velvet 4.1.0 avx2
So, if Patricia can hold 300k EAS-Scoring, this would make Patricia the most aggressive engine on a high Elo-level on the planet.
And playing such aggressive chess makes Patricia perfect for human players and their electronic chessboards. But for this, Patricia would need a skill-parameter for playing weaker and an Android-binary (for ChessForAndroid, Droidfish etc).
But for you as an engine-developer, I recommend to look on the single stats more than on the overall EAS-score:
Looking at the numbers of Velvet 4.1.0 (see above), we see, that Velvet 4.1.0 is on the same level with sacs (35%/36%) as Patricia, but Velvet 4.1.0 is better with short wins (42% to 28% (Patricia)) and weaker with the bad draws (14.89% to 7.62%).
So, what we learn here, is: Room for EAS-score-improvement for Patricia is (perhaps) the number of short wins. But nonetheless, Patricias EAS-stats are already really impressive and definitly worth a release - no doubt about that IMO.
-
- Posts: 216
- Joined: Tue Jan 31, 2023 4:34 pm
- Full name: Adam Kulju
Re: patricia devlog
I've been thinking about how to make Patricia win faster. The only thing I can think of right now is scale down the evaluations when there's low material, so that she's motivated to keep attacking rather than win a piece and trade into a safely winning endgame. It seems difficult to force wins quickly against strong opponents, though.pohl4711 wrote: ↑Tue Mar 05, 2024 7:08 am The only engine ever better than 300000 EAS was OpenTal, which I tested in my testing-environment for developing own settings for the TheKing-engine (to use them in my TheKing Chesscomputer). OpenTal is very weak (Elo 2300), because it plays insane (sacs, which dont make sense). Stronger engines (superhuman (3000 Elo or better)) never reached 300000 EAS points...
In my old SPCC-ratinglist, Velvet 4.1.0 had the highest EAS-score of 282189 points:
1 282189 35.59% 42.40% 14.89% 64 Velvet 4.1.0 avx2
So, if Patricia can hold 300k EAS-Scoring, this would make Patricia the most aggressive engine on a high Elo-level on the planet.
And playing such aggressive chess makes Patricia perfect for human players and their electronic chessboards. But for this, Patricia would need a skill-parameter for playing weaker and an Android-binary (for ChessForAndroid, Droidfish etc).
But for you as an engine-developer, I recommend to look on the single stats more than on the overall EAS-score:
Looking at the numbers of Velvet 4.1.0 (see above), we see, that Velvet 4.1.0 is on the same level with sacs (35%/36%) as Patricia, but Velvet 4.1.0 is better with short wins (42% to 28% (Patricia)) and weaker with the bad draws (14.89% to 7.62%).
So, what we learn here, is: Room for EAS-score-improvement for Patricia is (perhaps) the number of short wins. But nonetheless, Patricias EAS-stats are already really impressive and definitly worth a release - no doubt about that IMO.
go and star https://github.com/Adam-Kulju/Patricia!
-
- Posts: 2557
- Joined: Sat Sep 03, 2011 7:25 am
- Location: Berlin, Germany
- Full name: Stefan Pohl
Re: patricia devlog
I dont say, that it is easy. I just wanted to point out, that the single-stats of the EAS-Tool can give useful hints to engine developers, where to find potential for making the engine getting a better EAS-score.Whiskers wrote: ↑Tue Mar 05, 2024 7:13 pmI've been thinking about how to make Patricia win faster. The only thing I can think of right now is scale down the evaluations when there's low material, so that she's motivated to keep attacking rather than win a piece and trade into a safely winning endgame. It seems difficult to force wins quickly against strong opponents, though.pohl4711 wrote: ↑Tue Mar 05, 2024 7:08 am The only engine ever better than 300000 EAS was OpenTal, which I tested in my testing-environment for developing own settings for the TheKing-engine (to use them in my TheKing Chesscomputer). OpenTal is very weak (Elo 2300), because it plays insane (sacs, which dont make sense). Stronger engines (superhuman (3000 Elo or better)) never reached 300000 EAS points...
In my old SPCC-ratinglist, Velvet 4.1.0 had the highest EAS-score of 282189 points:
1 282189 35.59% 42.40% 14.89% 64 Velvet 4.1.0 avx2
So, if Patricia can hold 300k EAS-Scoring, this would make Patricia the most aggressive engine on a high Elo-level on the planet.
And playing such aggressive chess makes Patricia perfect for human players and their electronic chessboards. But for this, Patricia would need a skill-parameter for playing weaker and an Android-binary (for ChessForAndroid, Droidfish etc).
But for you as an engine-developer, I recommend to look on the single stats more than on the overall EAS-score:
Looking at the numbers of Velvet 4.1.0 (see above), we see, that Velvet 4.1.0 is on the same level with sacs (35%/36%) as Patricia, but Velvet 4.1.0 is better with short wins (42% to 28% (Patricia)) and weaker with the bad draws (14.89% to 7.62%).
So, what we learn here, is: Room for EAS-score-improvement for Patricia is (perhaps) the number of short wins. But nonetheless, Patricias EAS-stats are already really impressive and definitly worth a release - no doubt about that IMO.
-
- Posts: 157
- Joined: Tue Mar 05, 2019 3:43 pm
- Full name: Archimedes
Re: patricia devlog
As far as the last point is concerned, I can help.
Patricia 1.0 for Android:
https://sourceforge.net/projects/chess- ... p/download
Patricia 1.0 for Android (OEX):
https://sourceforge.net/projects/chess- ... k/download
-
- Posts: 216
- Joined: Tue Jan 31, 2023 4:34 pm
- Full name: Adam Kulju
Re: patricia devlog
I implemented another couple of features into Patricia:
first, I added a significant amount of material scaling; the less pieces there are on the board, the more the eval is scaled down, until endgame position evaluations are cut in half. This will incentivize Patricia to continue attacking when winning instead of settling for trading down into a winning endgame.
Second, I handed out an explicit bonus for attacking the enemy king. I tried this once before and it didn't work, but this time around it was more effective; probably because I now only evaluate the king safety for the non-root side (i.e. the side Patricia is trying to attack).
As a general note, I've found that making a lot of these bonuses side-specific is a great help overall in the engine. By side-specific I mean that it evaluates all aggression metrics from her point of view and not the opponent's; if it's her turn, she gets a bonus in evaluation, but if it's the opponent's turn, they get a penalty for it instead. This (a) makes sure that a +50 aggression bonus from Patricia's point of view is a -50 aggression penalty from the opponent's point of view, which lets the bonuses propagate back from the leaf nodes; and (b) makes it so that Patricia isn't constantly preoccupied with trying to keep her king safe.
Patricia now clocks in at an EAS score of 340k. I'm honestly quite surprised that the sacrifice rate isn't higher and will have to look at some games in order to determine what's going on, because when I watch her play Patricia throws away pawns every single game, usually within the first few moves. She plays the king's gambit against e5 and 2.b4 against the Sicilian.
first, I added a significant amount of material scaling; the less pieces there are on the board, the more the eval is scaled down, until endgame position evaluations are cut in half. This will incentivize Patricia to continue attacking when winning instead of settling for trading down into a winning endgame.
Second, I handed out an explicit bonus for attacking the enemy king. I tried this once before and it didn't work, but this time around it was more effective; probably because I now only evaluate the king safety for the non-root side (i.e. the side Patricia is trying to attack).
As a general note, I've found that making a lot of these bonuses side-specific is a great help overall in the engine. By side-specific I mean that it evaluates all aggression metrics from her point of view and not the opponent's; if it's her turn, she gets a bonus in evaluation, but if it's the opponent's turn, they get a penalty for it instead. This (a) makes sure that a +50 aggression bonus from Patricia's point of view is a -50 aggression penalty from the opponent's point of view, which lets the bonuses propagate back from the leaf nodes; and (b) makes it so that Patricia isn't constantly preoccupied with trying to keep her king safe.
Patricia now clocks in at an EAS score of 340k. I'm honestly quite surprised that the sacrifice rate isn't higher and will have to look at some games in order to determine what's going on, because when I watch her play Patricia throws away pawns every single game, usually within the first few moves. She plays the king's gambit against e5 and 2.b4 against the Sicilian.
Code: Select all
Rank EAS-Score sacs shorts draws moves Engine/player
-------------------------------------------------------------------
1 341284 32.69% 36.90% 06.21% 59 Patricia
2 118861 14.10% 29.41% 19.05% 65 Peacekeeper 1.50
3 85742 11.47% 14.85% 16.49% 70 Wahoo v3
4 83649 15.13% 16.48% 18.10% 70 Willow 2.8
5 75075 10.63% 16.65% 19.84% 71 Midnight v5
6 72457 10.92% 15.08% 20.35% 70 Polaris 1.6.1
7 56303 03.62% 18.30% 25.37% 68 Akimbo 0.4.1
-------------------------------------------------------------------
go and star https://github.com/Adam-Kulju/Patricia!
-
- Posts: 2557
- Joined: Sat Sep 03, 2011 7:25 am
- Location: Berlin, Germany
- Full name: Stefan Pohl
Re: patricia devlog
This looks really great. Now, the short wins are looking great, too.Whiskers wrote: ↑Fri Mar 08, 2024 7:52 am I implemented another couple of features into Patricia:
first, I added a significant amount of material scaling; the less pieces there are on the board, the more the eval is scaled down, until endgame position evaluations are cut in half. This will incentivize Patricia to continue attacking when winning instead of settling for trading down into a winning endgame.
Second, I handed out an explicit bonus for attacking the enemy king. I tried this once before and it didn't work, but this time around it was more effective; probably because I now only evaluate the king safety for the non-root side (i.e. the side Patricia is trying to attack).
As a general note, I've found that making a lot of these bonuses side-specific is a great help overall in the engine. By side-specific I mean that it evaluates all aggression metrics from her point of view and not the opponent's; if it's her turn, she gets a bonus in evaluation, but if it's the opponent's turn, they get a penalty for it instead. This (a) makes sure that a +50 aggression bonus from Patricia's point of view is a -50 aggression penalty from the opponent's point of view, which lets the bonuses propagate back from the leaf nodes; and (b) makes it so that Patricia isn't constantly preoccupied with trying to keep her king safe.
Patricia now clocks in at an EAS score of 340k. I'm honestly quite surprised that the sacrifice rate isn't higher and will have to look at some games in order to determine what's going on, because when I watch her play Patricia throws away pawns every single game, usually within the first few moves. She plays the king's gambit against e5 and 2.b4 against the Sicilian.
Code: Select all
Rank EAS-Score sacs shorts draws moves Engine/player ------------------------------------------------------------------- 1 341284 32.69% 36.90% 06.21% 59 Patricia 2 118861 14.10% 29.41% 19.05% 65 Peacekeeper 1.50 3 85742 11.47% 14.85% 16.49% 70 Wahoo v3 4 83649 15.13% 16.48% 18.10% 70 Willow 2.8 5 75075 10.63% 16.65% 19.84% 71 Midnight v5 6 72457 10.92% 15.08% 20.35% 70 Polaris 1.6.1 7 56303 03.62% 18.30% 25.37% 68 Akimbo 0.4.1 -------------------------------------------------------------------
The problem with single pawn sacs is, that there must be a sequence of 12 plies without antoher capture, when a pawn is sacrificed (sacrificing more than one pawn, only 8 plies are the limit), because otherwise the one pawn-sac detection, using pgn-extract, delivers too many "false-sac-alarms". But on the other hand, this can mean, that some one pawn sacs are not detected.
But IMO, this EAS-Scoring of your new version looks really fantastic. If you manage to keep the EAS-Score that high and gain some more Elo with the upcoming new Patricia-versions, all is fine!
-
- Posts: 18783
- Joined: Thu Mar 09, 2006 6:40 pm
- Location: US of Europe, germany
- Full name: Thorsten Czub
Re: patricia devlog
This patricia plays really interesting.
What seems like a fairy tale today may be reality tomorrow.
Here we have a fairy tale of the day after tomorrow....
Here we have a fairy tale of the day after tomorrow....
-
- Posts: 216
- Joined: Tue Jan 31, 2023 4:34 pm
- Full name: Adam Kulju
Re: patricia devlog
I trained a new neural network on the entire Willow database - 2.6 billion positions. The result is 170 elo stronger than old Patricia, however I need to retrain the nets to increase aggressiveness again and then do more tuning. This takes a while, I have an old CPU with a lot of threads that's very good for playing games, but not so good for training nets...
go and star https://github.com/Adam-Kulju/Patricia!