patricia devlog

[Whiskers]

Result of retrained net:

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    316141  28.18%  35.21%  07.25%   58   patricia-new  
   2    120565  15.97%  29.56%  19.26%   66   Peacekeeper 1.50  
   3     84464  15.28%  14.24%  18.46%   71   Willow 2.8  
   4     82545  11.88%  17.57%  19.70%   72   Midnight v5  
   5     72088  09.98%  12.83%  18.71%   72   Wahoo v3  
   6     63726  12.09%  12.39%  21.27%   73   Polaris 1.6.1  
   7     54245  03.75%  16.95%  26.67%   68   Akimbo 0.4.1  
-------------------------------------------------------------------

Ideally I want this to go up to 400k EAS before I release Patricia 2.0; the sacrifice rate especially needs some work. Also I need to see how this aggressiveness verifies with stronger (3000+) engines.

[BrendanJNorman]

Result of retrained net:

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    316141  28.18%  35.21%  07.25%   58   patricia-new  
   2    120565  15.97%  29.56%  19.26%   66   Peacekeeper 1.50  
   3     84464  15.28%  14.24%  18.46%   71   Willow 2.8  
   4     82545  11.88%  17.57%  19.70%   72   Midnight v5  
   5     72088  09.98%  12.83%  18.71%   72   Wahoo v3  
   6     63726  12.09%  12.39%  21.27%   73   Polaris 1.6.1  
   7     54245  03.75%  16.95%  26.67%   68   Akimbo 0.4.1  
-------------------------------------------------------------------

Ideally I want this to go up to 400k EAS before I release Patricia 2.0; the sacrifice rate especially needs some work. Also I need to see how this aggressiveness verifies with stronger (3000+) engines.

Following your work with great interest. As many others are.

Thanks for giving focus more to style than Elo.

Interesting and weak (er) at 3000 is much more interesting than vanilla 3300.

[carldaman]

Fascinating, as Mr. Spock would say, back in the day...

[Whiskers]

I wrote up an essay only to see it get deleted. Yay.

Some things I did:
- Add material and game length scaling, so that Patricia is motivated to keep pieces on the board and win quickly.
- Add bonuses for directly attacking the enemy king
- Add bonuses for castling opposite side
- Dial up the sacrifice bonuses, but don't scale them higher when Patricia is winning.

After talking with Stephan Pohl, who was very helpful in showing me reasons why some sacrifices weren't being counted, I tried to change the definition of "sacrifices" in Patricia to match the EAS tool's (for example, you have to go 8 plies without any captures after a sacrifice in order for it to be counted), but nothing worked, except for making it so that if it's not legal to take the "sacrificed" piece (e.g. taking advantage of pins), it doesn't count.

Even with the new net, Patricia's suffering a bit when it comes to strength, it's still only about 3000. But I easily have a couple hundred ELO in search left, and the EAS score is now release worthy.

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    408812  34.89%  42.43%  07.24%   55   Patricia  
   2    127632  14.96%  30.29%  19.79%   64   Peacekeeper 1.50  
   3     84339  10.03%  20.18%  20.70%   68   Midnight v5  
   4     83802  14.42%  18.27%  19.44%   70   Willow 2.8  
   5     79605  09.69%  16.47%  19.89%   71   Polaris 1.6.1  
   6     73764  10.89%  16.33%  19.56%   70   Wahoo v3  
   7     54188  03.96%  18.29%  29.38%   67   Akimbo 0.4.1  
-------------------------------------------------------------------

A couple games played by Patricia:

[pgn]
[Event "?"]
[Site "?"]
[Date "2024.03.31"]
[Round "573"]
[White "Patricia"]
[Black "Midnight v5"]
[Result "1-0"]
[TimeControl "10+0.1"]
[SetUp "1"]
[FEN "r1bqkbnr/pp1p1p1p/n3p1p1/2p5/2P5/2N1P3/PP1PBPPP/R1BQK1NR w KQkq - 0 1"]
[PlyCount "89"]
[GameDuration "00:00:26"]
[GameEndTime "2024-03-31T00:28:15.209 CDT"]
[GameStartTime "2024-03-31T00:27:48.932 CDT"]

1. d4 Nf6 2. Nf3 Bg7 3. e4 cxd4 4. Qxd4 Nb4 5. O-O Nc2 6. Qd6 Bf8 7. Qd1
Nxa1 8. e5 Ng8 9. b4 Bxb4 10. Nb5 Ne7 11. Qd6 Bxd6 12. Nxd6+ Kf8 13. Bh6+
Kg8 14. Bd3 Nf5 15. Bxf5 exf5 16. Rc1 f6 17. Re1 Qb6 18. exf6 Qxd6 19. Ng5
Qf8 20. Bxf8 Kxf8 21. Re7 b6 22. Rf7+ Ke8 23. Re7+ Kd8 24. Nf7+ Kc7 25.
Nxh8 Ba6 26. Rxh7 Rf8 27. Nxg6 Rxf6 28. Ne5 Rd6 29. h4 Nc2 30. h5 Na3 31.
Rxd7+ Rxd7 32. Nxd7 Bb5 33. cxb5 Nc4 34. h6 a6 35. bxa6 Nd6 36. Ne5 Nc4 37.
Nxc4 f4 38. h7 f3 39. h8=Q Kd7 40. Qd4+ Ke6 41. Qe4+ Kf6 42. a7 b5 43. a8=Q
Kg7 44. Qf5 bxc4 45. Qaf8# 1-0
[/pgn]

[pgn]
[Event "?"]
[Site "?"]
[Date "2024.03.30"]
[Round "196"]
[White "Wahoo v3"]
[Black "Patricia"]
[Result "0-1"]
[TimeControl "10+0.1"]
[SetUp "1"]
[FEN "rn1qkbnr/pp2pppp/2p5/3p4/5Pb1/PP6/1BPPP1PP/RN1QKBNR w KQkq - 0 1"]
[PlyCount "42"]
[GameDuration "00:00:16"]
[GameEndTime "2024-03-30T23:51:27.553 CDT"]
[GameStartTime "2024-03-30T23:51:11.084 CDT"]

1. h3 Bh5 2. g4 Bg6 3. Nf3 e5 4. fxe5 Nd7 5. Bg2 f6 6. O-O h5 7. Nh4 Bh7 8.
e6 hxg4 9. exd7+ Qxd7 10. hxg4 Qxg4 11. Nf3 Bd6 12. d3 Nh6 13. Nd4 Be4 14.
dxe4 Bh2+ 15. Kh1 Bf4 16. Kg1 Be3+ 17. Rf2 Qg3 18. Qe1 Ng4 19. Kf1 Rh1+ 20.
Bxh1 Nh2+ 21. Rxh2 Qg1# 0-1
[/pgn]

[Whiskers]

An LTC test (60 + 0.6") makes the disparity between Patricia and other engines even more stark. While all the other engines sacrificed less, won slower, and had far more draws, Patricia still maintained an EAS score of 400k.

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    406504  42.51%  37.18%  05.70%   58   Patricia  
   2     98798  13.62%  25.03%  23.72%   68   Peacekeeper 1.50  
   3     85961  17.93%  15.13%  19.13%   70   Willow 2.8  
   4     62120  12.89%  11.81%  22.31%   74   Polaris 1.6.1  
   5     58779  10.74%  11.20%  21.86%   75   Midnight v5  
   6     56248  12.31%  09.23%  22.18%   74   Wahoo v3  
   7     51426  04.92%  14.09%  23.51%   71   Akimbo 0.4.1  
-------------------------------------------------------------------

This is very good news! A couple more search patches to increase strength should be everything I need, and can easily be done with for example Late Move Pruning + Singular Extensions.

[Whiskers]

New baseline pool for Patricia. These engines are about equal strength or stronger, with the exception of poor Wahoo (I didn't expect it to perform that poorly!)

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    290153  31.42%  29.50%  08.47%   61   Patricia  
   2    126973  16.04%  22.23%  15.86%   64   Svart 6  
   3    116043  09.46%  31.21%  24.78%   60   Wahoo v4  
   4    110608  12.72%  26.20%  20.54%   64   Leorik 3.0.1  
   5    109185  13.46%  28.64%  21.11%   65   Velvet 3.1  
   6     79544  08.61%  18.72%  25.89%   68   StockNemo 5.7  
   7     52576  09.23%  08.87%  21.35%   83   Princhess 0.16  
-------------------------------------------------------------------

This is a very odd list to me, Velvet and Princhess so low? how? Princhess is one of the best style playing engines I've seen. Its draw rate on CCRL, at 31%, is lower than every single engine rated higher than 2865. And Velvet is, well, Velvet.

Patricia's no longer able to wipe opponents out at the speed of light; as most of its sacrifices are at least somewhat unsound, it often wins despite the sacrifices rather than because of them against these stronger opponents. However, the EAS score is still way higher than any other engine in the gauntlet, so I am satisfied with her performance and will start development of 2.1/3.0 using this pool.

I really can't help but shake the feeling that optimizing for a particular pool of engines isn't the best idea... maybe I should test using a gauntlet against more (like 20) engines?

[Rebel]

That's a super result, congrats.

[Guenther]

New baseline pool for Patricia. These engines are about equal strength or stronger, with the exception of poor Wahoo (I didn't expect it to perform that poorly!)

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    290153  31.42%  29.50%  08.47%   61   Patricia  
   2    126973  16.04%  22.23%  15.86%   64   Svart 6  
   3    116043  09.46%  31.21%  24.78%   60   Wahoo v4  
   4    110608  12.72%  26.20%  20.54%   64   Leorik 3.0.1  
   5    109185  13.46%  28.64%  21.11%   65   Velvet 3.1  
   6     79544  08.61%  18.72%  25.89%   68   StockNemo 5.7  
   7     52576  09.23%  08.87%  21.35%   83   Princhess 0.16  
-------------------------------------------------------------------

This is a very odd list to me, Velvet and Princhess so low? how? Princhess is one of the best style playing engines I've seen. Its draw rate on CCRL, at 31%, is lower than every single engine rated higher than 2865. And Velvet is, well, Velvet.

Patricia's no longer able to wipe opponents out at the speed of light; as most of its sacrifices are at least somewhat unsound, it often wins despite the sacrifices rather than because of them against these stronger opponents. However, the EAS score is still way higher than any other engine in the gauntlet, so I am satisfied with her performance and will start development of 2.1/3.0 using this pool.

I really can't help but shake the feeling that optimizing for a particular pool of engines isn't the best idea... maybe I should test using a gauntlet against more (like 20) engines?

Would you mind also to show the real scores for comparison?

[Rebel]

I really can't help but shake the feeling that optimizing for a particular pool of engines isn't the best idea... maybe I should test using a gauntlet against more (like 20) engines?

I always use 10 engines.

[Whiskers]

New baseline pool for Patricia. These engines are about equal strength or stronger, with the exception of poor Wahoo (I didn't expect it to perform that poorly!)

Code: Select all

Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    290153  31.42%  29.50%  08.47%   61   Patricia  
   2    126973  16.04%  22.23%  15.86%   64   Svart 6  
   3    116043  09.46%  31.21%  24.78%   60   Wahoo v4  
   4    110608  12.72%  26.20%  20.54%   64   Leorik 3.0.1  
   5    109185  13.46%  28.64%  21.11%   65   Velvet 3.1  
   6     79544  08.61%  18.72%  25.89%   68   StockNemo 5.7  
   7     52576  09.23%  08.87%  21.35%   83   Princhess 0.16  
-------------------------------------------------------------------

This is a very odd list to me, Velvet and Princhess so low? how? Princhess is one of the best style playing engines I've seen. Its draw rate on CCRL, at 31%, is lower than every single engine rated higher than 2865. And Velvet is, well, Velvet.

Patricia's no longer able to wipe opponents out at the speed of light; as most of its sacrifices are at least somewhat unsound, it often wins despite the sacrifices rather than because of them against these stronger opponents. However, the EAS score is still way higher than any other engine in the gauntlet, so I am satisfied with her performance and will start development of 2.1/3.0 using this pool.

I really can't help but shake the feeling that optimizing for a particular pool of engines isn't the best idea... maybe I should test using a gauntlet against more (like 20) engines?

Would you mind also to show the real scores for comparison?

ELO scores you mean? Sure, I’ll do it as soon as I get home. Patricia is pretty much right in the middle strength wise.