jorose wrote: ↑Tue Jul 16, 2024 11:56 am
Great work and congratulations on the new release!
I was just thinking, imagine we could teleport a Patricia binary 15 years into the past. People would be able to verify it is the strongest engine in existence, and could conclude that Patricia's playstyle is "optimal chess"
I just ran an 8 game match between Patricia 3 and Crafty 25.6, at 5m+3s, which Patricia 3 won 6-2. I felt kind of sad for Crafty, though. But I was happy to see Crafty escape a lost position with a stalemate tactic. Crafty is not bad.
In one game that Patricia lost, it took unreasonable time on one move. It didn't lose on time, but played with significant time deficit afterwards. I am not sure if it was cutechess-gui issue or some other thing that the developer needs to investigate.
chesskobra wrote: ↑Tue Jul 16, 2024 1:36 pm
In one game that Patricia lost, it took unreasonable time on one move. It didn't lose on time, but played with significant time deficit afterwards. I am not sure if it was cutechess-gui issue or some other thing that the developer needs to investigate.
It is an engine-problem. I already mentioned this to the author. There is a problem in the panic-time algorithm of Patricia. This will be fixed in later releases.
Werewolf wrote: ↑Tue Jul 16, 2024 12:12 pm
I'm a huge fan of this work, and I hate to be a party-pooper, and I know my observations probably don't align with EAS etc. however, from practical observation now of dozens of games Patricia 3 does seem quite a bit more conservative than Patricia 2, especially in the opening (although far better at finishing an opponent off).
What the author says:
"The main thing that separates 3.0 from 2.0 is that most of her aggressiveness now comes from the neural network evaluation instead of from search twiddling. This means that Patricia now fully believes in all of her sacrifices, and wins because of them rather than in spite of them."
So, Patricia 3 tries to sac only, when this makes sense. So, IMHO, it makes sense, that P3 does not play such aggressive (but strange) opening-moves / sacs.
When my tests are finished, we will see clearly, if P2 or P3 plays more aggressive. But I need some more days to finish...
But IMHO, an aggressive play coming from the neuralnet is much better, than coming from search-algorithms, because the sacs make (hopefully) more sense, because the neuralnet understands the position and the search-algorithm does not.
Werewolf wrote: ↑Tue Jul 16, 2024 12:12 pm
I'm a huge fan of this work, and I hate to be a party-pooper, and I know my observations probably don't align with EAS etc. however, from practical observation now of dozens of games Patricia 3 does seem quite a bit more conservative than Patricia 2, especially in the opening (although far better at finishing an opponent off).
What the author says:
"The main thing that separates 3.0 from 2.0 is that most of her aggressiveness now comes from the neural network evaluation instead of from search twiddling. This means that Patricia now fully believes in all of her sacrifices, and wins because of them rather than in spite of them."
So, Patricia 3 tries to sac only, when this makes sense. So, IMHO, it makes sense, that P3 does not play such aggressive (but strange) opening-moves / sacs.
When my tests are finished, we will see clearly, if P2 or P3 plays more aggressive. But I need some more days to finish...
But IMHO, an aggressive play coming from the neuralnet is much better, than coming from search-algorithms, because the sacs make (hopefully) more sense, because the neuralnet understands the position and the search-algorithm does not.
Noted, but do bear in mind the ancients all thought the King's Gambit and the Evans Gambit and the From Gambit made sense and were not "strange", but rather logical. They may not be optimal if the aim is best play, but if the aim is aggression...
While in principle I prefer the aggression to come from the network, I feel one should be cautious when attributing an understanding or lack thereof based on sacrifices coming from the search or evaluation. I think as humans we tend to attribute things to evaluation, because it feels easier for us to understand. That is to say I think we know on some level that we cannot grasp exactly the behavior of search trees whereas evaluation seems easier.
One could equally argue Patricia plays overly aggressive now due to a lack of understanding from its network as we could argue it comes from understanding from its network. I think all we can say for certain is the network is tuned in a way that results in aggressive play, which is the intention.
I think it is great Adam has demonstrated how you can adjust the engine's playstyle based on a clever way of filtering the data. I really think the data side of model training is still quite poorly understood, and I find this work inspiring.
Werewolf wrote: ↑Tue Jul 16, 2024 12:12 pm
I'm a huge fan of this work, and I hate to be a party-pooper, and I know my observations probably don't align with EAS etc. however, from practical observation now of dozens of games Patricia 3 does seem quite a bit more conservative than Patricia 2, especially in the opening (although far better at finishing an opponent off).
The following were all done with light kibitzing (around 30 seconds per position) with threads = 2 for Patricia 3. Testing for aggression was my desire.
Hi there and thank you for the concern. While I love coffeehouse gambits such as the King's Gambit and the Wing Gambit, stronger engines can easily defend it, leaving them simply better for no compensation after a while.
Patricia's network is trained off of Willow games; as Willow is 3500, unsound sacrifices tend to be punished in the training data, while sound sacrifices or extremely hard to defend sacrifices are rewarded. Patricia learned from this and is much more likely to play "good sacrifices" and angle for them several moves in advance; on the flip side, she won't play sacrifices that she thinks are easily refuted, even with the minor amount of search adjustment I still have.
Perhaps I could enable the algorithm that forces sacrifices whenever Patricia's strength is being limited by a UCI option? That would make her more fun to play against as a human, while still letting her go full-power against an engine.
Hi there and thank you for the concern. While I love coffeehouse gambits such as the King's Gambit and the Wing Gambit, stronger engines can easily defend it, leaving them simply better for no compensation after a while.
Patricia's network is trained off of Willow games; as Willow is 3500, unsound sacrifices tend to be punished in the training data, while sound sacrifices or extremely hard to defend sacrifices are rewarded. Patricia learned from this and is much more likely to play "good sacrifices" and angle for them several moves in advance; on the flip side, she won't play sacrifices that she thinks are easily refuted, even with the minor amount of search adjustment I still have.
Perhaps I could enable the algorithm that forces sacrifices whenever Patricia's strength is being limited by a UCI option? That would make her more fun to play against as a human, while still letting her go full-power against an engine.
Thanks for your explanation.
I wonder if the target market to hit is humans...going right up Magnus. That would make the engine incredibly useful for over-the-board prep, knowing that a "Patricia gambit" is unlikely to be refuted by any human, even a titled player.
Whiskers wrote: ↑Tue Jul 16, 2024 6:21 pm
Hi there and thank you for the concern. While I love coffeehouse gambits such as the King's Gambit and the Wing Gambit, stronger engines can easily defend it, leaving them simply better for no compensation after a while.
Patricia's network is trained off of Willow games; as Willow is 3500, unsound sacrifices tend to be punished in the training data, while sound sacrifices or extremely hard to defend sacrifices are rewarded. Patricia learned from this and is much more likely to play "good sacrifices" and angle for them several moves in advance; on the flip side, she won't play sacrifices that she thinks are easily refuted, even with the minor amount of search adjustment I still have.
Perhaps I could enable the algorithm that forces sacrifices whenever Patricia's strength is being limited by a UCI option? That would make her more fun to play against as a human, while still letting her go full-power against an engine.
I like the idea, but I recommend to make an extra (switch "Crazyness" on/off) engine-option, because IMHO a human player could like the "normal" Patricia playing-style and does not want to switch it off, everytime the Elo-Strength is limited...
Another idea comes from OpenTal: This engine has all of Tals openings hardcoded and stored in the engine. And starts thinking, only, when this hardcoded opening-book has no more moves.
All (normal and crazy) gambits on this planet could be easily hardcoded and stored in Particias code (there are not so many and so deep gambit-lines). Then, Patricia will always play a gambit, when there is no external openingline or openingbook in use. And for Ratinglist-testruns etc. this would not cause any "Elo-damage", because in these testruns, external opening-lines are used.
I just made an opening.pgn with all 3 old Noomen Gambit.pgns. I used pgn-extract to delete all lines with double endpositions, so all endpositions are unique, download from my GoogleDrive:
On opening books, this won't achieve the same aim. An opening book forces an engine to play a move, regardless of whether the engine itself would do so without the book.
From a human OTB preparation point of view the golden goose is to have an engine which WANTS to play aggressive, sacrificial chess which is semi-sound. I would not call this "crazy", but it is pro-risk.
When the engine ITSELF wants to play like this - without a book telling it to - it allows for opening research which is fruitful because the human analysist can use it to find new gambits in unexplored lines, having faith that if the engine likes the King's Gambit, then it will find other gambits of similar quality elsewhere...
