lucasart wrote:Interesting analysis. But conclusions based on Crafty cannot be considered universal. In my engine lazy eval never worked. In Stockfish it never worked either.
With so few errors in case 2, it suggests to me that Crafty's piece evaluation doesn't have enough bandwidth. 1p or 2p is a very narrow band. From the description it is not clear if king safety is supposed to fall in it (I suppose not), and the same for mobility/board control (I suppose it is, but then 2p is certainly not enough). The way I see it: NPS is the wrong focus if the evaluation doesn't have room to breathe. It just results in bigger trees.
Or it results in more wins? Remember, those values we use were produced by cluster testing. Any changes to them lower the Elo, so it is a firm "peak" for current Crafty anyway.
I don't quite follow the 2p comment for mobility and such. Those scores certainly do NOT add up to anywhere near 1-2p in Crafty, unless one side has all pieces and the other side has none, where the evaluation is already skewed wildly by the material advantage.
mvk wrote:The way I see it: NPS is the wrong focus if the evaluation doesn't have room to breathe. It just results in bigger trees.
Or it results in more wins? Remember, those values we use were produced by cluster testing. Any changes to them lower the Elo, so it is a firm "peak" for current Crafty anyway.
mvk wrote:The way I see it: NPS is the wrong focus if the evaluation doesn't have room to breathe. It just results in bigger trees.
Or it results in more wins? Remember, those values we use were produced by cluster testing. Any changes to them lower the Elo, so it is a firm "peak" for current Crafty anyway.
Is lazy eval off while tuning?
No. Unless I am adding some pretty large new eval term. In that case I just significantly increase the lazy eval margins, and run the original and new version with the same margins to compare apples to apples for the new term. I don't do very many "large" changes however, so this is not a very frequent thing.