Training the trainer: how is it done for Stockfish?

[syzygy]

When these positions arise in the middle game, Leela will often miss the correct continuation, just like it does in the endgame. The question then becomes "are positions that require extremely precise play equally likely to arise in the middle game as they are in the endgame?" I believe the answer to that is "no". I think that early in a chess game, there tend to be many workable strategies that can be followed, so positions requiring a single continuation of perfect play are not common. As you get closer to the endgame, more situations arise where the wrong move can make the difference between a draw and a loss. e.g. if you are one move too late moving your king toward an enemy pawn, you will lose a pawn race, and be down a queen.

And I guess the number of workable strategies when playing a classical engine is also a function of how "perfectly" the classical engine can play the position. In the endgame, a classical engine can often play perfectly, punishing any subtle error its opponent makes. In the middle game, a classical engine cannot fully "solve" the position and is likely to make subtle mistakes itself.

Where NNs could get into trouble are middle game positions with a relatively deep tactic that a classical engine can spot but an NN-based engine not. LC0's opening play is apparently good enough to usually avoid positions with such tactical traps.

[JollyJoker]

Comparing n-man to n+1 man, does n+1 man have a higher share of moves that don't worsen the outcome? Assuming we're considering that as the definition of a mistake, rather than having an opponent that can actually exploit the mistake.

[syzygy]

Comparing n-man to n+1 man, does n+1 man have a higher share of moves that don't worsen the outcome? Assuming we're considering that as the definition of a mistake, rather than having an opponent that can actually exploit the mistake.

I would say: compared with positions with n men, positions with n+1 men have a higher share of moves that aren't reliably evaluated by a classical engine (given the same time per move).

NNs are good at fuzzy things. With more pieces left on the board, chess is more fuzzy for classical engines, so NNs have better chances to outplay classical engines.