Hi... I'm working on engine research

[FireDragon761138]

Training a network for exactly reproducing win probability in principle desroys the possibility for the engine to convert a non-trivial win. E.g. a KBNK mate typically takes some 60 ply, but, as long as no B or N is blundered away, is always a certain win. There is no way a search would reach 60 ply if it has no guidance for what to prune; without pruning you must be happy to reach 8 ply. And there is no guidance, neither for prunig as for striving for intermediate goals (like getting the bare King into the corner), as all non-sacrificial end-leaves will have an identical 100% score. This reduces the engine to a random mover biased against giving away any material. No way its random walk through state space will ever bring a checkmate within the horizon. At some point it will stumble into the 50-move barrier, and there is nothing it can be done to avoid it.

A corrolary is that when an engine is trained to reproduce the true win rate on the games played by the latest version, it can never achieve the exact win rate. When it is still ignorant it will discover that it will have a higher chance of winning in positions where the mate is close, just because it has a larger chance to stumble on a position that has it within the search horizon. Those will then get higher score, which can be seen by the search from further away, so that the win rate there will also improve. But it has to keep a score gradient large enough to guide the play towards the mate from far away. And there is some minimum gradient for which this still works, because there will always be evaluation noise. So you will get some equilibrium, where positions far away from any mate only get, say, 90% win rate. Because that will indeed be their win rate, because the gradient from 90% to 100% is spread over so many moves that it has become so small that it fails to follow it in 10% of the cases.

Are you saying an engine can have precise, accurate analysis, or dominating play strength, but not really both at once?

The enemy's king is hiding in a desert with lots of high dunes obscuring vision at ground level. Two different companies of soldiers will attempt to kill him:

1. Stockfish (SF)

2. Quick checkmater (QC)

SF parachute into the middle of the desert, quickly getting close. Now they're there, though, they don't know which direction to travel in: they just have to guess, try a direction, and hope to uncover some clues.

QC start their assault from the edge of the desert, carefully charting their position as they go. This will lead them directly to the king - but it's too obvious and predictable, so they will get ambushed and killed.

Unbelievable BS. Chess is not life nor war, both of which are massively complex, unbounded and anything can happen. They are neither two sided, take it in turns to move operating on a tiny space of 64 squares with pieces that can move in six ways only. The "chess is life" concept is and always was a nonsense fantasy. Tal included.

Simple game theory concepts don't necessarily capture the complexity of chess. Chess may have nonlinear, emergent properties that only arise at different scales. Seeing chess as only 64 squares and 16 pieces is just one way to look at the resulting game.

[FireDragon761138]

Some mistakes of stockfish in my queen handicap games

[d]rn1qkbnr/ppp1pppp/8/3p1bB1/3P4/8/PPP1PPPP/RN2KBNR w KQkq - 2 3 am Nc3 bm c3

Latest stockfish blundered a pawn by Nc3 when c3 is correct.

[d]rn1qkb1r/ppp1pppp/5nb1/3p2B1/3P3P/2N5/PP2PPP1/2R1KBNR w Kkq - 1 6 am h5

[d]rn1qkb1r/pp2pppp/2p3b1/3p2Bn/3P4/2N2N2/PP2PPP1/2R1KB1R w Kkq - 0 8 am e4

[d]r6r/pp2k1pp/4pnp1/Pq5n/4p3/4P3/1P4P1/3R1RK1 w - - 0 20

Latest stockfish played Rfe1 when it is possible to delay mate by one more moves by a pawn move Rd2 Rd4 Rf2 Rde1,Ra1,Rb1,Rc1

The latest model in the engine I am working on selects the correct moves in all the FEN positions (in the first example, c4 is chosen immediately as the best move when using the kibbitzer in LucasChess), so it seems like my philosophical approach is validated, at least in the examples you give. Stockfish isn't giving an "objective" evaluation based on the board state, but optimizations for winning against other engines, most of which either have only pragmatic design parameters (similar to Stockfish), or lack the kind of community support for optimizations to the code.

I've been trying to nail down the best epoch to use the NNUE, running different tournaments against each other, and refining it till I get down to the best epoch. The older epochs don't do as good a job selecting the optimum moves, whereas the latest, best epoch (87) sees the moves immediately.

[towforce]

Training a network for exactly reproducing win probability in principle desroys the possibility for the engine to convert a non-trivial win. E.g. a KBNK mate typically takes some 60 ply, but, as long as no B or N is blundered away, is always a certain win. There is no way a search would reach 60 ply if it has no guidance for what to prune; without pruning you must be happy to reach 8 ply. And there is no guidance, neither for prunig as for striving for intermediate goals (like getting the bare King into the corner), as all non-sacrificial end-leaves will have an identical 100% score. This reduces the engine to a random mover biased against giving away any material. No way its random walk through state space will ever bring a checkmate within the horizon. At some point it will stumble into the 50-move barrier, and there is nothing it can be done to avoid it.

A corrolary is that when an engine is trained to reproduce the true win rate on the games played by the latest version, it can never achieve the exact win rate. When it is still ignorant it will discover that it will have a higher chance of winning in positions where the mate is close, just because it has a larger chance to stumble on a position that has it within the search horizon. Those will then get higher score, which can be seen by the search from further away, so that the win rate there will also improve. But it has to keep a score gradient large enough to guide the play towards the mate from far away. And there is some minimum gradient for which this still works, because there will always be evaluation noise. So you will get some equilibrium, where positions far away from any mate only get, say, 90% win rate. Because that will indeed be their win rate, because the gradient from 90% to 100% is spread over so many moves that it has become so small that it fails to follow it in 10% of the cases.

Are you saying an engine can have precise, accurate analysis, or dominating play strength, but not really both at once?

The enemy's king is hiding in a desert with lots of high dunes obscuring vision at ground level. Two different companies of soldiers will attempt to kill him:

1. Stockfish (SF)

2. Quick checkmater (QC)

SF parachute into the middle of the desert, quickly getting close. Now they're there, though, they don't know which direction to travel in: they just have to guess, try a direction, and hope to uncover some clues.

QC start their assault from the edge of the desert, carefully charting their position as they go. This will lead them directly to the king - but it's too obvious and predictable, so they will get ambushed and killed.

Unbelievable BS. Chess is not life nor war, both of which are massively complex, unbounded and anything can happen. They are neither two sided, take it in turns to move operating on a tiny space of 64 squares with pieces that can move in six ways only. The "chess is life" concept is and always was a nonsense fantasy. Tal included.

Simple game theory concepts don't necessarily capture the complexity of chess. Chess may have nonlinear, emergent properties that only arise at different scales. Seeing chess as only 64 squares and 16 pieces is just one way to look at the resulting game.

Chris misunderstood: I wasn't saying that chess is an accurate simulation of a company of soldiers capturing an enemy king in the desert. I was helping you to understand what HGM said.

"Gradient" refers to an indicator showing the direction of the win (in a chess engine, a flat gradient would be getting the same eval on move 2 as you got on move 1 - thus failing to indicate whether this move gets you closer to the win). This I likened to being in the desert and not being able to see anything that gives you clues as to which direction to walk. To complete the analogy:

* Stockfish will get you into a strong position close to the win, but may not be able to find the fastest route to the available win.

* An engine that could navigate to the win might not be able to get you to a winning position in the first place.

[ovenel]

From reading through this thread, it's a tad unclear what your goal is here. Is your aim to find a path towards stronger chess engines (however that may be defined) than the current standard architecture yields, or are you hoping to find a path towards more human-like and human-comprehendible chess engines?

Those are two different goals that yield different development paths for engines. The fact of the matter is that modern engines are vastly more capable at chess than humans can ever hope to be. Naturally, that means that many of the moves that an engine makes will be outside of human understanding, so we will oftentimes watch an engine run off with a game without any clear idea about what happened. We can look to the tablebases to see a purified version of this. In them, the sequence of winning moves from a given position in the tablebase is known for certain. There is no heuristic evaluation function involved here as the tablebases are built with an (essentially) exhaustive search of all possibilities to the end. What we find from these brute-force lines is still far beyond human understanding. We can comprehend how they were found, but to understand why certain moves win and others lose is still beyond us. The most capable possible chess engine would be equivalent to a tablebase which contains all possible chess positions. So naturally, the vast confusion that we see in the existing tablebases would exist in this theoretical maximum. That is, the better the chess engine, the more incomprehensible it will be to us.

For some, the allure is in this superhuman ability. After all, these are engines that are built by humans, and we've built something that is capable of so much more than we can aspire to ever achieve ourselves. If you want to speak to philosophy, you can look to Nietzsche and his idea of the Übermensch. Rather than his idea of deriving meaning from the hope of giving birth to an Übermensch, we can instead find this purpose in creating something far beyond our capabilities.

However, there is another branch of engine developers. We've already developed engines far beyond our understanding, so why not aim for something comprehensible? Rather than chasing further superhuman abilities, why not chase human ability? In this way, the chess engine will remain a tool for us to use. An automated opponent that is always willing to play another match, but also one that can actually feel like we are playing a human. Or maybe we want to get better at chess and want something that can teach us something that we are ready and able to learn. An example of an engine being developed under this mindset would be Maia Chess.

My assumption is that you are in the latter camp, but a lot of your discussion seems to be in the former. Are you hoping to be able to learn more from a chess engine, or are you under the impression that more human-like evaluation will yield even more superhuman play?

[towforce]

From reading through this thread, it's a tad unclear what your goal is here...

The goal doesn't seem to be fixed, but the clearest description we have is here - link.

It's not obvious to me how applying philosophical concepts is going to uncover "...the deeper structure of chess" (quoted from linked post), but sometimes progress is made by applying the concepts from one subject to another subject.

[FireDragon761138]

You can look at Lc0's lines and they are more comprehensible than Stockfish's, because Lc0's internal representation of chess is more coherent and holistic, able to preserve nonlinear responses in a way that Stockfish's NNUE can't.

My goal is to have an engine that doesn't tell me to sacrifice my bishop so that in 20 moves there's concrete compensation. I'm more interested in developing an engine that helps a human develop sound positional judgement. Think of more Karpov than Kasparov. And I'm going with a more alpha-beta style engine because it's more computationally efficient. My goal is more about epistemic humility and flexibility, producing a reasonably strong engine in the process that can run on typical hardware.

[ovenel]

You can look at Lc0's lines and they are more comprehensible than Stockfish's, because Lc0's internal representation of chess is more coherent and holistic, able to preserve nonlinear responses in a way that Stockfish's NNUE can't.

My goal is to have an engine that doesn't tell me to sacrifice my bishop so that in 20 moves there's concrete compensation. I'm more interested in developing an engine that helps a human develop sound positional judgement. Think of more Karpov than Kasparov. And I'm going with a more alpha-beta style engine because it's more computationally efficient. My goal is more about epistemic humility and flexibility, producing a reasonably strong engine in the process that can run on typical hardware.

I'll be honest, phrases like "epistemic humility and flexibility" confuse me more than anything. But my understanding is that you are striving for an engine that is based on more human-like play. So rather than playing a move which pays off far into the future, you want it to play a move that a human could appreciate. Would you agree that this is naturally going to entail a weaker engine overall? If the bishop sacrifice yields a better position 20 moves into the future, then the bishop sacrifice is the best move, regardless of whether you could possibly understand that to be the case.

Humans are not oracles of the game that can understand the full ramifications of any given move, so limiting an engine to play within the realms of human understanding is going to limit its potential. That is to say, a chess engine which is developed with the goal of being the strongest possible chess player is necessarily going to play moves outside of human comprehension. So it's not fair to critique an engine that was developed with this goal based on how well a human can understand its moves.

That isn't to say that human-like or human-comprehensible play is a bad or easy development goal for an engine. Merely that it's a different goal that is not chased by all chess engines. It sounds like the "philosophic problems behind Stockfish's evaluation" are predicated on the fact that it's hard for a human to understand why it prefers some moves or positions over others. That ability to understand its decisions is counter to Stockfish's development goal of being the best possible chess engine, so it seems unfair to consider this a problem. It might be a better critique for Maia as the criterion of human-understandability is much more inline with its goal of human-like play.

Ultimately, I would say that it's fair to say that Stockfish is hard to understand, and it's a laudable goal to develop a chess engine which is easy to understand. But it is not fair to say that this is a problem for Stockfish. Unless the problem that you refer to is that the lack of comprehensibility is hampering Stockfish's ability to improve further.

[syzygy]

I'll be honest, phrases like "epistemic humility and flexibility" confuse me more than anything.

He is just pulling every empty comment through an obfuscating LLM...

[mclane]

Training a network for exactly reproducing win probability in principle desroys the possibility for the engine to convert a non-trivial win. E.g. a KBNK mate typically takes some 60 ply, but, as long as no B or N is blundered away, is always a certain win. There is no way a search would reach 60 ply if it has no guidance for what to prune; without pruning you must be happy to reach 8 ply. And there is no guidance, neither for prunig as for striving for intermediate goals (like getting the bare King into the corner), as all non-sacrificial end-leaves will have an identical 100% score. This reduces the engine to a random mover biased against giving away any material. No way its random walk through state space will ever bring a checkmate within the horizon. At some point it will stumble into the 50-move barrier, and there is nothing it can be done to avoid it.

A corrolary is that when an engine is trained to reproduce the true win rate on the games played by the latest version, it can never achieve the exact win rate. When it is still ignorant it will discover that it will have a higher chance of winning in positions where the mate is close, just because it has a larger chance to stumble on a position that has it within the search horizon. Those will then get higher score, which can be seen by the search from further away, so that the win rate there will also improve. But it has to keep a score gradient large enough to guide the play towards the mate from far away. And there is some minimum gradient for which this still works, because there will always be evaluation noise. So you will get some equilibrium, where positions far away from any mate only get, say, 90% win rate. Because that will indeed be their win rate, because the gradient from 90% to 100% is spread over so many moves that it has become so small that it fails to follow it in 10% of the cases.

Are you saying an engine can have precise, accurate analysis, or dominating play strength, but not really both at once?

The enemy's king is hiding in a desert with lots of high dunes obscuring vision at ground level. Two different companies of soldiers will attempt to kill him:

1. Stockfish (SF)

2. Quick checkmater (QC)

SF parachute into the middle of the desert, quickly getting close. Now they're there, though, they don't know which direction to travel in: they just have to guess, try a direction, and hope to uncover some clues.

QC start their assault from the edge of the desert, carefully charting their position as they go. This will lead them directly to the king - but it's too obvious and predictable, so they will get ambushed and killed.

Unbelievable BS. Chess is not life nor war, [..] The "chess is life" concept is and always was a nonsense fantasy. Tal included.

Says the guy who once wrote:
(I do quote Mr. Chris W. In person:)

„ Von Manstein
============
If, as is said, chess is war, then there must be lessons to be learnt from military history. I have already alluded to the static, boring First World War style of the classical programs (and their programmers !). The opposite style can be found in several histories, Rommel in North Africa, Alexander the Great against Darius, Von Manstein in Russia. Alexander, despite being outnumbered many times, concentrated the powerful mobile part of his army, attacked the stronger Persians, cut through and went straight for Darius himself. The bulk of Darius's army was not engaged, but the battle was decisively won - a classic king attack. Von Manstein (and Rommel) both understood that the power of the outnumbered German army lay in superior staff work, concentration of forces, striking blows to knock the enemy off balance. The looking-glass chess program must contain knowledge of these dynamic elements; and it is only the looking-glass program that has the knowledge and evaluation time available to calculate such ephemerals.“

So suddenly this was all wrong ?

I can’t believe you argue against yourself.
What has happened to you ?!

This is a quote from THIS forum from 2007.

„ In a materialistic world, in the materialistic world of computer chess, Chess System Tal offers the alternative pathway of idealism.“

[towforce]

OK - so playing positional chess is more valuable here than beating other engines. It's not obvious how you'll train an NN to do that, so your best bet is write a hand-coded evaluation function (HCE), which rewards your pieces having good positional elements (connected, mutually supportive, attacking lines available etc) and punishes the opponent's pieces for the same.

You can save time by using a chess software library (or even an open source engine) for the rest of the program, so that you can concentrate on getting the eval to generate the chess you want to see.