stockfish 10 vs. Mephisto III S Glasgow

[Madeleine Birchfield]

Some engines have problems in late middlegames and complex endgames, because their search is not fast enough. That's SF's greatest advantage, imho. Indeed, we could say that the lack of "understanding" and "planning" can be seen in such positions. I am sure that we all witnessed engines "losing the plot", "playing without aim" in positions we feel we could handle better. Frank Q's stats could be interesting in this matter, and just a few days ago, John Stanback complained about Wasp being crushed in endgames by SlowChess.

Problems in late middlegames and complex endgames might indicate a weak endgame evaluation rather than (or in addition to) a weak search. Slowchess 2.8 is extremely strong at endgames because Jonathan Rosenthal specifically trained special neural networks for use in the endgame, which allows Slowchess to play at Stockfish 12 level in the endgame. I am not aware of any other engine that has done that. Meanwhile Wasp's endgame evaluation is pretty poor, having just switched to a small neural network for general evaluation in 5.00.

On the other hand, Slowchess 2.8's opening play is relatively weak, as they spent most of the time training the endgame networks rather than the the general network.

[mclane]

Back to the topic.
A plan is not a main line

Stockfish main line and score shows it has no plan.

[Madeleine Birchfield]

Back to the topic.
A plan is not a main line

Stockfish main line and score shows it has no plan.

Leela's main line and score shows it has no plan either by your standards.

[matejst]

Problems in late middlegames and complex endgames might indicate a weak endgame evaluation rather than (or in addition to) a weak search. Slowchess 2.8 is extremely strong at endgames because Jonathan Rosenthal specifically trained special neural networks for use in the endgame, which allows Slowchess to play at Stockfish 12 level in the endgame. I am not aware of any other engine that has done that. Meanwhile Wasp's endgame evaluation is pretty poor, having just switched to a small neural network for general evaluation in 5.00. On the other hand, Slowchess 2.8's opening play is relatively weak.

Of course, Madeleine, in such cases engines lack knowledge, but a fast, deep search can make for the lack of knowledge most often than not. Indeed, Jonathan Kreuzer understood well that engines most often than not win or lose games in technical positions. I remember than I tested K8 or K9 a year or two ago against Wasp, and while Wasp was OK in the opening, it lost easily in simple positions. It is interesting that weaknesses in the initial HCE of an engine remain weaknesses in the NN if not addressed.

[Jonathan Kreuzer is SlowChess' author, Jonathan Rosenthal is Winter's.]

[Madeleine Birchfield]

Of course, Madeleine, in such cases engines lack knowledge, but a fast, deep search can make for the lack of knowledge most often than not. Indeed, Jonathan Kreuzer understood well that engines most often than not win or lose games in technical positions. I remember than I tested K8 or K9 a year or two ago against Wasp, and while Wasp was OK in the opening, it lost easily in simple positions. It is interesting that weaknesses in the initial HCE of an engine remain weaknesses in the NN if not addressed.

[Jonathan Kreuzer is SlowChess' author, Jonathan Rosenthal is Winter's.]

In Stockfish's case, Stockfish uses a hybrid evaluation function that switches from its neural network evaluation to its old handcrafted evaluation as the game continues and the position gets simplified, and they do that because come a certain point, the speed gains from using a simpler and faster evaluation function provide more benefits than the accuracy of a stronger neural network would, as it would allow the engine to search more deeply.

I believe this would remain true regardless of how fast the search is in general.

[Rebel]

Back to the topic.
A plan is not a main line

Stockfish main line and score shows it has no plan.

Unfortunately for you chess is a search game. If humans were able to look 50 moves ahead no chess theory was needed, neither a plan.

[JohnW]

Stockfish is not understanding what it is doing. It is more a pocket calculator.

I was just thinking the same thing, Stockfish is like a pocket calculator. The desire to give a chess engine knowledge and teach it how to plan is like a desire to create a new pocket calculator that does math like a human would (instead of binary, logic gates etc). Why would anyone want to?

[Madeleine Birchfield]

I was just thinking the same thing, Stockfish is like a pocket calculator. The desire to give a chess engine knowledge and teach it how to plan is like a desire to create a new pocket calculator that does math like a human would (instead of binary, logic gates etc). Why would anyone want to?

Isn't that what the Stockfish team is doing right now after it has adopted neural networks for evaluation? How is reinforcement training neural networks, like what Tomasz Sobczyk is doing right now for Stockfish and what Dietrich Kappe is doing for Komodo Dragon, not teaching an a chess engine how to plan? Or if reinforcement training isn't giving a chess engine knowledge, then why is Leela, which Thorsten Czub likes to claim is different from Stockfish, any different from Stockfish and not just like another pocket calculator?

[JohnW]

I was just thinking the same thing, Stockfish is like a pocket calculator. The desire to give a chess engine knowledge and teach it how to plan is like a desire to create a new pocket calculator that does math like a human would (instead of binary, logic gates etc). Why would anyone want to?

Isn't that what the Stockfish team is doing right now after it has adopted neural networks for evaluation? How is reinforcement training neural networks, like what Tomasz Sobczyk is doing right now for Stockfish and what Dietrich Kappe is doing for Komodo Dragon, not teaching an a chess engine how to plan? Or if reinforcement training isn't giving a chess engine knowledge, then why is Leela, which Thorsten Czub likes to claim is different from Stockfish, any different from Stockfish and not just like another pocket calculator?

I am not an expert in neural networks but I think of them as databases or similar to opening books. I don't think the engine is actually planning in the human sense of the word. Just like the analogy of the GPS navigation system mentioned earlier, it's not really planning a route from point A to point B. It identifies where you are now and where you want to go then triangulates. Then with that information it looks up the route in a database. Sure it may appear to be planning but it's really not.

[Madeleine Birchfield]

I am not an expert in neural networks but I think of them as databases or similar to opening books.

Neural networks are nothing like databases or opening books. Instead, they are universal function approximators.

https://en.wikipedia.org/wiki/Universal ... on_theorem

In computer chess, the holy grail of evaluation functions is a 32 man endgame tablebase. The evaluation function takes in a board position and outputs a win, draw, or loss for white, which we could map to positive infinity, zero, and negative infinity on the real number line. However, we don't have a 32 man endgame tablebase yet, so we have to create our own evaluation function to approximate the theoretical value that a 32 man endgame tablebase would return, which would take in a board position and return some real number on the number line.

In the past, we would have to write our own handcrafted evaluation terms in our function, but using neural networks provide a shortcut as they could approximate any function, so we don't have to spend days trying to write a handcrafted evaluation function. Furthermore, chess is a complex enough of a game that it is simply near impossible enough to handcraft a good enough evaluation function for certain aspects of the game (opening, middlegame, positional understanding, et cetera), so neural networks are great for those aspects of the evaluation function.