Is Komodo trying to become like Stockfish? To Larry kaufman

[lkaufman]

It's true that we have narrowed the gap with Stockfish in search depth quite a bit with each new release, especially in MP mode. This is not because we want to be like Stockfish, but rather because I have been convinced for a long time that Komodo would be much stronger than Stockfish if only the depth gap was not so huge. It was not easy to increase the depth substantially without having the benefit get eaten up by missed moves, but we have managed to do it, and this is probably responsible for more than half of recent elo gains. Komodo's search is still quite different from Stockfish's, mostly due to more extensions, but we simply couldn't compete when we were outsearched by many plies. Despite intimate knowledge of the details of both engines, I still don't understand why Stockfish still reaches noticeably higher search depths than Komodo in longer time control games especially, but it no longer seems to be hurting us so our focus has moved on. I wouldn't object to getting less depth if it helped our elo, but that doesn't look likely to me. It is possible that less selectivity or reduction would help at longer time controls, since we don't test much at anything beyond normal blitz levels, but I think it is unlikely based on my understanding of all the issues. But if someone out there gets significant evidence that our settings are not optimum at longer tc, please let us know. We can always run a test at a longer tc if there is good reason to do so.

[fern]

Most probably this has been thought and tested and perhaps is even already used, but I cannot restrain myself of asking you if would not be a good idea to have a combination of deep and wide search according situation. Wide search in the first, say, 5 ply, then deep search with some lines, then wide search with those lines mos important, and so and so, in a way not to choose in general one approach or the other, but related to circumstances.

Fern

[lkaufman]

Most probably this has been thought and tested and perhaps is even already used, but I cannot restrain myself of asking you if would not be a good idea to have a combination of deep and wide search according situation. Wide search in the first, say, 5 ply, then deep search with some lines, then wide search with those lines mos important, and so and so, in a way not to choose in general one approach or the other, but related to circumstances.

Fern

Well, we (and other top engines) pretty much do this automatically; we prune more towards the end of a line, and try to search deeper in lines that we think are more important. Of course if we can improve our definition of what are important lines, we will get stronger. It's not easy though.

Larry

[Laskos]

Most probably this has been thought and tested and perhaps is even already used, but I cannot restrain myself of asking you if would not be a good idea to have a combination of deep and wide search according situation. Wide search in the first, say, 5 ply, then deep search with some lines, then wide search with those lines mos important, and so and so, in a way not to choose in general one approach or the other, but related to circumstances.

Fern

When the search becomes deeper, the first plies are searched more thoroughly too, on long searches it becomes almost full width at shallow depths. The tree grows both in depth and in width.

[fern]

I supposed that, but as you say, the point is how to do it.
Problem is you realize something is important after you missed it and you was crushed by it.

Again I will surely say something known since the stone age, but I wonder if some of that uncertainty could not be narrowed thinking in "geographical" terms. To say it very simple, you ut all your attention to pieces lurking around your King and in that situation you do not pay attention a what happens in the other corner of the board.
It is not a very scientific approach, but more or less is what we human do playing chess.

thanks for you polite replies
Fern

[fern]

Yes, Kai, I supposed something of that sort, but as Larry said, the point is how those changes are made. If just by scores, then we have in that a kind of circular reasonning.

Fern

[TShackel]

It's true that we have narrowed the gap with Stockfish in search depth quite a bit with each new release, especially in MP mode. This is not because we want to be like Stockfish, but rather because I have been convinced for a long time that Komodo would be much stronger than Stockfish if only the depth gap was not so huge. It was not easy to increase the depth substantially without having the benefit get eaten up by missed moves, but we have managed to do it, and this is probably responsible for more than half of recent elo gains. Komodo's search is still quite different from Stockfish's, mostly due to more extensions, but we simply couldn't compete when we were outsearched by many plies. Despite intimate knowledge of the details of both engines, I still don't understand why Stockfish still reaches noticeably higher search depths than Komodo in longer time control games especially, but it no longer seems to be hurting us so our focus has moved on. I wouldn't object to getting less depth if it helped our elo, but that doesn't look likely to me. It is possible that less selectivity or reduction would help at longer time controls, since we don't test much at anything beyond normal blitz levels, but I think it is unlikely based on my understanding of all the issues. But if someone out there gets significant evidence that our settings are not optimum at longer tc, please let us know. We can always run a test at a longer tc if there is good reason to do so.

Thanks Larry for your response. I'm glad you think that you've managed to increase depth without missing important or critical moves which is sometimes a tough compromise to make. That makes me happier knowing that. I'm also glad you're open to reducing depth if it means an increase an elo.

I know many people may think that's strange, that you can actually increase elo by reducing depth, when it seems more depth would usually increase elo. But by seeing more and being a little less selective and less aggressive in reductions it may end up being a better search structure for the evaluation function and thus increase elo. For instance if more moves were seen by komodo being a little less selective and aggressivve in reductions then the evaluation function could help see an important strategic maneuver in a move that would've been passed by with increased selectivity and reductions.

Now I understand pruning, making the search tree smaller, and late move reductions all help search pertinent or useful lines while eliminating all the useless lines that aren't worth calculating. I'm not suggesting komodo goes back to deep blue brute force, but rather they may find the evaluation function is more effective with a different search structure.

Thanks again Larry and Mark. Keep up the good work with Komodo.

Sincerely,

Tim.

[Laskos]

Yes, Kai, I supposed something of that sort, but as Larry said, the point is how those changes are made. If just by scores, then we have in that a kind of circular reasonning.

Fern

An engine like Komodo is not missing moves of 20 year old, almost full width engines. In a 10 second search on a modern PC, say shown depth of Komodo is 20, but that doesn't say a lot. Depth 20 is searched narrowly indeed, some lines go even to depth 30 or 40, but first 6-7 plies are searched almost full width, just like in the old engines, and even better. I think what you complained in other threads about modern engines being too selective and pruning too much has another cause. Basically Komodo is too strong for us, and such a strong play is often pretty boring. The main reason is that all top engines at default settings take the adversary to be of their own strength. They are optimizing the moves for an extremely strong adversary, not to beat me, a patzer. Simply, they take us too seriously. 20 years ago, engines were say 2200 ELO, so they played moves which were nicer against patzers, but weak against a 3200 ELO player. And what you ask is theoretically doable even with Komodo, say, to play wrong combinations which beat gloriously a patzer like me, but are mistakes against a strong engine. Probably if a demand of such feature is high, Komodo's authors may consider it. But there are other engines having these features, and there are weak engines doing this by default.

[bob]

Most probably this has been thought and tested and perhaps is even already used, but I cannot restrain myself of asking you if would not be a good idea to have a combination of deep and wide search according situation. Wide search in the first, say, 5 ply, then deep search with some lines, then wide search with those lines mos important, and so and so, in a way not to choose in general one approach or the other, but related to circumstances.

Fern

When the search becomes deeper, the first plies are searched more thoroughly too, on long searches it becomes almost full width at shallow depths. The tree grows both in depth and in width.

That's not necessarily true. For example, assume you do LMR at the root. In that case the first ply gets NARROWER as you go deeper, because as the depth increases so does the LMR reduction amount. Each iteration trims more off of the later moves. That's the only way to keep the EBF at 2.0 and lower. Any extra effort near the root blows up way out in the tree.

Near the tips what you describe actually happens, because forward pruning is only done with N plies of the tips of the branches. Each additional ply of search pushes that pruning out one ply farther, which I suppose might be considered searching more carefully near the root...

[fern]

I do not have an idea from where you got the idea I am asking for weak engines....
No...
I was just musing a bit about search techniques and that is all.
And I had as reference no my paltry 2100 or so elo player level but the best of them..
In fact, when I play with the idea to have some chances, I do it againts one of my dedicated units, which goes from 1300 to 2200 Elo.

Fern