Which version of Komodo is the strongest, at long time control ?

[Chessqueen]

At long time control which version of Komodo is the strongest ? We can NOT judge based on just a few games played on the TCEC Website, but there is a strong indication that the MTCS is NOT stronger ?
https://tcec-chess.com

[lkaufman]

At long time control which version of Komodo is the strongest ? We can NOT judge based on just a few games played on the TCEC Website, but there is a strong indication that the MTCS is NOT stronger ?
https://tcec-chess.com

As far as we know, regular Komodo is stronger than Komodo MCTS at any time limit for actual game play. But if you want to display two or more lines of analysis, not just the best line, then Komodo MCTS is stronger, definitely so with three or more lines for each engine.

[Chessqueen]

At long time control which version of Komodo is the strongest ? We can NOT judge based on just a few games played on the TCEC Website, but there is a strong indication that the MTCS is NOT stronger ?
https://tcec-chess.com

As far as we know, regular Komodo is stronger than Komodo MCTS at any time limit for actual game play. But if you want to display two or more lines of analysis, not just the best line, then Komodo MCTS is stronger, definitely so with three or more lines for each engine.

If that is the case that regular Komodo is stronger, why don't you tell TCEC to remove Komodo MCTS from Division P, and only let Komodo regular which is the only one that has a chance to get 3rd place behind Stockfish, and LCZero and that is if it score better than Alliestein and Stoofvlees II 12 in Division P.
https://tcec-chess.com

[lkaufman]

At long time control which version of Komodo is the strongest ? We can NOT judge based on just a few games played on the TCEC Website, but there is a strong indication that the MTCS is NOT stronger ?
https://tcec-chess.com

As far as we know, regular Komodo is stronger than Komodo MCTS at any time limit for actual game play. But if you want to display two or more lines of analysis, not just the best line, then Komodo MCTS is stronger, definitely so with three or more lines for each engine.

If that is the case that regular Komodo is stronger, why don't you tell TCEC to remove Komodo MCTS from Division P, and only let Komodo regular which is the only one that has a chance to get 3rd place behind Stockfish, and LCZero and that is if it score better than Alliestein and Stoofvlees II 12 in Division P.
https://tcec-chess.com

I don't follow your reasoning. Komodo MCTS qualified. There are normally other engines in the top group that are also probably weaker than normal Komodo, why not toss all of them out too? Komodo MCTS, just like another "weaker" engine, could get lucky, or could turn out to be stronger than we think under TCEC conditions.

[leavenfish]

At long time control which version of Komodo is the strongest ? We can NOT judge based on just a few games played on the TCEC Website, but there is a strong indication that the MTCS is NOT stronger ?
https://tcec-chess.com

As far as we know, regular Komodo is stronger than Komodo MCTS at any time limit for actual game play. But if you want to display two or more lines of analysis, not just the best line, then Komodo MCTS is stronger, definitely so with three or more lines for each engine.

If that is the case that regular Komodo is stronger, why don't you tell TCEC to remove Komodo MCTS from Division P, and only let Komodo regular which is the only one that has a chance to get 3rd place behind Stockfish, and LCZero and that is if it score better than Alliestein and Stoofvlees II 12 in Division P.
https://tcec-chess.com

I don't follow your reasoning. Komodo MCTS qualified. There are normally other engines in the top group that are also probably weaker than normal Komodo, why not toss all of them out too? Komodo MCTS, just like another "weaker" engine, could get lucky, or could turn out to be stronger than we think under TCEC conditions.

I love analyzing with MCTS!

However I do not fully understand the statement, "If you want to display two or more lines of analysis, not just the best line, then Komodo MCTS is stronger, definitely so with three or more lines for each engine". I assume you are talking about game conditions; so how is it that during 'play', the first line of three of (say, Stockfish 10) is simply weaker than the first line of K 13.1? Conceptually I can't wrap my brain around that being so at both blitz and long time controls.


And why with even just 1 pv would Komodo MTSC be weaker than regular Komodo if the internal evals are the same?

[jp]

However I do not fully understand the statement, "If you want to display two or more lines of analysis, not just the best line, then Komodo MCTS is stronger, definitely so with three or more lines for each engine".

For multiPV, AB Komodo is "wasting" computation it normally wouldn't do to give you the full second and lower lines, so its performance declines.

And why with even just 1 pv would Komodo MTSC be weaker than regular Komodo if the internal evals are the same?

Because it's not just the evaluation function. The difference is MC vs AB (or at least the way Komodo implements them).

[lkaufman]

At long time control which version of Komodo is the strongest ? We can NOT judge based on just a few games played on the TCEC Website, but there is a strong indication that the MTCS is NOT stronger ?
https://tcec-chess.com

As far as we know, regular Komodo is stronger than Komodo MCTS at any time limit for actual game play. But if you want to display two or more lines of analysis, not just the best line, then Komodo MCTS is stronger, definitely so with three or more lines for each engine.

If that is the case that regular Komodo is stronger, why don't you tell TCEC to remove Komodo MCTS from Division P, and only let Komodo regular which is the only one that has a chance to get 3rd place behind Stockfish, and LCZero and that is if it score better than Alliestein and Stoofvlees II 12 in Division P.
https://tcec-chess.com

I don't follow your reasoning. Komodo MCTS qualified. There are normally other engines in the top group that are also probably weaker than normal Komodo, why not toss all of them out too? Komodo MCTS, just like another "weaker" engine, could get lucky, or could turn out to be stronger than we think under TCEC conditions.

I love analyzing with MCTS!

However I do not fully understand the statement, "If you want to display two or more lines of analysis, not just the best line, then Komodo MCTS is stronger, definitely so with three or more lines for each engine". I assume you are talking about game conditions; so how is it that during 'play', the first line of three of (say, Stockfish 10) is simply weaker than the first line of K 13.1? Conceptually I can't wrap my brain around that being so at both blitz and long time controls.


And why with even just 1 pv would Komodo MTSC be weaker than regular Komodo if the internal evals are the same?

Komodo and Komodo MCTS use completely different searches to choose the best move. Basically, normal Komodo (like Stockfish, Houdini, etc.) assumes that the opponent will always play the move Komodo would play for that side, and decides what move is best assuming "perfect" play. Komodo MCTS is more realistic (or modest, if you like), in that it does not assume it can always predict the opponent's best move, or even find the best move for itself in a given position, and picks a move based on the likelihood of various moves being chosen at each point. So far, regular Komodo beats Komodo MCTS, though the margin is getting small. But if you force the engine to determine the best 3 moves (for example), Stockfish or normal Komodo will have to spend roughly triple the time to do this, since normally it doesn't care what move is second or third best. But Komodo MCTS by its very nature gathers statistics for all moves, so at no extra cost it can see which move is second best, third best, etc. Since regular Komodo isn't quite strong enough to give Komodo MCTS a double time handicap, even being forced to choose the 2 best moves is enough to put it behind MCTS, and 3 moves isn't even close.

[Nordlandia]

I know Fritz GUI support extra time for white. So it's possible to test time handicap with regular Komodo up against Komodo Monte Carlo Larry.

[lkaufman]

I know Fritz GUI support extra time for white. So it's possible to test time handicap with regular Komodo up against Komodo Monte Carlo Larry.

Yes, I've tested it with our own tester and double time is pretty close, with a slight edge to MCTS. Of course the exact results depend on time control, number of threads, etc. but I think the above will be generally accurate.

[Ovyron]

Conceptually I can't wrap my brain around that being so at both blitz and long time controls.

Because no programmer had ever cared about alternative moves in chess positions, they make the engine use 100% of its resources to find the best possible move and play it, and that's it, and they spend 100% of their time improving the engine so that this best move is better, and the rest doesn't matter. MultiPV or the searchmoves feature are almost an after-thought, most engines implement it naively (it was easy for me to suggest a change that improved Stockfish's elo at MultiPV by a significant amount), but most engines don't ever have a way for the engine to show an alternative to what they consider best, and their programmers only care about elo.

That's why when you switch to MultiPV, the engine switches into "dumb mode", it was not designed to analyze more than 1 move at a time, such a thing was never optimized, so you get incredibly poor results. Ironically, switching it on in some positions is the only way for the engine to find some moves, because those are ones it wasn't designed to be good at, because such positions are rare so being good at them doesn't bring the elo points, and all that matters is the elo.

There are notable exceptions, like engines designed to be weakened to provide an interesting opponents for people, and others that allow creating interesting personalities to put flavor in the games. And Komodo MCTS is such a thing, specially designed to be able to provide alternative moves in positions, easily surpassing the performance of stronger engines that weren't designed for this.

Of course if this MCTS system to give the best possible alternative moves was implemented in Stockfish instead of Komodo, we'd be getting much better alternatives than what Komodo can provide. The reason we don't have the best possible engine to show alternative moves is Stockfish's Licence, it is wanted that Komodo's code remains closed because it's commercial, the main reason the MCTS system was implemented was to increase Komodo's sales, so it wouldn't have made sense to implement some Stockfish MCTS that would outperform Komodo at this, because at least like this, it could be claimed that Komodo is the best in the world at something, and that requires closed source.