Komodo Dragon 2.5 released.

[Karol Majewski]

Hi Larry, I noticed that during analysis in MCTS mode the output is not being written into the log file. In standard mode everything works fine. Is this a bug?

[mjlef]

Hi Larry, I noticed that during analysis in MCTS mode the output is not being written into the log file. In standard mode everything works fine. Is this a bug?

The log file was just meant for debugging, so MCTS search information is not posted there. In fact, only shortened PV and other information is posted there even in non-MCTS search. If you have a specific use case you cn email us at the address on the web site and we can consider possible changes. You are the only person who has ever reported this though, so I suspect the demand is low. We would need to add a lot more information to log files, and it probably make more sense to use the UCI interface directly since it is complete.

Mark

[pohl4711]

KomodoChess has just released Dragon 2.5 at KomodoChess.com.

Huge experimental RoundRobin tournament (10500 games, 3min+1sec) with 3 engines (Stockfish 14, KomodoDragon 2.5 and KomodoDragon 2.5 MCTS), each with 5 different MultiPV-settings (1,2,3,5 and 7, were 1 is the normal, default playing mode). Goal: Measure, how much Elo is lost by calculating more than one PV-line. And to measure, if Dragon 2.5 MCTS has less Elo-loss, than the AlphaBeta-engines, when MultiPV is 3 or higher...
I think, the results are pretty interesting, especially, when you use engines for analyzing human games using the MultiPV-mode.

https://www.sp-cc.de/experiments.htm

Well, MultiPV mode in Stockfish is unnecessarily expensive.
With MCTS you always have full MultiPV info available by default!

If you're interested I can provide a SF version with a small change which
should give noticeably better results.

At the moment, I watch the ChesstourFinals with Carlsen on youtube, where Stockfish is used for analyzing with pv=3. That reminded me, that analyzing human chess is one of the most important fields of use for the top-engines. And my experiment has shown very clear, that MCTS is clearly better here, because pv=7 is possible without any Elo-loss. So, I think, it is a really bad thing, that MCTS is not part of the official Stockfish. MCTS in the next official SF release is, what the chess players really need IMHO.

[Michel]

KomodoChess has just released Dragon 2.5 at KomodoChess.com.

Huge experimental RoundRobin tournament (10500 games, 3min+1sec) with 3 engines (Stockfish 14, KomodoDragon 2.5 and KomodoDragon 2.5 MCTS), each with 5 different MultiPV-settings (1,2,3,5 and 7, were 1 is the normal, default playing mode). Goal: Measure, how much Elo is lost by calculating more than one PV-line. And to measure, if Dragon 2.5 MCTS has less Elo-loss, than the AlphaBeta-engines, when MultiPV is 3 or higher...
I think, the results are pretty interesting, especially, when you use engines for analyzing human games using the MultiPV-mode.

https://www.sp-cc.de/experiments.htm

Well, MultiPV mode in Stockfish is unnecessarily expensive.
With MCTS you always have full MultiPV info available by default!

If you're interested I can provide a SF version with a small change which
should give noticeably better results.

At the moment, I watch the ChesstourFinals with Carlsen on youtube, where Stockfish is used for analyzing with pv=3. That reminded me, that analyzing human chess is one of the most important fields of use for the top-engines. And my experiment has shown very clear, that MCTS is clearly better here, because pv=7 is possible without any Elo-loss. So, I think, it is a really bad thing, that MCTS is not part of the official Stockfish. MCTS in the next official SF release is, what the chess players really need IMHO.

This is not related to MCTS. It is due to the fact that the SF developers have chosen to give equal resources to all pv's in a multipv search.

[lkaufman]

KomodoChess has just released Dragon 2.5 at KomodoChess.com.

Huge experimental RoundRobin tournament (10500 games, 3min+1sec) with 3 engines (Stockfish 14, KomodoDragon 2.5 and KomodoDragon 2.5 MCTS), each with 5 different MultiPV-settings (1,2,3,5 and 7, were 1 is the normal, default playing mode). Goal: Measure, how much Elo is lost by calculating more than one PV-line. And to measure, if Dragon 2.5 MCTS has less Elo-loss, than the AlphaBeta-engines, when MultiPV is 3 or higher...
I think, the results are pretty interesting, especially, when you use engines for analyzing human games using the MultiPV-mode.

https://www.sp-cc.de/experiments.htm

Well, MultiPV mode in Stockfish is unnecessarily expensive.
With MCTS you always have full MultiPV info available by default!

If you're interested I can provide a SF version with a small change which
should give noticeably better results.

At the moment, I watch the ChesstourFinals with Carlsen on youtube, where Stockfish is used for analyzing with pv=3. That reminded me, that analyzing human chess is one of the most important fields of use for the top-engines. And my experiment has shown very clear, that MCTS is clearly better here, because pv=7 is possible without any Elo-loss. So, I think, it is a really bad thing, that MCTS is not part of the official Stockfish. MCTS in the next official SF release is, what the chess players really need IMHO.

This is not related to MCTS. It is due to the fact that the SF developers have chosen to give equal resources to all pv's in a multipv search.

It's true that giving less resources (by for example more reduction) to later moves in multipv will raise the quality of the best move chosen, but MCTS does this much better for two reasons. 1. It is totally free, not just less costly 2. If two or more moves are nearly tied for being best, they will get nearly equal resources. Only significantly worse moves will get significantly less resources. With A/B you just have the order of the moves to go by, not how much they differ. MCTS handles MultiPV exactly the way it should be handled, with the quality of analysis being in proportion to the merit of the move, not just to its position on a list.

[Cornfed]

At the moment, I watch the ChesstourFinals with Carlsen on youtube, where Stockfish is used for analyzing with pv=3. That reminded me, that analyzing human chess is one of the most important fields of use for the top-engines. And my experiment has shown very clear, that MCTS is clearly better here, because pv=7 is possible without any Elo-loss. So, I think, it is a really bad thing, that MCTS is not part of the official Stockfish. MCTS in the next official SF release is, what the chess players really need IMHO.

If it could be done with hurting the online experience, okay.

But, lets be honest here, pv7 is really overkill in most positions. pv7 could lead to a need to redesign of some online interfaces for the additional screen clutter of irrelevant information. This IS after all 'human chess' with human commentators...I know my eyes are on the main screen with (right now) Peter and Simon and the game they are commentating on (I think without help of engines ?)....and sometimes I look at the space efficient 'eval bar'. I personally never look at even the pv3 at the bottom of the chess.24 website I am on.

I think one could argue that a pv 7 (or 6 or 5 etc) for a move with no context really does nothing at all to enhance the experience for either really good, good, poor...really poor chess playing viewers...it is just a string of possible moves with a number attached to it. For example, why is 34. Qe2 +2.43 while 34 f5 +4.45 ???

[cma6]

"recent Stockfish engine output during analysis with the Win/Draw/Loss percentage "

Is that output available only in CBase/SF version?
When using SF in Aquarium, I don't see Win/Draw/Loss percentage.

[dkappe]

But, lets be honest here, pv7 is really overkill in most positions. pv7 could lead to a need to redesign of some online interfaces for the additional screen clutter of irrelevant information. This IS after all 'human chess' with human commentators...I know my eyes are on the main screen with (right now) Peter and Simon and the game they are commentating on (I think without help of engines ?)....and sometimes I look at the space efficient 'eval bar'. I personally never look at even the pv3 at the bottom of the chess.24 website I am on.

I think one could argue that a pv 7 (or 6 or 5 etc) for a move with no context really does nothing at all to enhance the experience for either really good, good, poor...really poor chess playing viewers...it is just a string of possible moves with a number attached to it. For example, why is 34. Qe2 +2.43 while 34 f5 +4.45 ???

I recall a discussion with GM Hammer about lc0 a few years ago. His complaint was that with multipv=4, the quality past the first move was often pretty questionable. If you’re familiar with UCT, that stands to reason. You get multipv “for free,” but it isn’t very good. Don’t know if they’ve done anything to fix that.

At any rate, we may not think that multipv isn’t useful for the majority of chess players, but what we think doesn’t really matter much.

[lkaufman]

But, lets be honest here, pv7 is really overkill in most positions. pv7 could lead to a need to redesign of some online interfaces for the additional screen clutter of irrelevant information. This IS after all 'human chess' with human commentators...I know my eyes are on the main screen with (right now) Peter and Simon and the game they are commentating on (I think without help of engines ?)....and sometimes I look at the space efficient 'eval bar'. I personally never look at even the pv3 at the bottom of the chess.24 website I am on.

I think one could argue that a pv 7 (or 6 or 5 etc) for a move with no context really does nothing at all to enhance the experience for either really good, good, poor...really poor chess playing viewers...it is just a string of possible moves with a number attached to it. For example, why is 34. Qe2 +2.43 while 34 f5 +4.45 ???

I recall a discussion with GM Hammer about lc0 a few years ago. His complaint was that with multipv=4, the quality past the first move was often pretty questionable. If you’re familiar with UCT, that stands to reason. You get multipv “for free,” but it isn’t very good. Don’t know if they’ve done anything to fix that.

At any rate, we may not think that multipv isn’t useful for the majority of chess players, but what we think doesn’t really matter much.

The quality of multipv with MCTS engines depends greatly on whether there are multiple moves of similar merit or not. If one move is pretty clearly best, the quality of the other lines will be low. But if say three moves are essentially too close to call, then the quality of the analysis of those three should be comparable. That is why I think multipv with MCTS is a good combination, you get quality analysis of the lines that have merit.

[dkappe]

The quality of multipv with MCTS engines depends greatly on whether there are multiple moves of similar merit or not. If one move is pretty clearly best, the quality of the other lines will be low. But if say three moves are essentially too close to call, then the quality of the analysis of those three should be comparable. That is why I think multipv with MCTS is a good combination, you get quality analysis of the lines that have merit.

I was just speaking about lc0, not Komodo/Dragon MCTS. There you would sometimes get a 3rd of 4th “best” move that would permit mate.