Komodo MCTS

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
AdminX
Posts: 5115
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Re: Komodo MCTS

Post by AdminX » Thu Jun 20, 2019 9:58 am

Modern Times wrote:
Wed Jun 19, 2019 6:05 pm
lkaufman wrote:
Wed Jun 19, 2019 4:07 pm
sound67 wrote:
Wed Jun 19, 2019 8:34 am
Ovyron wrote:
Tue Jun 18, 2019 9:13 pm
Now try the same match at MultiPV=4 (as some people use in analysis) :wink: :wink:
I used the maximum number of cores for the engines that my P5 processor allowed, MultiPV=5. Ponder=Off.
Did you set MultiPV = 5 for ALL engines, not just Komodo?
Just Komodo.

Your point about MPV and analysis is well taken. Komodo MCTS is superb for analysis, and is even more compelling it you don't have a multi-core monster machine.
I agree, It delivers what is advertised. I was I was stunned by the results of a small match with Shredder 13 and wonder how long till it starts touching the top 3 as well. My only complaint is wtih the MTCS endgames.
Attachments
KMCTS.png
KMCTS.png (12.31 KiB) Viewed 1142 times
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

sound67
Posts: 4
Joined: Tue May 14, 2019 10:39 am
Full name: Thomas Muething

Re: Komodo MCTS

Post by sound67 » Thu Jun 20, 2019 10:17 am

lkaufman wrote:
Wed Jun 19, 2019 4:07 pm
Did you set MultiPV = 5 for ALL engines, not just Komodo? With that setting, Komodo 13.02 MCTS is MUCH stronger than normal Komodo, and about the same strength as Stockfish 10 (also using MultiPV). If you are getting results with Komodo MCTS below normal Komodo, something is wrong.
I set the MultiPV for all engines to 5, otherwise used the default values and the Perfect2017 book for each side.

According to the CCRL list (https://ccrl.chessdom.com/ccrl/4040/), even the regular Komodo 11 is stronger than the Komodo 12 MCTS, and the makers of the program also implicitly state that for version 13 as they recommend MCTS for detecting unusual moves in analysis, but regular Komodo 13 for gameplay.

The weakness of the MCTS engine came into clear focus to me when time began to run out.

leavenfish
Posts: 269
Joined: Mon Sep 02, 2013 6:23 am

Re: Komodo MCTS

Post by leavenfish » Thu Jul 11, 2019 4:54 am

I saw a screen shot somewhere of Komodo MCTS and I know it had multiple CPU's involved...3 or something as there were 3 lines being looked at. It showed something like depth = 24/17; had to be in a Chessbase GUI.

Exactly what does that depth mean in Komodo?

It struck me because I remember Ntirlis in Aagaards THINKING INSIDE THE BOX say that a depth of 21/35 means that the principle variation is 21 ply deep but there other lines where the engine was looking 35 ply deep.

lkaufman
Posts: 3647
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo MCTS

Post by lkaufman » Thu Jul 11, 2019 5:38 am

leavenfish wrote:
Thu Jul 11, 2019 4:54 am
I saw a screen shot somewhere of Komodo MCTS and I know it had multiple CPU's involved...3 or something as there were 3 lines being looked at. It showed something like depth = 24/17; had to be in a Chessbase GUI.

Exactly what does that depth mean in Komodo?

It struck me because I remember Ntirlis in Aagaards THINKING INSIDE THE BOX say that a depth of 21/35 means that the principle variation is 21 ply deep but there other lines where the engine was looking 35 ply deep.
With Komodo MCTS (as also with Lc0), the first number is artificial, as there is no iteration in the normal sense. It is just a number calculated from the number of MCTS nodes generated, kind of like a substitute for depth in an A/B engine. The second number is the maximum depth reached by any MCTS node, but note that there might be another 15 plies or so of search that takes place AT that terminal MCTS node. So it is far from the maximum depth reached in the search.
Komodo rules!

Dann Corbit
Posts: 9901
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Komodo MCTS

Post by Dann Corbit » Thu Jul 11, 2019 5:41 am

leavenfish wrote:
Thu Jul 11, 2019 4:54 am
I saw a screen shot somewhere of Komodo MCTS and I know it had multiple CPU's involved...3 or something as there were 3 lines being looked at. It showed something like depth = 24/17; had to be in a Chessbase GUI.

Exactly what does that depth mean in Komodo?

It struck me because I remember Ntirlis in Aagaards THINKING INSIDE THE BOX say that a depth of 21/35 means that the principle variation is 21 ply deep but there other lines where the engine was looking 35 ply deep.
The normal meanings are as follows:
Even depth is not interpreted the same by every engine, as (for instance) some engines do not count single reply extensions.
But, in general the first number is the number of plies completed in the alpha-beta search.
The second number is selective search depth and it can mean anything the author wants.
It could be the alpha-beta depth + qsearch length.
It could be the total distance to the deepest line examined.
I think the intention was to add any search extensions but exactly what seldepth means is not clearly spelled out.
I think also that the description above is OK

Now, MCTS and NN programs have some other meaning for depth, because the do not generally even perform an alpha beta search
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

rcmaddox
Posts: 173
Joined: Mon Oct 14, 2013 6:59 pm
Location: Winder, GA

Re: Komodo MCTS

Post by rcmaddox » Thu Jul 11, 2019 6:47 am

I've been reluctant to post in threads about Leela or Komodo MCTS because, frankly, I don't have a good conceptual understanding of how these engines work. I hate to publicly display my limitations, but here goes...

Last night, just for kicks, I watched Komodo 13.02 MCTS play a series of games against Houdini 3 and Deep Shredder 13. Komodo MCTS did well, modestly outscoring both engines.

I noticed that if I turn off adjudication, so that a won game must be played to checkmate, Komodo would play horrible chess in trivially won endgames. For example, in one game, Komodo ended up with K and Q vs a lone K. Komodo seemed to just randomly check the opposing king, with no plan to combine with its own king to deliver checkmate. I watched Komodo chase the opposing king around the board for maybe 15 moves - then I just stopped the game. I suppose Komodo would have eventually stumbled into checkmate but life is short.

So ... Komodo made strong moves in the middle game but (seemingly) pointless moves when it had an overwhelming advantage in the endgame.

Is this normal? I did have 5-man Syzygy enabled. All this took place on my modest laptop.

lkaufman
Posts: 3647
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo MCTS

Post by lkaufman » Thu Jul 11, 2019 5:37 pm

rcmaddox wrote:
Thu Jul 11, 2019 6:47 am
I've been reluctant to post in threads about Leela or Komodo MCTS because, frankly, I don't have a good conceptual understanding of how these engines work. I hate to publicly display my limitations, but here goes...

Last night, just for kicks, I watched Komodo 13.02 MCTS play a series of games against Houdini 3 and Deep Shredder 13. Komodo MCTS did well, modestly outscoring both engines.

I noticed that if I turn off adjudication, so that a won game must be played to checkmate, Komodo would play horrible chess in trivially won endgames. For example, in one game, Komodo ended up with K and Q vs a lone K. Komodo seemed to just randomly check the opposing king, with no plan to combine with its own king to deliver checkmate. I watched Komodo chase the opposing king around the board for maybe 15 moves - then I just stopped the game. I suppose Komodo would have eventually stumbled into checkmate but life is short.

So ... Komodo made strong moves in the middle game but (seemingly) pointless moves when it had an overwhelming advantage in the endgame.

Is this normal? I did have 5-man Syzygy enabled. All this took place on my modest laptop.
This happens all the time with Lc0 as well. They know it's a win in fifty moves or less, but until they get to the point where they have to find the right moves to win in fifty moves they play randomly. So they don't miss the win, they just appear to be "trolling". Maybe we can find a better way.
Komodo rules!

carldaman
Posts: 1687
Joined: Sat Jun 02, 2012 12:13 am

Re: Komodo MCTS

Post by carldaman » Thu Jul 11, 2019 6:55 pm

The Lc0 endgame trolling is supposedly due to how its NN are trained, if my understanding is correct - (and that's always doubtful :| ), although one would hope that with few pieces on the board the MCTS search wouldn't have that much trouble finding a more direct win.

Now, what could be Komodo's trouble here, since it uses no NNs? Is it the MCTS algorithm that's the culprit after all?
I would sure like to have some clarity on this. Komodo is still being advertised as a great positional engine, but this endgame trolling clearly goes against assumption, as far as the MCTS version goes. Can't the syzygy bases help at all?

(I suppose one could always switch back to regular non-MCTS Komodo for endgame analysis.)

leavenfish
Posts: 269
Joined: Mon Sep 02, 2013 6:23 am

Re: Komodo MCTS

Post by leavenfish » Fri Jul 12, 2019 2:45 am

lkaufman wrote:
Thu Jul 11, 2019 5:38 am
leavenfish wrote:
Thu Jul 11, 2019 4:54 am
I saw a screen shot somewhere of Komodo MCTS and I know it had multiple CPU's involved...3 or something as there were 3 lines being looked at. It showed something like depth = 24/17; had to be in a Chessbase GUI.

Exactly what does that depth mean in Komodo?

It struck me because I remember Ntirlis in Aagaards THINKING INSIDE THE BOX say that a depth of 21/35 means that the principle variation is 21 ply deep but there other lines where the engine was looking 35 ply deep.
With Komodo MCTS (as also with Lc0), the first number is artificial, as there is no iteration in the normal sense. It is just a number calculated from the number of MCTS nodes generated, kind of like a substitute for depth in an A/B engine. The second number is the maximum depth reached by any MCTS node, but note that there might be another 15 plies or so of search that takes place AT that terminal MCTS node. So it is far from the maximum depth reached in the search.
Interesting indeed.

Of course 'depth search' ( or something like time used) provides a user with some comfort that an engine has chewed upon a position long enough to have some measure of 'trust' in the output....or in comparing the output vs the 'depth' of search for a traditional A/B engine.

But I can see where this could be useful in my opening/early middlegame analysis. I think I will get the Chessbase version, thanks!!

lkaufman
Posts: 3647
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo MCTS

Post by lkaufman » Fri Jul 12, 2019 4:48 am

carldaman wrote:
Thu Jul 11, 2019 6:55 pm
The Lc0 endgame trolling is supposedly due to how its NN are trained, if my understanding is correct - (and that's always doubtful :| ), although one would hope that with few pieces on the board the MCTS search wouldn't have that much trouble finding a more direct win.

Now, what could be Komodo's trouble here, since it uses no NNs? Is it the MCTS algorithm that's the culprit after all?
I would sure like to have some clarity on this. Komodo is still being advertised as a great positional engine, but this endgame trolling clearly goes against assumption, as far as the MCTS version goes. Can't the syzygy bases help at all?

(I suppose one could always switch back to regular non-MCTS Komodo for endgame analysis.)
It seems that the combination of MCTS with Syzygy is the problem. When using Syzygy, all lines that mate within 50 moves look equally good to MCTS. If you turn off Syzygy MCTS will mate much more efficiently with queen vs king. I think that Syzygy does add a couple elo points to Komodo MCTS, but it can sure make it look stupid or trollish.
Komodo rules!

Post Reply