Komodo 12 and MCTS

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Joost Buijs
Posts: 958
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: Komodo 12 and MCTS

Post by Joost Buijs » Mon May 28, 2018 6:28 pm

sovaz1997 wrote:
Mon May 28, 2018 5:22 pm
mjlef wrote:
Mon May 28, 2018 2:22 am
I would love someone to try another experiment. Take the Leela neural network, and just write a classic search for it, using the Leela neural network in place of an eval, but include all the standard cutoffs. If easier, you could convert the win percentage back to a centipawn score, and try all the standard pruning tricks we do in programs like Stockfish. I would love to see how it performs. Neural networks can in many ways see future moves via piece cooperation, attack and defense terms. It might be a very powerful program.
Interesting idea. I thought about this before :)

Replacing the regular eval with the Leela NN probably won't work, the NN is ~1000 times slower and this will wreck your engine tactically. Maybe there is a way to make some kind of hybrid, using the NN closer to the root for pruning decisions and using the classic search for the remaining things.

mjlef
Posts: 1424
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: Komodo 12 and MCTS

Post by mjlef » Mon May 28, 2018 8:23 pm

Joost Buijs wrote:
Mon May 28, 2018 6:28 pm
sovaz1997 wrote:
Mon May 28, 2018 5:22 pm
mjlef wrote:
Mon May 28, 2018 2:22 am
I would love someone to try another experiment. Take the Leela neural network, and just write a classic search for it, using the Leela neural network in place of an eval, but include all the standard cutoffs. If easier, you could convert the win percentage back to a centipawn score, and try all the standard pruning tricks we do in programs like Stockfish. I would love to see how it performs. Neural networks can in many ways see future moves via piece cooperation, attack and defense terms. It might be a very powerful program.
Interesting idea. I thought about this before :)

Replacing the regular eval with the Leela NN probably won't work, the NN is ~1000 times slower and this will wreck your engine tactically. Maybe there is a way to make some kind of hybrid, using the NN closer to the root for pruning decisions and using the classic search for the remaining things.
The nn score could be converted to centipawns, and those used in the regular search. Looking at the length or PVs in Monte Carlo programs, they seem to be much shorter than PV lengths of traditional searchers. But the nn based programs seem to compensate by in effect, doing a kind of tradeoff in the eval, which makes it looks several plies deeper. Something like that must be learned in the nn, or Leela would be slaughtered tactically.

Anyway, just some thoughts. If anyone tries the experiment, let me know.

mjlef
Posts: 1424
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: Komodo 12 and MCTS

Post by mjlef » Mon May 28, 2018 8:28 pm

Hurnavich wrote:
Mon May 28, 2018 1:03 pm
I get it running but it has slow speed and depth and not sure what it is supposed to be doing

K12 normal works perfect
In MCTS mode, Komodo is going to not be seeing depth the same as in normal mode. In fact, the depth report is really just the number of Monte Carlo Tree nodes converted (log (nodes) / log (constant) ). Some GUIs need a depth or they are not happy, and we need some kind of depth value in case a user tell it to go depth 10.

Komodo is actually seeing deeper than the PV lines is shows, but is using this extra depth to estimate min percentages.

Although we have good data for strength, it is hard to test for "style". We find MCTS much less likely to worry about material, and more likely to leave fewer good responses for its opponents. At least this is our impression.

lkaufman
Posts: 3675
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 12 and MCTS

Post by lkaufman » Mon May 28, 2018 9:09 pm

Hurnavich wrote:
Mon May 28, 2018 1:03 pm
I get it running but it has slow speed and depth and not sure what it is supposed to be doing

K12 normal works perfect

I think that earlier you couldn't get it to run in the Fritz GUI because you selected the "Monte Carlo" Fritz option, which only works with Fritz and Rybka. I gather that you figured out that you have to choose the MCTS UCI option in Komodo. The NPS is only Monte Carlo nodes per second, so it is something like a thousand times slower than total nodes per second. The utility of it is rather limited at the moment, but we should have an upgrade for the MCTS option pretty soon that will make it a lot stronger and more useful, especially on good hardware.
Komodo rules!

CositasBuenas
Posts: 105
Joined: Tue Aug 03, 2010 5:36 pm

Re: Komodo 12 and MCTS

Post by CositasBuenas » Tue May 29, 2018 4:50 am

Great news--can't wait!

Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: Komodo 12 and MCTS

Post by Milos » Tue May 29, 2018 1:50 pm

mjlef wrote:
Mon May 28, 2018 8:23 pm
Joost Buijs wrote:
Mon May 28, 2018 6:28 pm
sovaz1997 wrote:
Mon May 28, 2018 5:22 pm

Interesting idea. I thought about this before :)

Replacing the regular eval with the Leela NN probably won't work, the NN is ~1000 times slower and this will wreck your engine tactically. Maybe there is a way to make some kind of hybrid, using the NN closer to the root for pruning decisions and using the classic search for the remaining things.
The nn score could be converted to centipawns, and those used in the regular search. Looking at the length or PVs in Monte Carlo programs, they seem to be much shorter than PV lengths of traditional searchers. But the nn based programs seem to compensate by in effect, doing a kind of tradeoff in the eval, which makes it looks several plies deeper. Something like that must be learned in the nn, or Leela would be slaughtered tactically.

Anyway, just some thoughts. If anyone tries the experiment, let me know.
As Joost said, it's a bad idea. There is an excel sheet with LC0 performance vs. SF in terms of number of nodes (https://docs.google.com/spreadsheets/d/ ... li=1#gid=0).
As you can see 1 LC0 node evaluated with the best network atm is around 360Elo weaker than 10k nodes of SF9.
On fast single core you can evaluate roughly 40 LC0 nodes per second. OTOH you can search at least 1.5 million nodes of SF9. The ratio is close to 40k. So you'd end up with at least 500Elo weaker engine.

glennsamuel32
Posts: 85
Joined: Sat Dec 04, 2010 4:31 pm
Location: 223

Re: Komodo 12 and MCTS

Post by glennsamuel32 » Tue May 29, 2018 9:00 pm

Chessbase just sent me a promo email...

Image

shrapnel
Posts: 1157
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Komodo 12 and MCTS

Post by shrapnel » Wed May 30, 2018 1:04 pm

glennsamuel32 wrote:
Tue May 29, 2018 9:00 pm
Chessbase just sent me a promo email...

Image
Projecting itself as the next AlphaZero. :lol: ...when its actually LIGHT YEARS behind AlphaZero.
Lefler can NEVER hope to match the Brilliant Programmers at Google, many of whom I'm proud to say, are of Indian Origin !
Now, that's the perfect example of misleading Advertisement.
Daniel Shawul and Pohl were right.
Komodo is relying on Lies and Deceit to further its Sales.
Shameful !
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Asus ROG Strix OC 11 GB Geforce 2080 Ti

CositasBuenas
Posts: 105
Joined: Tue Aug 03, 2010 5:36 pm

Re: Komodo 12 and MCTS

Post by CositasBuenas » Wed May 30, 2018 3:05 pm

Nonsense! This is simply an example of chessbase's promotion/advertisment mumbo-jumbo!

Tdunbug
Posts: 47
Joined: Sun Mar 06, 2016 11:46 pm

Re: Komodo 12 and MCTS

Post by Tdunbug » Wed May 30, 2018 3:31 pm

shrapnel wrote:
Wed May 30, 2018 1:04 pm
glennsamuel32 wrote:
Tue May 29, 2018 9:00 pm
Chessbase just sent me a promo email...

Image
Projecting itself as the next AlphaZero. :lol: ...when its actually LIGHT YEARS behind AlphaZero.
Lefler can NEVER hope to match the Brilliant Programmers at Google, many of whom I'm proud to say, are of Indian Origin !
Now, that's the perfect example of misleading Advertisement.
Daniel Shawul and Pohl were right.
Komodo is relying on Lies and Deceit to further its Sales.
Shameful !
I am not quite that the last part of your statement is true. Neither Larry or Mark have made such claims as to what Chessbase have done. Matter of fact, I am willing to put my neck out and say that Chessbase has it's own marketing agenda that is totally separate of the komodo team.

Post Reply