TalkChess.com

Tdunbug wrote: ↑Tue May 15, 2018 1:42 am I don't speak much here but I do want to say that MCTS is a great idea and look forward to it being improved. At the very least, it is a fresh new idea that can be tested.

Daniel Shawul wrote: ↑Mon May 14, 2018 9:53 pm There is so much bullshit here. The authors are throwing around ideas to benefit from the A0 hype. E.g.
a) We use averaging => style is much better like A0's
b) MCTS better against humans. I bet standard komodo is much better against computers, humans and cockroaches
c) MCTS better in multi-pv mode, I highly doubt it
d) "Although tactical strength is lower with MCTS, positional play and judgment may well be better in many positions". Pure bullshit as they
don't have NN evaluation which account for A0's positional strength. In general, MCTS maybe better strategically but not at the level they are suggesting without the NN
e) "Evals don't fluctuate so suddenly or wildly." So what? That is infact bad since you will have to spend many more simulations to realize
you are screwed with some shallow tactics.

Daniel

shrapnel wrote: ↑Tue May 15, 2018 6:55 am

Daniel Shawul wrote: ↑Mon May 14, 2018 9:53 pm
b) MCTS better against humans. I bet standard komodo is much better against computers, humans and cockroaches
c) MCTS better in multi-pv mode, I highly doubt it

Daniel

Awesome Post. Ruthless Dissection.
Game, Set and Match !

SzG wrote: ↑Tue May 15, 2018 10:16 am

Hurnavich wrote: ↑Tue May 15, 2018 8:45 am Hi,

A little confused here will this run as normal Komodo and are you saying ran as normal it is about 5 elo stronger as none MCTS?
or is it better to stick with 11.3.1 for normal play?

thanks in advance

5 Elos better than 11.3.1. It will surely show after you have played 1,000,000 games with them.

mjlef wrote: ↑Tue May 15, 2018 1:39 pmIt takes a few thousand games to prove this, not one million.

lkaufman wrote: ↑Tue May 15, 2018 7:33 am

shrapnel wrote: ↑Tue May 15, 2018 6:55 am

Daniel Shawul wrote: ↑Mon May 14, 2018 9:53 pm
b) MCTS better against humans. I bet standard komodo is much better against computers, humans and cockroaches
c) MCTS better in multi-pv mode, I highly doubt it

Daniel

Awesome Post. Ruthless Dissection.
Game, Set and Match !

Just to address two points (many of the others were just speculation), I agree that standard Komodo is stronger against humans, but I would bet that Komodo MCTS is stronger against humans than similarly rated alpha-beta engines such as (assuming my 3000 CCRL 40/40 estimate is accurate) Gull 2 or Arasan 20.5. It is surely a better strategy against fallible humans not to assume they will see things exactly as the engine does. Maybe Komodo MCTS has to get within 100 or 150 elo of normal Komodo before it will be tougher for humans to beat. On the second point, again we don't claim that Komodo MCTS is currently superior to normal Komodo in multi-pv mode, only that the elo gap shrinks drastically as the number of moves analyzed increases, and may vanish entirely with all moves analyzed. So here too, when and if we get within a couple hundred elo of normal Komodo, the MCTS version should be stronger with enough moves being analyzed.

I just want to add that I'm surprised that Daniel appears to be so negative about MCTS, when I will freely state that his relative success in making a Scorpio MCTS not too far in strength from normal Scorpio was a significant factor in our decision to devote a month to this project. Would it really be so surprising if we also got Komodo MCTS up to say 100 elo below the level of normal Komodo in a few months?

SzG wrote: ↑Tue May 15, 2018 2:46 pm

mjlef wrote: ↑Tue May 15, 2018 1:39 pm

SzG wrote: ↑Tue May 15, 2018 10:16 am

5 Elos better than 11.3.1. It will surely show after you have played 1,000,000 games with them.

Komodo 12 is about 5 elo stronger than Komodo 11.3.1 in normal (non MCTS) mode. It takes a few thousand games to prove this, not one million.

Mark

Well, one million may have been an exaggeration but if you study the 40/4 CCRL list you can find that it requires about 12,000 games to bring down the error margins to about 6 (in the case of SlowChess), which by square adding them is still almost 9 for two engines. To prove a 5 Elo difference the error margin of each engine must stay below about 3, so I am sure that the required number of games will be close to 100,000 for each engine.