To Larry Kaufman

[RJN]

I am not the only one who has noticed many more SF users lately, hopefully Komodo update will restore the balance


One month ago, after a period of Komodo dominance, our statistics already showed a comeback for Stockfish. Then version SF 7 came out and even more players switched from Komodo to Stockfish, just be curious and giving it a try. This was our users' choice in the 16th Engine Rapid Masters:

21 Stockfish
11 Komodo
02 Stockfish & Komodo
02 Synapse*

http://www.infinitychess.com/Page/Publi ... spx?id=251

a few months before:

The times they are a changin‘... was a famous song by Bob Dylan 1964. Well, he was not singing about computer chess, but the title comes up to my mind and is in my ear, whenever any big changes are happening, and during the 14th Engine Rapid Masters I felt we are just witnessing a remarkable change, that could also be called a comeback by Stockfish. I don’t know, why this happens, but it showed already in the 13th EM that Komodo was losing its dominance in rapid chess. More players than in the previous tours switched from Komodo to Stockfish, and this is what happened in the 14th EM too. Komodo users were still a majority of 21 players, but Stockfish users were 14, while 4 players were alternately using both engines in the tour. Cryptic as usual had only one representative.

http://www.infinitychess.com/Page/Publi ... spx?id=244

[cdani]

The problem is that both Komodo and Stockfish have tuned their evals at bullet-chess levels, for the obvious reason that it's impossible to play enough games to test changes at long time controls.

Do you tune like Stockfish, only searching for minimum LOS to accept a patch?

If it's the case, Komodo is stronger than Stockfish at long time control only by luck, and the same for short time control. It's my conclusion seeing the results of how I tune. Of course I cannot be 100% sure, but is what I think.

[mjlef]

Hi Mark!

We are actively working on Komodo every day.

That's good to read.
Could you please revise the old bug of not working "Persistent Hash" too?
You remember our many emails about the issue?

Here:
http://www.talkchess.com/forum/viewtopi ... 87&t=59143
is one more test postion one would like to use MV- mode, that's annoying with komodo.

If you start with 2 variants, 34.h5 is second best only for a short time, then it's out of the 2, you try with 3, same thing.

But the really bad thing about komodo is, as I wrote here once and again sooner too, that every time you change the number of variants in MV- mode, komodo deletes its hahs and you start all over with empty hash again.

Almost every GUI- command (e.g. even changing the output- language) given during running analysis deletes komodo's hash, and so does "Save Hash" for sure too.

Could you have a look at the issue once in a while again?
I guess you might know the problem well since "Persistent Hash" in komodo- options exists, don't you?

Still hoping

Persistent Hash problems have to do with GUIs that sent hash Clear commands when they should not. I have contacted the GUI authors that seem to do this and asked for an fix. At lease one says they are working on it. I also have made some changes in how Komodo "ages" the hash table with Persistent Hash, which improves it performance. I am trying to code around the GUI bugs, but not everything is possible without someone not liking a change. I have some change sin Analysis Mode also, but will wait to describe them closer to a release.

[mjlef]

and are working on some very interesting ideas which are new

Some ideas inspired by AlphaGo?

No, at leas not yet. But Larry and I often talk about Monte Carlo methods and want to do some work on them in the future.

[Werewolf]

and are working on some very interesting ideas which are new

Some ideas inspired by AlphaGo?

No, at leas not yet. But Larry and I often talk about Monte Carlo methods and want to do some work on them in the future.

On big systems with many cores that could be interesting. In a month or two Intel will release new Xeons and just 2 of them will provide 44 physical cores...

I've always felt Monte Carlo hasn't reached its full potential in chess.

[lkaufman]

The problem is that both Komodo and Stockfish have tuned their evals at bullet-chess levels, for the obvious reason that it's impossible to play enough games to test changes at long time controls.

Do you tune like Stockfish, only searching for minimum LOS to accept a patch?

If it's the case, Komodo is stronger than Stockfish at long time control only by luck, and the same for short time control. It's my conclusion seeing the results of how I tune. Of course I cannot be 100% sure, but is what I think.

We tune somewhat like SF, but not so rigidly, much more judgement is involved in deciding what gets promoted. But you are overlooking that it's not just a question of tuning eval parameters, it's which eval terms you have that is more important. We have eval terms that are just not in stockfish, perhaps because they don't have a grandmaster or even international master advising them as far as I know.
But the gap in relative strength at bullet vs. long tc is too large to attribute to these extra terms in my opinion.

[cdani]

We tune somewhat like SF, but not so rigidly, much more judgement is involved in deciding what gets promoted. But you are overlooking that it's not just a question of tuning eval parameters, it's which eval terms you have that is more important. We have eval terms that are just not in stockfish, perhaps because they don't have a grandmaster or even international master advising them as far as I know.

It's what I try to do, but with my limited strength of just 2100. I add eval terms that possibly are not in any other engine, the ones that I think will help more; you can see the list of changes I publish here at Talkchess. Deciding patches like this has some higher possibilities to help more. So I'm not overlooking this, just I didn't told it.

But the gap in relative strength at bullet vs. long tc is too large to attribute to these extra terms in my opinion.

Here I think that is like I told. I see relative often that changing a parameter sometimes helps more at longer time control, even if it hurts at stc, so of course I keep the change. It happened enought times to be able to say that one can tune for stc and win may be 50 elo relative easy, losing something undefined at tournament level.

[TShackel]

We are actively working on Komodo every day. Since it is impossible to predict when we will have enough elo and features, I don't want anyone holding me to any estimates of a release date. But it we would like to have another Komodo 9.x release, followed by Komodo 10 in the next several months. But we have no dates yet, since we want more elo and are working on some very interesting ideas which are new.

Mark

Hi Mark,

I won't hold you to a release date, I just wondered if you had anything definite to share. It's interesting you're thinking of another 9.x release instead of 10.0 right away.

I'm also very glad to hear you're working on "very interesting ideas that are new". It's alwayss nice to hear innovative programming ideas that improive the strategic in closed positions or attacking play of komdoo.

Keep up the good work and thanks for checking in.

Sincerely,

Tim.

[TShackel]

We would very much like to improve Komodo's blitz/bullet chess, since that's the one area where Stockfish seems to have a small edge. But we don't yet know why Stockfish is stronger at bullet level play, so it's hard to fix this except by generally improving Komodo. We can say it's because our better eval is a bit slower, but we're not that much slower than Stockfish, so there is something else going on that we would love to identify and fix. One clue: we have never been able to make "probcut" work for us, although it seems to work fine in stockfish. No idea why this is so.

Thanks for the info larry. Yea I suppose it would be nice to be superior to stockfish in both long and short time control to be truly called superior. But I of course prefer results from longer time control games, as that's more serious chess. But losing at shorter time controls can't be fun either.

Sincerely,

Tim.

[lkaufman]

We tune somewhat like SF, but not so rigidly, much more judgement is involved in deciding what gets promoted. But you are overlooking that it's not just a question of tuning eval parameters, it's which eval terms you have that is more important. We have eval terms that are just not in stockfish, perhaps because they don't have a grandmaster or even international master advising them as far as I know.

It's what I try to do, but with my limited strength of just 2100. I add eval terms that possibly are not in any other engine, the ones that I think will help more; you can see the list of changes I publish here at Talkchess. Deciding patches like this has some higher possibilities to help more. So I'm not overlooking this, just I didn't told it.

But the gap in relative strength at bullet vs. long tc is too large to attribute to these extra terms in my opinion.

Here I think that is like I told. I see relative often that changing a parameter sometimes helps more at longer time control, even if it hurts at stc, so of course I keep the change. It happened enought times to be able to say that one can tune for stc and win may be 50 elo relative easy, losing something undefined at tournament level.

What is stc and ltc for you? How can you ever test anything at a level slower than blitz (say 3' + 2")? For Stockfish, ltc is still bullet chess.