Komodo 4 on long time control

[mclane]

i doubt that don is frustrated robert. my results show komodo ahead of houdini.

[Frank Quisinsky]

Hi Don,

I can't see that my list have other results as the others. In the most of cases the others don't have enough opponents or games.

IPON is playing with the half on time I used.

But if you compare with IPON:

Houdini 2.0c x64 in SWCR = 3.022 ELO
Houdini 2.0 x64 in IPON = 3.022 ELO

I can't see that IPON is playing with SSE.

Komodo 3.0x x64 in SWCR = 2.962 ELO
Komodo 3.0x x64 SSE in IPON = 2.966 ELO

Again, in the most of cases the others don't have enough games or opponents. In the most of cases in CCRL.

CEGT is great but not to compare because CEGT used ELOstat. Bayesian will give you a higher ELO with lesser draw games.

Let us look for Komodo 3.0 x64

In SWCR Shredder 12 have 2.800 ELO
I calculate with ELOstat
In CEGT Shredder 12 have 2.982

= +182 ELO

Komodo 3.0 x64 = 3.145 - 182 = 2.963

With other words ...

Komodo 3.0 x64 in
CEGT = 2.963
IPON = 2.966
SWCR = 2.960

And now let us look in CCRL ...

Shredder 12 x64 1 Core = 3.059 = -259 ELO
Komodo 3 x64 SSE = 3.216 - 259 = 2.957 ELO

Again with other words ...

Komodo 3.0 x64 SSE (CCRL) = 2.957
Komodo 3.0 x64 SSE (IPON) = 2.966
Komodo 3.0 x64 (SWCR) = 2.960
Komodo 3.0 x64 (CEGT) = 2.963

Nice results and all is wonderful

Best
Frank

[Frank Quisinsky]

Hi Don,

I believe we have the same opinion in case of Houdini
Can't see any bad things in your comments and like what you wrote here.

Thats what I mean.
I wish me a number 1 where the programmer is present for any discuss and to be able to find out own programmings.

Have a nice evening!
Must sleeping ... had a hard week at work and after my work in creating a Fritz book with all my material.

With other words, for the next days I have enough computer chess

Best
Frank

[Frank Quisinsky]

Hi,

yes the perfect book is great, I think one of the best available books. With more games you will see that Komodo is great in endgames, in my opinion here the number 1 with 1 Core.

You can make a very little experiment (this one can be make fast).
Load the SWCR database and sort the games via ChessBase with number of moves. And now sort the Houdini and Komodo games. You can search games Komodo lost if more as 80 moves are done.

Each other statistic you will create here you will have the same opinion. Komodo should the first engine for analyzes in endgames if you are useing 1 core.

Best
Frank

[MM]

i doubt that don is frustrated robert. my results show komodo ahead of houdini.

I think that Komodo, sooner or later, will overtake Houdini at long time control or tournament time control. I mean 90'+30s or more that is much more than 40/40. I have Houdini 2.0, i see how it plays, its strenght is the tactical ability and the agressive style. I think that a strategical/positional engine like Komodo is stronger more than the time control is longer.

Regards

[mclane]

but houdini, similar to all other clone engines, is often stupid in certain positions where search cannot help at all.

komodo is a different kind of beast.

this also counts for hiarcs btw.or stockfish.


all those programs KNOW more than houdini/rybka...

and sometimes it is important to KNOW something.

[Houdini]

I think that Komodo, sooner or later, will overtake Houdini at long time control or tournament time control. I mean 90'+30s or more that is much more than 40/40. I have Houdini 2.0, i see how it plays, its strenght is the tactical ability and the agressive style. I think that a strategical/positional engine like Komodo is stronger more than the time control is longer.

Regards

It is always convenient to speculate about very long TC, say 1 day/move, as nobody will ever be able to verify any claim you make.
What is obvious is that at longer TC less errors will be made and more draws will be played. This means that very naturally, as the weaker engine makes less mistakes, the Elo gap narrows at longer TC. This is not related to any "scaling" of engines, a similar situation is observed in human chess (higher skill level = more draws).

In case you're interested in a real long TC result, see Pal Larkin's Houdini 1.5-Komodo 3 match at 40 moves in 120 min + 20 moves in 60 min + remainder in 30 min plus 10 seconds per move. See http://www.talkchess.com/forum/viewtopic.php?t=40215.
The end result after 100 long games was 56-44 for Houdini 1.5, which corresponds to +42 Elo (56 %).

While the number of games is clearly insufficient to make any accurate Elo estimate (again the basic problem of using very long TC!), this data point shows an Elo difference that is very similar to the value found in the IPON and SWCR lists played at about 10 times faster pace.

Note that the same long TC myths were being spread about Rybka 4, until the TCEC tournament demonstrated that even at very long TC (and on powerful 6-core hardware) Houdini was clearly superior, with an Elo difference that was not very different from the value found for much faster games.

Cheers,
Robert

[Lion]

We will almost surely release before mid-Dec. The delay has been in getting MP working, as Don gets sidetracked making program improvements. Today we made a nice gain that should put us too close to call with Houdini at 40/40 based on CCRL ratings, though we will probably not catch Houdini at blitz levels with this release. The CCRL and CEGT tests show us scaling much better than Houdini, and also show Houdini 2.0 scaling much worse than Houdini 1.5. In fact we may pass Houdini 2.0 on some lists without passing Houdini 1.5! This lower rating of 2.0 vs. 1.5 is why people assume that Houdini is static.

Most (in fact all except 2) rating lists show a definite improvement for Houdini 2, from single-core to 6 CPU, typically about 20 Elo points.
For example, take a look at the most recent result: the SWCR rating that was published today for Houdini. Houdini 2.0c shows +24 Elo and is 60 Elo ahead of Komodo 3.

Hi !
I have lot of respect for the work you are doing on Komodo MP and will very likely purchase it.

Nevertheless, I have never seen reel data with pgn database to backup the comment that Houdini performs less good at longer time controls. Can you provide and such data ?

Also looking at CCRL and other rating lists, they are not playing 3min games, so the over 50 ELO lead Houdini has on the current field is IMHO not going to disappear at longer time controls.

As for now, Houdini is the best at any time control, but I am looking forward to see Komodo overpassing it !

Respectful regards
SWCR is played at 40/10 with Ponder ON. This is not miles away from the 40/40 with ponder off. On the average the difference in search depth must be a single ply, there is no reason to expect any fundamental difference in performance. Why do you pretend that this would be the case?

The bottom-line is that you cherry-pick one or two rating list results - while dismissing all the other results as "blitz" - to make claims about the strength or scaling of Houdini - about which you know very little (as you admit yourself). Please stop the nonsense.

Robert

It has come to our attention that Komodo benefit's much more from the SSE instructions that other programs, and that in the SWCR testing Frank uses the lowest common denominator, which is the komodo binary with SSE off. Houdini automatically switches it on so it is not a fair test at all. For this reason and others I don't fully trust the SWCR and I mean no disrespect to the honorable Frank who is a good guy and I consider a friend, but I don't believe his test is run scientifically. In fact I don't know if there are many who have the background to understand what a fair test is, even though they do their best you can often see their own superstitions and biases affecting the results, usually unintended.

We are not cherry picking any more than you are, this is the second time that you picked the most favorable list for your advertising campaign.

However you slice and dice it, there is no longer much difference between Houdini and other programs and Houdini will be overtaken soon unless you get off your butt and start making some more improvements to Ivanhoe. We have already closed much if not all the gap and we continue to make progress and we do it mostly with our own ideas and always with our own code, not someone else's. So what users will get with Komodo is a program that is substantially different that all the other Ippo's such as Houdini.

There IS a huge discrepancy based on time whether you refuse to acknowledge it or not. Screaming loudly doesn't change anything. We have run a lot of tests with Houdini which shows without any doubt that Houdini has major superiority at really fast time controls and this superiority continues to drop sharply with longer time controls. It's also the same exact behavior see with the Ippo programs which Houdini is based on. It's possible that Houdini suddenly starts going back up at 40/2 hours for some bizarre reason we don't understand, but there is absolutely no question that is true with time controls less than 40/10.

You seem to think that making aggressive denials and strong statements is something that will make people back off or think that you are honorable but that doesn't work for most of us. You used this same tactic with your vehement denials that Houdini has anything to do with Ivanhoe (other than just some ideas, a statement that you think "covers" your lie) which any idiot can see is not true. In the early days of Houdini you had promised to "release the sources at some time in the future" and we all had to laugh knowing that you could NEVER do that, funny how that has never happened and now you are trying to peddle it off as an original work.

[MM]

I think that Komodo, sooner or later, will overtake Houdini at long time control or tournament time control. I mean 90'+30s or more that is much more than 40/40. I have Houdini 2.0, i see how it plays, its strenght is the tactical ability and the agressive style. I think that a strategical/positional engine like Komodo is stronger more than the time control is longer.

Regards

It is always convenient to speculate about very long TC, say 1 day/move, as nobody will ever be able to verify any claim you make.
What is obvious is that at longer TC less errors will be made and more draws will be played. This means that very naturally, as the weaker engine makes less mistakes, the Elo gap narrows at longer TC. This is not related to any "scaling" of engines, a similar situation is observed in human chess (higher skill level = more draws).

In case you're interested in a real long TC result, see Pal Larkin's Houdini 1.5-Komodo 3 match at 40 moves in 120 min + 20 moves in 60 min + remainder in 30 min plus 10 seconds per move. See http://www.talkchess.com/forum/viewtopic.php?t=40215.
The end result after 100 long games was 56-44 for Houdini 1.5, which corresponds to +42 Elo (56 %).

While the number of games is clearly insufficient to make any accurate Elo estimate (again the basic problem of using very long TC!), this data point shows an Elo difference that is very similar to the value found in the IPON and SWCR lists played at about 10 times faster pace.

Note that the same long TC myths were being spread about Rybka 4, until the TCEC tournament demonstrated that even at very long TC (and on powerful 6-core hardware) Houdini was clearly superior, with an Elo difference that was not very different from the value found for much faster games.

Cheers,
Robert

Hello Mr Houdart,

first of all, i never talked about 1 day/move. I talked about tournament TC or longer, meaning 90'+30s or 120'+30' or something.

second: i didnt speculate, why should i? i m not a part of Komodo's team. I have no interest to try to convince people that Komodo is the future and your engine will be weaker soon. I just expressed my opinion, totally indipendent from any other.

About the rest:

yes, more time to think, less errors will be made, high chances of more draws but probably also more chances to win for Komodo.

Yes i agree, this not necessary related to the scaling.

About the two matches against Rybka on TCEC (that i observed) it is true that you came out as a clear winner but it also true that in some games Houdini showed a very bad treatment of some endgames and a big problem of evaluation. I refer especially at a game in which Rybka (with white) had a free pawn and placed its rook behind forcing Houdini's rook to block phisycally that pawn.
Usually those position, as well known, are easily won by white that only needs to centralize the king, make a zugzwang for black and go and grab all pawns of the other side of the board (black rook is blocked to prevent the pushing of white' pawn).

Well, i remember that for many moves Houdini gave just a little advantage to white and Rybka scored +2, +3 ect.

i think that Houdini should need some important chess knowledge about endgames.

Anyway there are 2 things: 1st: we are talking about Rybka, not Komodo.
2nd. the number of games of TCEC was very limited. If i remember well Rybka, at start of the second match was leading by 4 wins to 1.

You teach me that around 500/1000 games are needed.

Now the most important fact:

i am talking about the next Komodo. I know, everybody knows that actually, Houdini is clearly stronger than Komodo.
Lately Komodo's authors claimed improvements for 30 points elo or more.

If it is true, your engine risks to lose the throne of the best engine in long time control because the gap will reduce strongly.

And, last thing, the fact that your engine is mainly tactical drives Houdini to be stronger at fast TC.
Komodo has totally another style of playing and long time control advantages it.

it is like a competition on running 400 metres between the world champion of 100 metres and the world champions of 800 metres.

I agree with you, of course, that, for now, you lead, and, for now, there are only words about future, results will make the word ''end''.

p.s. Please release Houdini 3, i want it not less than i want komodo 4.

Best regards

[tano-urayoan]

i didnt speculate, why should i

Are you serious almost all of your threads are speculation. Look at the title of this thread Komodo 4 does not yet exist and yet you are writing about it.