Houdini 3

[beram]

That looks promising for Komodo 4.x. Just out of curiosity, according to your own tests, how much stronger you consider Houdini 2.0 than Houdini 1.5 on this TC ?

[lkaufman]

By "this time control" I suppose you mean the 15' + 15" with ponder on? I would say based on the available evidence that at that level there is no measurable difference in strength between the two Houdini versions on one core. Probably on 6 cores Houdini 2 is a few elo stronger than 1.5; there seems to be some evidence that MP scaling was improved in Houdini 2.

[Sedat Canbaz]

By "this time control" I suppose you mean the 15' + 15" with ponder on? I would say based on the available evidence that at that level there is no measurable difference in strength between the two Houdini versions on one core. Probably on 6 cores Houdini 2 is a few elo stronger than 1.5; there seems to be some evidence that MP scaling was improved in Houdini 2.

Wow...very good news !!!

Finely we will see Komodo mp as a strong and serious opponent against Houdini

Btw,are the games (which are played in your test) available for downloading ?

Thanks,
Sedat

[lkaufman]

By "this time control" I suppose you mean the 15' + 15" with ponder on? I would say based on the available evidence that at that level there is no measurable difference in strength between the two Houdini versions on one core. Probably on 6 cores Houdini 2 is a few elo stronger than 1.5; there seems to be some evidence that MP scaling was improved in Houdini 2.

Wow...very good news !!!

Finely we will see Komodo mp as a strong and serious opponent against Houdini

Btw,are the games (which are played in your test) available for downloading ?

Thanks,
Sedat

I could make them available when the test is done. Bear in mind, these are single core games, so perhaps not as interesting as MP games. Komodo is now 4 games ahead.

[Sedat Canbaz]

By "this time control" I suppose you mean the 15' + 15" with ponder on? I would say based on the available evidence that at that level there is no measurable difference in strength between the two Houdini versions on one core. Probably on 6 cores Houdini 2 is a few elo stronger than 1.5; there seems to be some evidence that MP scaling was improved in Houdini 2.

Wow...very good news !!!

Finely we will see Komodo mp as a strong and serious opponent against Houdini

Btw,are the games (which are played in your test) available for downloading ?

Thanks,
Sedat

I could make them available when the test is done. Bear in mind, these are single core games, so perhaps not as interesting as MP games. Komodo is now 4 games ahead.

Yes...i have no much interest over single core games (played by mp engines)
I thought that you are testing Komodo mp

But however,i will be very glad if you inform us about Komodo mp results

Best,
Sedat

[geots]

[quote="mclane & Stockfish.

Maybe my explanation is even more simple but true: Rybka, Ippo & Co are simply better. The weaker engines just benefit from the bigger draw rates at big depths/long time controls.

This might be a factor for Houdini, but certainly Ivanhoe (Ippo), Rybka, and all other Ippo-related programs are not stronger than Komodo and not measurably stronger than SF except at bullet chess, so this cannot be the explanation here.

There seem to be two theories to explain the observed scaling behavior:
1. Komodo (and perhaps also SF) are more intelligent but slower, and that this tradeoff usually (but not always) favors the fast programs in blitz and the smart programs at long time controls.
2. For whatever reason, the search in Komodo (and perhaps SF) scales better than the search in the Rybka/Ippo family.

Both could be true. If the second is true, can anyone suggest WHY SF might scale better in search than Ivanhoe et al?

I don't think ''intelligence'' is the right word. There are two different approaches to the search in my view.

Ippo family engines have a search for which they find tactical moves in a very short time.
I think it is a question of search (and evaluation). Like humans. Some humans search mainly for tactical moves.
I'm not a programmer but i think it depends by the evaluation that an engine gives to each move that it analyses.
In this way it could happen that all the moves that don't give a ''break'' in the evaluation are discarded or analysed less time (and perhaps they are good positional moves).
If this would be true, it is logical that these engines find more often and more quickly tactical moves.

On the other hand Komodo plays mainly positionally.
Komodo often manouevres his pieces and pawns for many moves without playing any tactical move.

Probably it depends by the fact that Komodo gives a special evaluation on some moves that lead to some positional patterns.

I mean, probably during the search, Komodo seems to make the opposite of the engines of the ippo family.

Then Komodo analyzes more the ''positional moves'' (probably because it consider them better than other moves with its evaluation) and gives less time or discards some tactical moves.

In fact, sometimes Komodo overlooks some tactical moves, and it depends by the search or sometimes by the evaluation.

If all i wrote would be true, it is logical that in very fast time control, ippo family engines excel.

With more time to think, a positional engine like Komodo has 2 advantages:

1. What Komodo overlooks or evaluates bad in a few seconds can be seen well with more time available.

2.The weight of positional play of Komodo increases a lot and becomes more important than its (relative) weakness in tactics.

Generally speaking i think that the strenght of Komodo is that it plays nice positional and logical moves. It seems that it uses the search just to verify that everything is ok.

On the other hand the strenght of ippo family engines is the approach of the search (i think built mainly for tactical moves) and the evaluation that allows them to find sometimes tactical moves apparently hidden.

In long time control, especially in very long time control, tactics ability is almost useless because the opponent (if he has a good search and evaluation) has all the time to see it.

So the thing that has more importance is the positional/strategical sensibility.


Best Regards

I am coming to the conclusion that the relative weakness of both Komodo and Stockfish at bullet speeds vs. slower tests (which is obvious if you compare any bullet list with say the IPON list) is not mainly a question of slowness due to more or slower evaluation. I say this because I noticed that Rybka 4.1 shares the same behavior as Ivanhoe and all the other top programs (except Komodo and Stockfish) in that it is also relatively stronger at bullet speeds than at normal speeds. If we make the assumption that Rybka 4.1 is at least not substantially "dumber" than Rybka 3, I can say that the evaluation I did for Rybka 3 was also rather "smart" and "slow", rather like Komodo, yet Rybka 4.1 shares the behavior of Ippo and family. So I can only conclude that the behavior is due to some aspect of the Ippo and Rybka search that is not present in either Komodo or Stockfish. Since we have at least tried almost every search idea in Ippo (and rejected many of them), I can't guess what that aspect could be. If we can solve this mystery perhaps we can make Komodo as competitive with Houdini at bullet and blitz chess as it already appears to be at longer time controls.

Interesting that you mention the Ippos, Rybka, Houdini- and, I would have to imagine you include Critter. The reason I imagine you use Critter is because you have no choice, really. What amazes me about this pile of engines you have gathered- you don't include Strelka. You can make a great argument for leaving it out- but the argument becomes futile and silly when you leave all the others in. You want to have it both ways. Sorry. Strelka is not MP- but it uses 32bit and 64bit. You use Komodo with either- which noting its lack of strength in 32-bit would have to be 64. We shall both monitor a 50 game match with Komodo on 1cpu vs. Strelka 5.1. You pick the time limit of your choice. In a 50 game match you can start with white- and I will give you the gift of a win in game 1 with no moves. It won't amount to a dot in the universe. Any time control of your choice- I am sorry- Strelka 5.1 will go thru Komodo like Sherman thru Georgia. But that is not being cruel- no other engines can play with him either- outside of Houdini and Critter. And the last match, Critter got his butt kicked. Look, when I say any time limits of your choice- that is up to and thru 40/40. No slower- simply because it is a waste of my time. As it will change nothing, and at 63 time becomes a valuable commodity.


Best regards Larry- it's certainly nothing personal,

george


Please no bars and graphs and formuli that are supposed to show where one program intersects with another. There has already been too much talk and not enough "walk". Let's get the match on.

[Uri Blass]

I understood from posts of other people that strelka5.1 is very similiar to houdini1.5 1 cpu in the move choice so it is not interesting to test komodo against strelka5.1(note that it is not based on my observations and I never even tried strelka5.1 after reading these claims).

[lkaufman]

[quote="mclane & Stockfish.

Maybe my explanation is even more simple but true: Rybka, Ippo & Co are simply better. The weaker engines just benefit from the bigger draw rates at big depths/long time controls.

This might be a factor for Houdini, but certainly Ivanhoe (Ippo), Rybka, and all other Ippo-related programs are not stronger than Komodo and not measurably stronger than SF except at bullet chess, so this cannot be the explanation here.

There seem to be two theories to explain the observed scaling behavior:
1. Komodo (and perhaps also SF) are more intelligent but slower, and that this tradeoff usually (but not always) favors the fast programs in blitz and the smart programs at long time controls.
2. For whatever reason, the search in Komodo (and perhaps SF) scales better than the search in the Rybka/Ippo family.

Both could be true. If the second is true, can anyone suggest WHY SF might scale better in search than Ivanhoe et al?

I don't think ''intelligence'' is the right word. There are two different approaches to the search in my view.

Ippo family engines have a search for which they find tactical moves in a very short time.
I think it is a question of search (and evaluation). Like humans. Some humans search mainly for tactical moves.
I'm not a programmer but i think it depends by the evaluation that an engine gives to each move that it analyses.
In this way it could happen that all the moves that don't give a ''break'' in the evaluation are discarded or analysed less time (and perhaps they are good positional moves).
If this would be true, it is logical that these engines find more often and more quickly tactical moves.

On the other hand Komodo plays mainly positionally.
Komodo often manouevres his pieces and pawns for many moves without playing any tactical move.

Probably it depends by the fact that Komodo gives a special evaluation on some moves that lead to some positional patterns.

I mean, probably during the search, Komodo seems to make the opposite of the engines of the ippo family.

Then Komodo analyzes more the ''positional moves'' (probably because it consider them better than other moves with its evaluation) and gives less time or discards some tactical moves.

In fact, sometimes Komodo overlooks some tactical moves, and it depends by the search or sometimes by the evaluation.

If all i wrote would be true, it is logical that in very fast time control, ippo family engines excel.

With more time to think, a positional engine like Komodo has 2 advantages:

1. What Komodo overlooks or evaluates bad in a few seconds can be seen well with more time available.

2.The weight of positional play of Komodo increases a lot and becomes more important than its (relative) weakness in tactics.

Generally speaking i think that the strenght of Komodo is that it plays nice positional and logical moves. It seems that it uses the search just to verify that everything is ok.

On the other hand the strenght of ippo family engines is the approach of the search (i think built mainly for tactical moves) and the evaluation that allows them to find sometimes tactical moves apparently hidden.

In long time control, especially in very long time control, tactics ability is almost useless because the opponent (if he has a good search and evaluation) has all the time to see it.

So the thing that has more importance is the positional/strategical sensibility.


Best Regards

I am coming to the conclusion that the relative weakness of both Komodo and Stockfish at bullet speeds vs. slower tests (which is obvious if you compare any bullet list with say the IPON list) is not mainly a question of slowness due to more or slower evaluation. I say this because I noticed that Rybka 4.1 shares the same behavior as Ivanhoe and all the other top programs (except Komodo and Stockfish) in that it is also relatively stronger at bullet speeds than at normal speeds. If we make the assumption that Rybka 4.1 is at least not substantially "dumber" than Rybka 3, I can say that the evaluation I did for Rybka 3 was also rather "smart" and "slow", rather like Komodo, yet Rybka 4.1 shares the behavior of Ippo and family. So I can only conclude that the behavior is due to some aspect of the Ippo and Rybka search that is not present in either Komodo or Stockfish. Since we have at least tried almost every search idea in Ippo (and rejected many of them), I can't guess what that aspect could be. If we can solve this mystery perhaps we can make Komodo as competitive with Houdini at bullet and blitz chess as it already appears to be at longer time controls.

Interesting that you mention the Ippos, Rybka, Houdini- and, I would have to imagine you include Critter. The reason I imagine you use Critter is because you have no choice, really. What amazes me about this pile of engines you have gathered- you don't include Strelka. You can make a great argument for leaving it out- but the argument becomes futile and silly when you leave all the others in. You want to have it both ways. Sorry. Strelka is not MP- but it uses 32bit and 64bit. You use Komodo with either- which noting its lack of strength in 32-bit would have to be 64. We shall both monitor a 50 game match with Komodo on 1cpu vs. Strelka 5.1. You pick the time limit of your choice. In a 50 game match you can start with white- and I will give you the gift of a win in game 1 with no moves. It won't amount to a dot in the universe. Any time control of your choice- I am sorry- Strelka 5.1 will go thru Komodo like Sherman thru Georgia. But that is not being cruel- no other engines can play with him either- outside of Houdini and Critter. And the last match, Critter got his butt kicked. Look, when I say any time limits of your choice- that is up to and thru 40/40. No slower- simply because it is a waste of my time. As it will change nothing, and at 63 time becomes a valuable commodity.


Best regards Larry- it's certainly nothing personal,

george


Please no bars and graphs and formuli that are supposed to show where one program intersects with another. There has already been too much talk and not enough "walk". Let's get the match on.

If you are talking about Komodo 4, you are probably right that Strelka 5.1 would be favored in a match at 40/40, since I understand it to be just a slightly weaker relative of Houdini 1.5. But although I have never tested Strelka, I would be willing to make a substantial wager that the current Komodo would win a hundred game match at that time control on an i7 or similar 64 bit SSE4 machine, if we can find a neutral trustworthy party to hold the stakes and run the match. I say this based on the assumption that Strelka 5.1 is at least marginally weaker than Houdini (1.5 or 2.0, I don't care); if this is not true please enlighten me. I realize that this offer is a bit unfair to you, as you don't have any Komodo newer than 4.0.

[MM]

By "this time control" I suppose you mean the 15' + 15" with ponder on? I would say based on the available evidence that at that level there is no measurable difference in strength between the two Houdini versions on one core. Probably on 6 cores Houdini 2 is a few elo stronger than 1.5; there seems to be some evidence that MP scaling was improved in Houdini 2.

Wow...very good news !!!

Finely we will see Komodo mp as a strong and serious opponent against Houdini

Btw,are the games (which are played in your test) available for downloading ?

Thanks,
Sedat

I could make them available when the test is done. Bear in mind, these are single core games, so perhaps not as interesting as MP games. Komodo is now 4 games ahead.

Does it mean that you are not testing MP?
This would mean that the release of MP is far, because you need much time to test MP, is it so?



Regards

[lkaufman]

We're redoing MP because the first version wasn't good enough to release. But the new version should be ready in a few days. It's true that getting MP optimized would take some time, but we don't have to do that for the first release, as long as it is not too far from par. We can have an optimized MP for the next version; our versions aren't spaced so far apart as others.