Houdini 3

[Mike S.]

komodo and stockfish are intelligent evaluation programs.
ippo/ivanhoe or rybka etc. are stupid search programs.

Simple explanations are always popular, but I disagree. Is there ANY statistical prove of the above? Is there any rating list, tournament or test suite result which can back up this view? I don't think so. I think it is just a comfortable, simple approach to escape from a difficult complexity.

But is it true?

IF we have stupid and intelligent programs, the logic is wrong that the stupid ones should be stronger in bullet. The opposite would be true. Because if the tendency is that deep search beats knowledge, that should happen the more the bigger the depths are. But that is not at all what we observe if we compare Ippos & Rybka to Komodo & Stockfish.

Maybe my explanation is even more simple but true: Rybka, Ippo & Co are simply better. The weaker engines just benefit from the bigger draw rates at big depths/long time controls.

[Uri Blass]

komodo and stockfish are intelligent evaluation programs.
ippo/ivanhoe or rybka etc. are stupid search programs.

Simple explanations are always popular, but I disagree. Is there ANY statistical prove of the above? Is there any rating list, tournament or test suite result which can back up this view? I don't think so. I think it is just a comfortable, simple approach to escape from a difficult complexity.

But is it true?

IF we have stupid and intelligent programs, the logic is wrong that the stupid ones should be stronger in bullet. The opposite would be true. Because if the tendency is that deep search beats knowledge, that should happen the more the bigger the depths are. But that is not at all what we observe if we compare Ippos & Rybka to Komodo & Stockfish.

Maybe my explanation is even more simple but true: Rybka, Ippo & Co are simply better. The weaker engines just benefit from the bigger draw rates at big depths/long time controls.

I see no evidence that Rybka is better than Komodo.
Komodo seems to be stronger than Rybka at all time controls(maybe not in bullet but the testers in the rating lists do not test bullet).


http://www.husvankempen.de/nunn/40_40%2 ... liste.html
3 Komodo 4.0 x64 1CPU 2982 15 15 1229 60.3% 2910
9 Deep Rybka 4.1 x64 1CPU 2965 12 12 1611 59.7% 2897

http://computerchess.org.uk/ccrl/4040/c ... ons_only=1
CCRL 40/40

Komodo 4 64-bit 3220 +35 −35
Komodo 4 64-bit SSE 3219 +17 −17
Rybka 4.1 64-bit 3208 +15 −15

http://computerchess.org.uk/ccrl/404.li ... ons_only=1

4 Komodo 4 64-bit 3225 +11 −11 61.0% −84.7 36.2%
6 Rybka 4.1 64-bit 3205 +12 −12 63.0% −93.8 37.6%

[Mike S.]

I see no evidence that Rybka is better than Komodo.

Correct, but that is just because Rybka 4.1 is many months older than Komodo 4.

I don't like speculations much but in this case, I guess there will not be many who doubt that Rybka 5 will be stronger than Komodo 4. If Komodo 5 will be stronger than Rybka 5, that is a different question!

[Uri Blass]

I can add that I also disagree with simple explanations.

If a program scales better it can be thanks to evaluation that is tuned for long time control but it can be also thanks to other factors.

There is knowledge that is productive for long time control and there is knowledge that is productive for blitz.

I already gave an example for knowledge that is productive in blitz in previous post.

I can also give example to evaluation difference that is productive at long time control(maybe my example is wrong but the same idea can happen with a different example).

evaluating unbalanced positions
What is stronger piece or 3 pawns?

It is possible that the piece is stronger but you need a deep search to find the right moves and without deep search the pawns win because the plan for the pawn is easier and you need only to advance them.

In this case increasing the value of piece can help at longer time control but be counter productive at blitz.

This may not be a good example and I did not do research about it but it is possible that the same idea is correct for other evaluation terms and the evaluation weight that is best for blitz is not best for longer time control.

[mclane]

komodo and stockfish are intelligent evaluation programs.
ippo/ivanhoe or rybka etc. are stupid search programs.

Simple explanations are always popular, but I disagree.

ok.

Is there ANY statistical prove of the above?

if you play very slow games, maybe you let us know

how should it ? the world is testing bullet and blitz games...

Is there any rating list, tournament or test suite result which can back up this view?

no. its my observation from watching 40/120 games and correspondance chess positions with the named programs.

how shall i quantify it when it is about quality ?

I don't think so. I think it is just a comfortable, simple approach to escape from a difficult complexity.

But is it true?

IMO it is.

IF we have stupid and intelligent programs, the logic is wrong that the stupid ones should be stronger in bullet.

really ?

i don't think you are right with your claim.

in bullet games, in blitz games, the fast searchers are FASTER in depth then the intelligent searchers.

so the stupid programs are stronger in bullet, because they make more progress in the search tree. but as the tree explodes, the progress decreases. the slow programs reached a depth where they do not oversee simple tactics and they can even overtake the fast searchers. this happens for a very long time in computerchess. always on different levels, but there is IMO no big difference between Mephisto MM2 beeing good at blitz and Mephisto III being bad at blitz but both making different kind of progress with time passing.

Mephisto III makes bigger progress in playing strength than Mephisto MM2 because MM2 is stupid and Mephisto III is intelligent.

similar things happen with komodo/stockfish and rybka-clones.

only that it is on a much higher level. instead of 3 or 7 plies, it is about 20 plies.

The opposite would be true.

no.

Because if the tendency is that deep search beats knowledge, that should happen the more the bigger the depths are. But that is not at all what we observe if we compare Ippos & Rybka to Komodo & Stockfish.

Maybe my explanation is even more simple but true: Rybka, Ippo & Co are simply better. The weaker engines just benefit from the bigger draw rates at big depths/long time controls.

take hiarcs. hiarcs is very strong on the ipad. but weaker on the pc in relation to the other programs. that is because the architecture of todays pc's completely supports fast searchers. its all about speed and depth .

kind of race between simple CPUs (arm architecture) and Motorolas monster CPUs 680x0.

IMO hiarcs is a much better tool for analysis than rybka is. although rybka might be stronger in bullet or blitz games then hiarcs.
same counts for stockfish/komodo.

in bullet and blitz games, hiarcs is outsearched. it has more knowledge but is outsearched 4-5 plies.

if you let them compute 40/120 games, or even longer, the knowledge suddenly brings a new quality into the whole thing, and the intelligent program can see something BEYOND the horizont the fast searcher cannot see.

[mclane]

I see no evidence that Rybka is better than Komodo.

Correct, but that is just because Rybka 4.1 is many months older than Komodo 4.

I don't like speculations much but in this case, I guess there will not be many who doubt that Rybka 5 will be stronger than Komodo 4. If Komodo 5 will be stronger than Rybka 5, that is a different question!

we should learn to ask: stronger at WHICH TIME CONTROL instead of writing: stronger/better.

a fast searcher is maybe good in blitz and bullet.

but it can be completely different if you give the intelligent program enough time to compensate ...

btw i doubt that there will ever be a rybka 5.

[Mike S.]

I think your viewpoints are valid assumptions and I cannot prove which is (more) right and (more) wrong, considering the stupid/intelligent engines topic.

But I hesitate to apply old explanation patterns of the past to the current situation. We have much bigger depths now, even at short time controls. I cannot prove it but I feel the "stupid/intelligent/fast/slow etc." categories of the past are outdated. - I may be wrong.

For the others who follow this debate, I'd like to mention that Thorsten and I (and many others) are computer chess fans for ~30 years now, and it doesn't seem that more time makes the conclusions we draw each, more similar. Maybe it makes them more different.

Anyway, computer chess remains a great adventure of the mind.

[S.Taylor]

Hi Robert,

Don't forget to dedicate some resources for testing at long time controls too as some people here are implying Houdini 1.5 is stronger than version 2 at the regular OTB time control (40 moves in 120 minutes).

Best regards

Mel, some of the most vocal of the "some people here" have never actually tested Houdini 2 . If you have any doubts about Houdini 2 at long TC, check the neighboring thread, or the very interesting matches of various engines against Deep Junior 13. Houdini 2 scores about 10% better than its closest competitors.

As for Houdini 3, it will do quite well at long TC - some search algorithm improvements will have an increasing effect with higher search depths.

Robert

I hope you do the best you can in long TC.
To me, blitz and quicker are secondary, but not worth investing anything at any expense from long TC.

[Sedat Canbaz]

Basically I would like to know if Komodo (and Stockfish) are weak at bullet chess because we are doing something wrong, or whether we are stronger at slower chess because Ippo/Ivanhoe is doing something wrong (or both). Comments anyone?

In my SCCT Auto232 Rating (4m+2s),Ivanhoe performed in almost same identical playing strength as Komodo,Stockfish,Robopolito

SCCT Auto232 Rating (4m+2s)

Code: Select all

Rank  Name                       Elo    +    -   games score oppo. draws
  7 Stockfish 2.2.2 JA x64 6c    3360   20   20   675   51%  3354   56%
  9 Ivanhoe 46hm x64 6c          3353   22   22   540   50%  3354   66%
 11 Robopolito 0.10 x64 6c       3351   23   23   541   58%  3300   62% 

Code: Select all

Rank  Name                       Elo    +    -   games score oppo. draws
  38 Ivanhoe B46fa x64 1c        3196   14   14  1475   50%  3197   57%
  39 Komodo 3.0 x64 1c           3194   12   12  2466   50%  3170   39%

----------------------------------------

I noticed also that Ivanhoe is doing not bad even on slower time controls (45m+15s) too:




Many thanks to all chess engine programers,your efforts are much appreciated


Best Regards,
Sedat

[lkaufman]

[quote="mclane & Stockfish.

Maybe my explanation is even more simple but true: Rybka, Ippo & Co are simply better. The weaker engines just benefit from the bigger draw rates at big depths/long time controls.

This might be a factor for Houdini, but certainly Ivanhoe (Ippo), Rybka, and all other Ippo-related programs are not stronger than Komodo and not measurably stronger than SF except at bullet chess, so this cannot be the explanation here.

There seem to be two theories to explain the observed scaling behavior:
1. Komodo (and perhaps also SF) are more intelligent but slower, and that this tradeoff usually (but not always) favors the fast programs in blitz and the smart programs at long time controls.
2. For whatever reason, the search in Komodo (and perhaps SF) scales better than the search in the Rybka/Ippo family.

Both could be true. If the second is true, can anyone suggest WHY SF might scale better in search than Ivanhoe et al?