Number 1 engine on long time controls

[Werewolf]

If the aim is to show Houdini is bad at LTC (or that Komodo is strong at LTC) I would have thought we'd want as many processors involved as possible as it allows us to simulate LTC in less time!

cute, but flawed.

Sounds like something my parents used to say...

But if the comparison was between Houdini 1.5 and Houdini 2 (BOTH have SP and MP versions) and the aim was to show Houdini 2 was inferior to Houdini 1.5 at LTC - wouldn't the MP results have some relevance then?
We'd be hitting higher depths.

[Uri Blass]

What I'm doing here is a study, not marketing. The data shows that Houdini does not scale well. I publish this and my methodology so that it can be duplicated and verified by others. I run 2000 games because we get a lot of nonsense statements from people who run 10 games matches to draw conclusions, so I would invite anyone to duplicate this test with more games.

You don't just report data, you SPECULATE about how good your own engine Komodo might be at some time control you haven't actually used in your test.
In addition you SPECULATE about a commercial competitor, Houdini 2, a program you've never actually used.

As a forum moderator, don't you feel a little bit ethically challenged when writing this marketing blabla disguised as "a study"?

Robert

I'm not going to engage in a pissing contest with you. Make anything out of that you wish. Meanwhile I'll refer you to this long time control rating list:

http://www.husvankempen.de/nunn//40120n ... liste.html

1 elo difference is not convincing and I am not sure that komodo continue to earn more from time when you double the speed(I can see diminishing returns from doubling in the difference between houdini and komodo so it may be possible that at longer time control than 120/40 the gain that komodo earns relative to houdini is zero and houdini is at the same level as komodo at 120/40 or slower time control).

[Sedat Canbaz]

Just my 2 cents over this issue

With respect to all,honestly i dont care a lot about results which are played without at full performance

And i don't see any reason, why the top mp engines should be tested at least 150-200 Elo weaker than their real strength?!

I hope, its not due to that '1 core ratings' are the most accurate testings

I hardly doubt that one day that i will create a new rating list,which will be based on only 1 core engines

But in this case,if one day i will create a rating list on only 1 core engines,then my conditions will be:
-No opening books
-Only 32 bit engines
-No endgames

Note:just only under such conditions we can say:it is a accurate right testing (no any advantage ...)

In other words,i prefer to drive my auto machine with 150km -200km per hour (on a good highway) than 30km-50km per hour

For example,i drive with 30km-50km per hour,when there is catastrophe or policeman


Best,
Sedat

[bhlangonijr]

Sounds like something my parents used to say...

But if the comparison was between Houdini 1.5 and Houdini 2 (BOTH have SP and MP versions) and the aim was to show Houdini 2 was inferior to Houdini 1.5 at LTC - wouldn't the MP results have some relevance then?
We'd be hitting higher depths.

My only point is that the statement "[MP versions]...allow us to simulate LTC in less time" doesn't hold true - even worse if you are using the SP versions as a point of reference.

One may have implemented a more accurate move ordering when splitting the search, for example, in a newer MP version of a certain engine. That would make the new MP version a lot better than that of the older versions , but due to the fact that we specifically improved MP search in the newer engine. You can think it otherwise, maybe it was introduced a minor bug in this move ordering in the newer version, and it would do worse against older MP versions, etc...

The Elo gain measured between different versions (MP or not) might be result of a lot of things combined, not only one.

[bhlangonijr]

Just my 2 cents over this issue

With respect to all,honestly i dont care a lot about results which are played without at full performance

And i don't see any reason, why the top mp engines should be tested at least 150-200 Elo weaker than their real strength?!

I hope, its not due to that '1 core ratings' are the most accurate testings

I hardly doubt that one day that i will create a new rating list,which will be based on only 1 core engines

But in this case,if one day i will create a rating list on only 1 core engines,then my conditions will be:
-No opening books
-Only 32 bit engines
-No endgames

Note:just only under such conditions we can say:it is a accurate right testing (no any advantage ...)

In other words,i prefer to drive my auto machine with 150km -200km per hour (on a good highway) than 30km-50km per hour

For example,i drive with 30km-50km per hour,when there is catastrophe or policeman


Best,
Sedat

Sedat,

Anyone would want to test the engines with the best performance it gets. A different thing is to draw (wrong) conclusions based on that.

[Don]

1 elo difference is not convincing and I am not sure that komodo continue to earn more from time when you double the speed(I can see diminishing returns from doubling in the difference between houdini and komodo so it may be possible that at longer time control than 120/40 the gain that komodo earns relative to houdini is zero and houdini is at the same level as komodo at 120/40 or slower time control).

1 ELO doesn't mean anything , the error margin is over 20 ELO for both programs. My point is that this is pretty powerful evidence that Komodo scales better than Houdini 1.5, I don't know the precise level where it becomes superior but one would have to bury his head in the sand and ignore all data to believe that this is not the logical conclusion.

It seems oddly illogical to think that the trend will suddenly reverse or stop - do you have some sound basis for thinking that is likely or are you just saying that anything is possible, even when there is no supporting evidence we can identify? I am willing to entertain any good theories you have on this and maybe they can even be tested.

The diminishing returns you noticed is not odd at all, in fact it would be really odd if that didn't happen wouldn't it? With each doubling BOTH programs continue to improve and it gets more and more difficult to win a game for EITHER program. It's logical that the better scaling program would increase it's score with each doubling but not by a constant factor. I think it's pretty illogical to see this as being an odd thing or indication that there is a scaling issue.

[Sedat Canbaz]

Just my 2 cents over this issue

With respect to all,honestly i dont care a lot about results which are played without at full performance

And i don't see any reason, why the top mp engines should be tested at least 150-200 Elo weaker than their real strength?!

I hope, its not due to that '1 core ratings' are the most accurate testings

I hardly doubt that one day that i will create a new rating list,which will be based on only 1 core engines

But in this case,if one day i will create a rating list on only 1 core engines,then my conditions will be:
-No opening books
-Only 32 bit engines
-No endgames

Note:just only under such conditions we can say:it is a accurate right testing (no any advantage ...)

In other words,i prefer to drive my auto machine with 150km -200km per hour (on a good highway) than 30km-50km per hour

For example,i drive with 30km-50km per hour,when there is catastrophe or policeman


Best,
Sedat

Sedat,

Anyone would want to test the engines with the best performance it gets. A different thing is to draw (wrong) conclusions based on that.

Look Ben-Hur,

Just i'd like to mention again that there is no perfect ratings,each testing has own advantages,disatavatages

Sure anybody has own right and all Testers can run their engines in any conditions as they wish (good luck to all)

We live in democracy world and i think i have a full right to impress my opinion too or is it prohibited ?!

About my statements are wrong or not
Honestly i dont expect that all people around the world will agree with me
And i am well-informed too that i should drive with no more than 90-120 km per hour

A few notes about some good rating lists,
Probably SSDF Rating should be called as 'Father' of the ratings
SSDF is as an example/model of the right/true testing,really its very useful rating (exception they can't test all engines)


Another very useful sites are CEGT/SWCR/CCRL,especially i visit these ratings to check the performance for 1 core engine standings


And last (as i mentioned before),i respect all works,but i will prefer to work in my own way


Regards,
Sedat

[Uri Blass]

1 elo difference is not convincing and I am not sure that komodo continue to earn more from time when you double the speed(I can see diminishing returns from doubling in the difference between houdini and komodo so it may be possible that at longer time control than 120/40 the gain that komodo earns relative to houdini is zero and houdini is at the same level as komodo at 120/40 or slower time control).

1 ELO doesn't mean anything , the error margin is over 20 ELO for both programs. My point is that this is pretty powerful evidence that Komodo scales better than Houdini 1.5, I don't know the precise level where it becomes superior but one would have to bury his head in the sand and ignore all data to believe that this is not the logical conclusion.

It seems oddly illogical to think that the trend will suddenly reverse or stop - do you have some sound basis for thinking that is likely or are you just saying that anything is possible, even when there is no supporting evidence we can identify? I am willing to entertain any good theories you have on this and maybe they can even be tested.

The diminishing returns you noticed is not odd at all, in fact it would be really odd if that didn't happen wouldn't it? With each doubling BOTH programs continue to improve and it gets more and more difficult to win a game for EITHER program. It's logical that the better scaling program would increase it's score with each doubling but not by a constant factor. I think it's pretty illogical to see this as being an odd thing or indication that there is a scaling issue.

It is not odd but I think that you are going to see the same trend with different numbers if you test komodo against a version of komodo that is twice faster(probably it is going to earn less than what it earns relative to houdini but still earns something so the fact that there is a trend that suggests that the weakest program scale better by itself prove nothing).

My conjecture is that komodo is better than houdini at very long time control but
I did not see a convincing evidence and I admit that my guess may be wrong.

There may be algorithms that scale better at very fast time control but stop to scale better at slow time control and are even worse at very slow time control.

For example null move without special verification search that many programs use may be one of them.

At very fast time control the program does not reach depth that is big enough to use null move pruning so the gain is 0.
At very slow time control not using null move is better because null move
make errors and the program that does not use null move pruning may solve the game.

[Sedat Canbaz]

One thing more,i forgot to mention about my own rating list:SCCT (but not only me,even i see also that some experts are forget to mention SCCT)

I wonder sometimes why ?,is that can be the reason:
-Perfect books series are very weak books
-My used hardwares are too slow (old-dated)
-or maybe there is something wrong in my test conditions
-or perhaps SCCT games are not so useful

It seems, i am missing something,can anybody explain me why ???
I need your feedback please,in this way i can improve my work (thanks in advance)

Its true also that i can not test all engines with slower time controls, due to i have no free time for all
With respect to all,many of the people still did not dream what kind of free efforts i spent for Computerchess's progress

But however,i am already accustomed to live in this way

And many thanks to all chess friends,for your supports during over many years



Best,
Sedat

[IGarcia]

1 ELO doesn't mean anything , the error margin is over 20 ELO for both programs. My point is that this is pretty powerful evidence that Komodo scales better than Houdini 1.5, I don't know the precise level where it becomes superior but one would have to bury his head in the sand and ignore all data to believe that this is not the logical conclusion.

Why it has to become superior? Your thinking is like more time = better moves. You then extrapolate those better moves to be the best (perfect?) at some LTC.

I understand the trend, witch I see as a fact, is because more time = less blunders. By having more time is possible to make less mistakes, find some difficult to find answer to escape into a draw in a very complex position. The trend you see probably is just because komodo is losing less games (not winning more).

It can be said that less errors is the same that better move, but I'm meaning other thing: With better moves you can gain initiative and games, with less errors you can equalize or save some games.

It seems oddly illogical to think that the trend will suddenly reverse or stop - do you have some sound basis for thinking that is likely or are you just saying that anything is possible, even when there is no supporting evidence we can identify? I am willing to entertain any good theories you have on this and maybe they can even be tested.

Code: Select all

World English Dictionary
asymptote  (ˈæsɪmˌtəʊt)
 
— n
	a straight line that is closely approached by a plane curve so that the perpendicular distance between them decreases to zero as the distance from the origin increases to infinity
 
[C17: from Greek asumptōtos  not falling together, from a- 1  + syn-  + ptōtos  inclined to fall, from piptein  to fall]

Collins English Dictionary - Complete & Unabridged 10th Edition
2009 © William Collins Sons & Co. Ltd. 1979, 1986 © HarperCollins
Publishers 1998, 2000, 2003, 2005, 2006, 2007, 2009
Cite This Source