Number 1 engine on long time controls

[FriedmannC]

Which engine do you think will be the winner of a 24 games match played against HOUDINI 2.c under the next conditions: 2h30' for the first 40 moves, 1h15' for the next 20 moves and 1 hour for the rest of the game, with 15 seconds increment starting with the first move. The hardware would be at least 6 - 8 cores for each engine (12 cores would be awesome).
1. DEEP RYBKA 4.1
2. KOMODO 4 MP
3. STOCKFISH 2.2.2
4. CRITTER 1.4
5. CRYPTIC
6. THINKER (the private version)
7. STRELKA 5 MP
I would be grateful to those who own that big hardware and are willing to set up the clash of the titans! Only this way we will find out what engine is the best on LTC!

[Uri Blass]

Which engine do you think will be the winner of a 24 games match played against HOUDINI 2.c under the next conditions: 2h30' for the first 40 moves, 1h15' for the next 20 moves and 1 hour for the rest of the game, with 15 seconds increment starting with the first move. The hardware would be at least 6 - 8 cores for each engine (12 cores would be awesome).
1. DEEP RYBKA 4.1
2. KOMODO 4 MP
3. STOCKFISH 2.2.2
4. CRITTER 1.4
5. CRYPTIC
6. THINKER (the private version)
7. STRELKA 5 MP
I would be grateful to those who own that big hardware and are willing to set up the clash of the titans! Only this way we will find out what engine is the best on LTC!

You forgot Houdini1.5 as a candidate to win a match at LTC.

I also do not see why do you include Komodo4 MP that does not exist today and not Deep Rybka5 or Houdini3.

[Don]

Which engine do you think will be the winner of a 24 games match played against HOUDINI 2.c under the next conditions: 2h30' for the first 40 moves, 1h15' for the next 20 moves and 1 hour for the rest of the game, with 15 seconds increment starting with the first move. The hardware would be at least 6 - 8 cores for each engine (12 cores would be awesome).
1. DEEP RYBKA 4.1
2. KOMODO 4 MP
3. STOCKFISH 2.2.2
4. CRITTER 1.4
5. CRYPTIC
6. THINKER (the private version)
7. STRELKA 5 MP
I would be grateful to those who own that big hardware and are willing to set up the clash of the titans! Only this way we will find out what engine is the best on LTC!

If you were asking about SP version then I would pick Komodo at those time controls on modern i7 or AMD hardware. Also, I would pick Houdini 1.5 over Houdini 2.0 at long time controls.

[Kingghidorah]

If you were asking about SP version then I would pick Komodo at those time controls on modern i7 or AMD hardware. Also, I would pick Houdini 1.5 over Houdini 2.0 at long time controls.[/quote]

DOn why do u say that about 1.5 over 2.0, optimized better for longer time controls?

[Don]

If you were asking about SP version then I would pick Komodo at those time controls on modern i7 or AMD hardware. Also, I would pick Houdini 1.5 over Houdini 2.0 at long time controls.

DOn why do u say that about 1.5 over 2.0, optimized better for longer time controls?[/quote]

If you look at the lists you will notice that at the shorter time controls Houdini 2.0 is a big improvement, but at the longer time controls you see Houdini 1.5 catching up.

Anyone that pays attention can easily see that every program has different scaling characteristics. The Ippo clones clearly are dominant at fast time controls but stockfish starts to catch up at longer time controls. Our own internal testing makes this completely obvious - depending on the levels some program are sure to be stronger or weaker than others and it's consistent.

Here is an interesting study I'm doing, which shows Komodo scaling relative to Houdini 1.5:

Code: Select all

                                                                                                           
Level where 00 is 6 + 0.1 and each successive level is double.                                             
                                                                                                           
                     Komodo                                                                                
            HOUDINI   gains                                                                                
            -------  ------                                                                                
 Level 00 -  +143.3                                                                                        
 Leval 01 -   +97.0   +46.3                                                                                
 Leval 02 -   +74.6   +22.4                                                                                
 Level 03 -   +52.8   +21.8                                                                                
 Level 04 -   +39.5   +13.3                                                                                
 Level 05 -   +27.0   +12.5                                                                                

This is a long running test on a slow laptop. Komodo gains several ELO relatively to Houdini for each successful doubling. There is still significant error in 2000 games (each level is 2000 games) so it's hard to be precise, but after 3 more doubling's Komodo will be winning against Houdini 1.5 if it picks up 10 more ELO each time. But the amount it gains per doubling appears to drop a bit with each doubling too so it's really difficult to predict the level at which Komodo is superior.

This is a development version of Komodo which is a little bit stronger than our release version.

[Houdini]

If you look at the lists you will notice that at the shorter time controls Houdini 2.0 is a big improvement, but at the longer time controls you see Houdini 1.5 catching up.

Anyone that pays attention can easily see that every program has different scaling characteristics. The Ippo clones clearly are dominant at fast time controls but stockfish starts to catch up at longer time controls. Our own internal testing makes this completely obvious - depending on the levels some program are sure to be stronger or weaker than others and it's consistent.

Here is an interesting study I'm doing, which shows Komodo scaling relative to Houdini 1.5:

Code: Select all

                                                                                                           
Level where 00 is 6 + 0.1 and each successive level is double.                                             
                                                                                                           
                     Komodo                                                                                
            HOUDINI   gains                                                                                
            -------  ------                                                                                
 Level 00 -  +143.3                                                                                        
 Leval 01 -   +97.0   +46.3                                                                                
 Leval 02 -   +74.6   +22.4                                                                                
 Level 03 -   +52.8   +21.8                                                                                
 Level 04 -   +39.5   +13.3                                                                                
 Level 05 -   +27.0   +12.5                                                                                

This is a long running test on a slow laptop. Komodo gains several ELO relatively to Houdini for each successful doubling. There is still significant error in 2000 games (each level is 2000 games) so it's hard to be precise, but after 3 more doubling's Komodo will be winning against Houdini 1.5 if it picks up 10 more ELO each time. But the amount it gains per doubling appears to drop a bit with each doubling too so it's really difficult to predict the level at which Komodo is superior.

LOL. The usual marketing bullshit from you and Larry.
Basically what you demonstrate is that:
1) At every TC you tested, even the free Houdini 1.5 is clearly stronger than Komodo 4.
2) At fast TC Komodo is very bad.

All the rest is speculation and extrapolation, with a clear commercial motivation - including the usual denigrating of Houdini 2.0.

Way to go.
Robert

[Don]

If you look at the lists you will notice that at the shorter time controls Houdini 2.0 is a big improvement, but at the longer time controls you see Houdini 1.5 catching up.

Anyone that pays attention can easily see that every program has different scaling characteristics. The Ippo clones clearly are dominant at fast time controls but stockfish starts to catch up at longer time controls. Our own internal testing makes this completely obvious - depending on the levels some program are sure to be stronger or weaker than others and it's consistent.

Here is an interesting study I'm doing, which shows Komodo scaling relative to Houdini 1.5:

Code: Select all

                                                                                                           
Level where 00 is 6 + 0.1 and each successive level is double.                                             
                                                                                                           
                     Komodo                                                                                
            HOUDINI   gains                                                                                
            -------  ------                                                                                
 Level 00 -  +143.3                                                                                        
 Leval 01 -   +97.0   +46.3                                                                                
 Leval 02 -   +74.6   +22.4                                                                                
 Level 03 -   +52.8   +21.8                                                                                
 Level 04 -   +39.5   +13.3                                                                                
 Level 05 -   +27.0   +12.5                                                                                

This is a long running test on a slow laptop. Komodo gains several ELO relatively to Houdini for each successful doubling. There is still significant error in 2000 games (each level is 2000 games) so it's hard to be precise, but after 3 more doubling's Komodo will be winning against Houdini 1.5 if it picks up 10 more ELO each time. But the amount it gains per doubling appears to drop a bit with each doubling too so it's really difficult to predict the level at which Komodo is superior.

LOL. The usual marketing bullshit from you and Larry.
Basically what you demonstrate is that:
1) At every TC you tested, even the free Houdini 1.5 is clearly stronger than Komodo 4.
2) At fast TC Komodo is very bad.

All the rest is speculation and extrapolation, with a clear commercial motivation - including the usual denigrating of Houdini 2.0.

Way to go.
Robert

What I'm doing here is a study, not marketing. The data shows that Houdini does not scale well. I publish this and my methodology so that it can be duplicated and verified by others. I run 2000 games because we get a lot of nonsense statements from people who run 10 games matches to draw conclusions, so I would invite anyone to duplicate this test with more games.

[Houdini]

What I'm doing here is a study, not marketing. The data shows that Houdini does not scale well. I publish this and my methodology so that it can be duplicated and verified by others. I run 2000 games because we get a lot of nonsense statements from people who run 10 games matches to draw conclusions, so I would invite anyone to duplicate this test with more games.

You don't just report data, you SPECULATE about how good your own engine Komodo might be at some time control you haven't actually used in your test.
In addition you SPECULATE about a commercial competitor, Houdini 2, a program you've never actually used.

As a forum moderator, don't you feel a little bit ethically challenged when writing this marketing blabla disguised as "a study"?

Robert

[IGarcia]

What I'm doing here is a study, not marketing. The data shows that Houdini does not scale well. I publish this and my methodology so that it can be duplicated and verified by others. I run 2000 games because we get a lot of nonsense statements from people who run 10 games matches to draw conclusions, so I would invite anyone to duplicate this test with more games.

You don't just report data, you SPECULATE about how good your own engine Komodo might be at some time control you haven't actually used in your test.
In addition you SPECULATE about a commercial competitor, Houdini 2, a program you've never actually used.

As a forum moderator, don't you feel a little bit ethically challenged when writing this marketing blabla disguised as "a study"?

Robert

hehehe, good point. The "study" conclusion is:
At infinite TC komodo will draw Houdini 1.5 !!

[Don]

What I'm doing here is a study, not marketing. The data shows that Houdini does not scale well. I publish this and my methodology so that it can be duplicated and verified by others. I run 2000 games because we get a lot of nonsense statements from people who run 10 games matches to draw conclusions, so I would invite anyone to duplicate this test with more games.

You don't just report data, you SPECULATE about how good your own engine Komodo might be at some time control you haven't actually used in your test.
In addition you SPECULATE about a commercial competitor, Houdini 2, a program you've never actually used.

As a forum moderator, don't you feel a little bit ethically challenged when writing this marketing blabla disguised as "a study"?

Robert

I'm not going to engage in a pissing contest with you. Make anything out of that you wish. Meanwhile I'll refer you to this long time control rating list:

http://www.husvankempen.de/nunn//40120n ... liste.html