Number 1 engine on long time controls

[lkaufman]

I expect that 8 core machines will be commonly available in a few months, but that's plenty of time for us to address this problem.

[Uri Blass]

Makes sense to me.
6 cores is nearly a standart today, more cores means today that you are not anymore in "normal" computer world....

regards

I think that most people do not use more than 4 cores today and it is not going to be changed in the next 2 years.

[Don]

But isn't "scaling" wrong term here? I think it's meaning how program benefits from more CPUs and Ippo family is not worse in this respect. But what's better word I don't have idea..

Scaling is clearly the right word here. Scaling refers to throwing more Hardware at a problem and getting satisfactory improvements from do so. The "more hardware" can be faster CPU, more cores, more processors, more memory, faster networking, etc.

It's possible for a single processor program to not scale at. Too much forward pruning at high depths for example could give you a program that improves for a while but basically hits a wall, cannot improve beyond some modest ELO level.

[beram]

If you were asking about SP version then I would pick Komodo at those time controls on modern i7 or AMD hardware. Also, I would pick Houdini 1.5 over Houdini 2.0 at long time controls.

DOn why do u say that about 1.5 over 2.0, optimized better for longer time controls?

If you look at the lists you will notice that at the shorter time controls Houdini 2.0 is a big improvement, but at the longer time controls you see Houdini 1.5 catching up.

The difference between Komodo 3 and 4 is as little as the difference between Houdini 1.5 and 2.0

Anyone that pays attention can easily see that every program has different scaling characteristics. The Ippo clones clearly are dominant at fast time controls but stockfish starts to catch up at longer time controls.

This statement is false, as you can easily observe in the latest LTC-match of Pal Larkin between Stockfish and Ivanhoe.

I agree on Robert Houdart that you are bashing his program
You better concentrate on making Komodo stronger at 4/40 or 40/40 TC,
only 30 ELO points to go and Komodo is number one, but until for now Houdini 2.0 is undisputably the number one program.

[Don]

I never tried to predict that it would be 35 ELO stronger nor did I even say at which level it would pass Houdini.

I asked to you several posts ago. Witch level you think Komodo will pass Houdini acording to your data? You never reply.

Its fair to ask because you are suggesting at some point komodo will pass. Will be interesting to hear the predictions from you.

respectfully, Ignacio.

I'm still trying to understand the data I am seeing so it would premature to make predictions. I don't remember you asking but if I did see it I probably was not prepared to give you an answer.

in the 40/120 CEGT rating list Komodo is showing as 1 ELO stronger than Houdini 1.5 based on 400 Komodo games and 600 Houdini games. These are time adjusted for older hardware so it's not really 40/120, it's something faster. The error margin there is something like 20 ELO which is somewhat similar to what Milos reports.

If you want to pin me down to a figure I cannot give you one, but I would say that at 40/120 on modern hardware we are within 10 ELO in either direction. The latest development version of Komodo will be about 20 more than that.

http://www.husvankempen.de/nunn//40120n ... liste.html

I asked several post ago. question was:
Still, maybe you are right. Can you tell us witch time control komodo will overtake?

And I don't want to pin you. I'm just curious to see the answer of so many predictions. So much efforts to estimate a hypothetical overtake ... ok, we are adults, everyone can decide what to do with his time.

Don't worry about not wanting to pin me down - since I don't know the answer I won't give you one and that can be something you can worry about. For us it's not very interesting anyway since we don't plan to stop improving Komodo and we think everyone else will continue to improve their programs and Houdart will continue to improve Ivanhoe and so on.

I have to keep repeating this, but I did not do this study to figure out when we would overtake Houdini, I did it to study scaling issues. It only became interesting at all when the 40/120 rating list indicated that Komodo 4 may have already caught up at long time controls. I'm also interesting in knowing if Robert has improved Ivanhoe enough to change it's scaling characteristics.

I have done this same type of study between Komodo versions and versions of other programs too and it's something Larry and I have been well aware of for a very long time.

What I'm finding surprising is that almost nobody else knows about scaling issues and their importance - they seem almost oblivious to the concept.

Jesus, nice work, but is just suppositions. You imagine komodo will keep preforming in some way (not lineal as proposed before) and then this and that... then you conlusion, (no offense please) is imaginary.

To help you understand this, there are some good web sites that explain interpolation and curve fitting and such. It's a valid scientific concept and in fact we couldn't do much of anything without it. I don't want to go into too much detail here but I'll give you a simple example. If you throw a ball high in the air at a given angle, you can predict almost exactly where it will land if you know the wind speed, gravity, speed and other factors. If someone said, "how do you know it will land on the ground since that hasn't happened yet?" you would probably think them to be a fool because every human has some built in ability to interpolate. We also know that chess program get better when you give them more time or CPU power to think. We don't just know that it will happen we can predict HOW MUCH stronger it will be with a fairly high degree of accuracy.

This is the same concept. You can see the trend and what has been done here with Jesus, myself and others is perfectly valid. You call it "supposition" but that's probably not the best word even though it's partly supposition. We know almost exactly where there earth will be in 500 years and what days there will be a full moon and that is supposition too, it's based on predicting things that have not happened yet.

By the way, I did not see Jesus make any sort of claims about anything - I think your words were way too strong. All he was doing was some curve fitting to predict the next data point, which is a perfectly valid scientific concept. You can read about these same kind of things on the web, a starting point is the Wikipedia on interpolation but there are probably better ones. It's a cool thing.

regards

[Don]

If you were asking about SP version then I would pick Komodo at those time controls on modern i7 or AMD hardware. Also, I would pick Houdini 1.5 over Houdini 2.0 at long time controls.

DOn why do u say that about 1.5 over 2.0, optimized better for longer time controls?

If you look at the lists you will notice that at the shorter time controls Houdini 2.0 is a big improvement, but at the longer time controls you see Houdini 1.5 catching up.

The difference between Komodo 3 and 4 is as little as the difference between Houdini 1.5 and 2.0

What does that have to do with anything? Komodo 3 was a huge jump, Komodo 4 was a small jump in strength. What is your point?

Anyone that pays attention can easily see that every program has different scaling characteristics. The Ippo clones clearly are dominant at fast time controls but stockfish starts to catch up at longer time controls.

This statement is false, as you can easily observe in the latest LTC-match of Pal Larkin between Stockfish and Ivanhoe.

I'm not talking about which program is superior, Ivanhoe and other Ippo-style programs are superior to Stockfish so I have to ask you the same question I asked before, what does that have to do with anything?

Ivanhoe dominates Stockfish at all time controls, but play it at game in 1 minute and I think you will understand my point.

[/quote]

I agree on Robert Houdart that you are bashing his program
[/quote]
Robert Houdart does not have a program. Just because he took Ivanhoe and improved it doesn't mean it's his program.

You better concentrate on making Komodo stronger at 4/40 or 40/40 TC,
only 30 ELO points to go and Komodo is number one, but until for now Houdini 2.0 is indisputably the number one program.

My interest is in the scaling properties of chess programs not which program is better except in the context of how they scale. I forgot that I was in the general forum and I've attracted fanatics and not the people who are interested in this for technical reasons.

[Dr.Wael Deeb]

If you were asking about SP version then I would pick Komodo at those time controls on modern i7 or AMD hardware. Also, I would pick Houdini 1.5 over Houdini 2.0 at long time controls.

DOn why do u say that about 1.5 over 2.0, optimized better for longer time controls?

If you look at the lists you will notice that at the shorter time controls Houdini 2.0 is a big improvement, but at the longer time controls you see Houdini 1.5 catching up.

The difference between Komodo 3 and 4 is as little as the difference between Houdini 1.5 and 2.0

What does that have to do with anything? Komodo 3 was a huge jump, Komodo 4 was a small jump in strength. What is your point?

Anyone that pays attention can easily see that every program has different scaling characteristics. The Ippo clones clearly are dominant at fast time controls but stockfish starts to catch up at longer time controls.

This statement is false, as you can easily observe in the latest LTC-match of Pal Larkin between Stockfish and Ivanhoe.

I'm not talking about which program is superior, Ivanhoe and other Ippo-style programs are superior to Stockfish so I have to ask you the same question I asked before, what does that have to do with anything?

Ivanhoe dominates Stockfish at all time controls, but play it at game in 1 minute and I think you will understand my point.

I agree on Robert Houdart that you are bashing his program
[/quote]
Robert Houdart does not have a program. Just because he took Ivanhoe and improved it doesn't mean it's his program.

You better concentrate on making Komodo stronger at 4/40 or 40/40 TC,
only 30 ELO points to go and Komodo is number one, but until for now Houdini 2.0 is indisputably the number one program.

My interest is in the scaling properties of chess programs not which program is better except in the context of how they scale. I forgot that I was in the general forum and I've attracted fanatics and not the people who are interested in this for technical reasons.[/quote]

Yep,you're a fanatics magnet Don

[h1a8]

If you were asking about SP version then I would pick Komodo at those time controls on modern i7 or AMD hardware. Also, I would pick Houdini 1.5 over Houdini 2.0 at long time controls.

DOn why do u say that about 1.5 over 2.0, optimized better for longer time controls?

If you look at the lists you will notice that at the shorter time controls Houdini 2.0 is a big improvement, but at the longer time controls you see Houdini 1.5 catching up.

Anyone that pays attention can easily see that every program has different scaling characteristics. The Ippo clones clearly are dominant at fast time controls but stockfish starts to catch up at longer time controls. Our own internal testing makes this completely obvious - depending on the levels some program are sure to be stronger or weaker than others and it's consistent.

Here is an interesting study I'm doing, which shows Komodo scaling relative to Houdini 1.5:

Code: Select all

                                                                                                           
Level where 00 is 6 + 0.1 and each successive level is double.                                             
                                                                                                           
                     Komodo                                                                                
            HOUDINI   gains                                                                                
            -------  ------                                                                                
 Level 00 -  +143.3                                                                                        
 Leval 01 -   +97.0   +46.3                                                                                
 Leval 02 -   +74.6   +22.4                                                                                
 Level 03 -   +52.8   +21.8                                                                                
 Level 04 -   +39.5   +13.3                                                                                
 Level 05 -   +27.0   +12.5                                                                                

This is a long running test on a slow laptop. Komodo gains several ELO relatively to Houdini for each successful doubling. There is still significant error in 2000 games (each level is 2000 games) so it's hard to be precise, but after 3 more doubling's Komodo will be winning against Houdini 1.5 if it picks up 10 more ELO each time. But the amount it gains per doubling appears to drop a bit with each doubling too so it's really difficult to predict the level at which Komodo is superior.

This is a development version of Komodo which is a little bit stronger than our release version.

Actually the longer the time control then the closer two engines will be in elo. So increasing the time control doesn't really show Komodo scaling better (or Houdini worst). Remember the theory is that chess is a draw.

Lastly, why not take some ideas from some open source programs to improve the mp ability of komodo? You already used some ideas from open source engines (not copy code) for other things. So it wouldn't hurt to get mp ideas as well.

[Terry McCracken]

I expect that 8 core machines will be commonly available in a few months, but that's plenty of time for us to address this problem.

A few months Larry? I think 8 core Ivy Bridge will come out in a few months but it won't be affordable like the 4 core Sandy Bridge. These will be High-End processors. In a couple of years 8 core will be mainstream.

[Werewolf]

I expect that 8 core machines will be commonly available in a few months, but that's plenty of time for us to address this problem.

A few months Larry? I think 8 core Ivy Bridge will come out in a few months but it won't be affordable like the 4 core Sandy Bridge. These will be High-End processors. In a couple of years 8 core will be mainstream.

I can't find an 8 core Ivy Bridge in Intel's Roadmap or on wikipedia's article on Ivy Bridge...do you have a source by any chance? An 8 core IB sounds amazing!