Number 1 engine on long time controls

[IGarcia]

To help you understand this, there are some good web sites that explain interpolation and curve fitting and such. [...]
[...] If you throw a ball high in the air at a given angle, you can predict almost exactly where it will land

I know what interpolation is, thanks. The example you give is based on physics laws such gravity, friction. The interpolation on komodo strength is not based in any law.

You have some initial data points and you are extending the curve several orders of magnitude (doubling each level). You imagine some sort of continuation. Milos tested a constant +12 elo, Jesus a more reasonable approach, still nobody knows the real curve.

By the way, I did not see Jesus make any sort of claims about anything - I think your words were way too strong.

For the ones not speaking native English some times is hard to say things and easy misunderstood. At all times asked "no offense". The word imaginary con be difficult but don't know how to say in other way. Not real or irreal? sounds offensive.

Maybe also you think I'm aggressive towards you, but I"m trying to express in a good way. Sorry if you feel this way.

[Don]

If you were asking about SP version then I would pick Komodo at those time controls on modern i7 or AMD hardware. Also, I would pick Houdini 1.5 over Houdini 2.0 at long time controls.

DOn why do u say that about 1.5 over 2.0, optimized better for longer time controls?

If you look at the lists you will notice that at the shorter time controls Houdini 2.0 is a big improvement, but at the longer time controls you see Houdini 1.5 catching up.

Anyone that pays attention can easily see that every program has different scaling characteristics. The Ippo clones clearly are dominant at fast time controls but stockfish starts to catch up at longer time controls. Our own internal testing makes this completely obvious - depending on the levels some program are sure to be stronger or weaker than others and it's consistent.

Here is an interesting study I'm doing, which shows Komodo scaling relative to Houdini 1.5:

Code: Select all

                                                                                                           
Level where 00 is 6 + 0.1 and each successive level is double.                                             
                                                                                                           
                     Komodo                                                                                
            HOUDINI   gains                                                                                
            -------  ------                                                                                
 Level 00 -  +143.3                                                                                        
 Leval 01 -   +97.0   +46.3                                                                                
 Leval 02 -   +74.6   +22.4                                                                                
 Level 03 -   +52.8   +21.8                                                                                
 Level 04 -   +39.5   +13.3                                                                                
 Level 05 -   +27.0   +12.5                                                                                

This is a long running test on a slow laptop. Komodo gains several ELO relatively to Houdini for each successful doubling. There is still significant error in 2000 games (each level is 2000 games) so it's hard to be precise, but after 3 more doubling's Komodo will be winning against Houdini 1.5 if it picks up 10 more ELO each time. But the amount it gains per doubling appears to drop a bit with each doubling too so it's really difficult to predict the level at which Komodo is superior.

This is a development version of Komodo which is a little bit stronger than our release version.

Actually the longer the time control then the closer two engines will be in elo. So increasing the time control doesn't really show Komodo scaling better (or Houdini worst). Remember the theory is that chess is a draw.

In principle that is true but not in practice at the levels we test at. If you were to run a very weak and a very strong program at 1 million years per move they might play pretty close in strength but they don't act like they are coming together at levels we can reasonably test.

I mentioned in some other thread that when we put king safety in Komodo a few years ago it changed the scaling characteristics. The version without king safety was actually stronger when doing a few ply as well as faster. But at 5 or 6 ply it went from being weaker to being stronger - then beyond that it went to being MUCH stronger. In time control games the difference in ELO INCREASED every time the level was raised. Since chess is probably a draw as you say I would expect the trend to reverse at some ridiculously high level but it's would not be a level that is practical to test.

Lastly, why not take some ideas from some open source programs to improve the mp ability of komodo? You already used some ideas from open source engines (not copy code) for other things. So it wouldn't hurt to get mp ideas as well.

I know how to do MP, I had an MP program before most or any PC programs had MP.

[Terry McCracken]

I expect that 8 core machines will be commonly available in a few months, but that's plenty of time for us to address this problem.

A few months Larry? I think 8 core Ivy Bridge will come out in a few months but it won't be affordable like the 4 core Sandy Bridge. These will be High-End processors. In a couple of years 8 core will be mainstream.

I can't find an 8 core Ivy Bridge in Intel's Roadmap or on wikipedia's article on Ivy Bridge...do you have a source by any chance? An 8 core IB sounds amazing!

Yeah, 4 & 6 core but 8 core or more is the high-end Xeon processors.

It may be a couple years before we see it in the mainstream. Sandy Bridge-E has 8 cores but only uses 6 with two cores disabled.

[beram]

My interest is in the scaling properties of chess programs not which program is better except in the context of how they scale. I forgot that I was in the general forum and I've attracted fanatics and not the people who are interested in this for technical reasons.

This thread started about number one engine at LTC and not on scaling issue.
You wrote that the ippos were much stronger on short time control (STC)
But at STC (4min 2sec, auto232 list Sedat Canbaz http://www.sedatcanbaz.com/chess/ratings/scct-auto232/) Stfish and Ivanhoe are equally strong, as is the same for them on LTC (Pal Larkin match 40 in 120 min - 20 in 60 min - remainder in 30 min plus 10 seconds per move)
http://www.talkchess.com/forum/viewtopi ... 33&t=42138

You also said that Houdini 2.0 is not stronger than 1.5 while you can see in the list of sedat canbaz and frank quisinsky that Houdini 2.0 is about 15-20 ELO beter, and although that is not much, the same counts for Komodo4 in relation to Komodo3

grts Bram
"There are liars, damned liars and statisticians."
Mark Twain

[Dr.Wael Deeb]

I expect that 8 core machines will be commonly available in a few months, but that's plenty of time for us to address this problem.

A few months Larry? I think 8 core Ivy Bridge will come out in a few months but it won't be affordable like the 4 core Sandy Bridge. These will be High-End processors. In a couple of years 8 core will be mainstream.

I can't find an 8 core Ivy Bridge in Intel's Roadmap or on wikipedia's article on Ivy Bridge...do you have a source by any chance? An 8 core IB sounds amazing!

Yeah, 4 & 6 core but 8 core or more is the high-end Xeon processors.

It may be a couple years before we see it in the mainstream. Sandy Bridge-E has 8 cores but only uses 6 with two cores disabled.

Agreed here....
Dr.D

[Don]

My interest is in the scaling properties of chess programs not which program is better except in the context of how they scale. I forgot that I was in the general forum and I've attracted fanatics and not the people who are interested in this for technical reasons.

This thread started about number one engine at LTC and not on scaling issue.

Isn't that really a scaling question in disguise? Of course it is!

You wrote that the ippos were much stronger on short time control (STC)

Yes, that is quite true and I'm just reporting it.

But at STC (4min 2sec, auto232 list Sedat Canbaz http://www.sedatcanbaz.com/chess/ratings/scct-auto232/) Stfish and Ivanhoe are equally strong, as is the same for them on LTC (Pal Larkin match 40 in 120 min - 20 in 60 min - remainder in 30 min plus 10 seconds per move)
http://www.talkchess.com/forum/viewtopi ... 33&t=42138

You also said that Houdini 2.0 is not stronger than 1.5 while you can see in the list of sedat canbaz and frank quisinsky that Houdini 2.0 is about 15-20 ELO beter, and although that is not much, the same counts for Komodo4 in relation to Komodo3

I'm not debating here, I'm reporting and trying to interpret what I see. So if you are saying that you have data that shows something else I'm cool with that. However you are not showing me any data with Sedat's test that addresses anything I brought up.

I was only comparing Houdini 1.5 to Houdini 2.0 single processor version. I do not know if 2.0 has a better MP implementation. If it does, then it's probably a superior product. I also don't understand the performance of the PRO version on Sedat's test. Is there any advantage to having the PRO version on a 4 core test?

The only way to reconcile my data with Sedat is to run the non-PRO version of each program in single processor mode, then we can compare to see if the data agree's or disagree's.

grts Bram
"There are liars, damned liars and statisticians."
Mark Twain

[noctiferus]

About Mark Twain's citation, I would like to add:
"...and those who don't understand statistics... "
(not intended to you, Bram, but to your citation).

I have some doubts about the methodology used to make these inferences...

[Terry McCracken]

I expect that 8 core machines will be commonly available in a few months, but that's plenty of time for us to address this problem.

A few months Larry? I think 8 core Ivy Bridge will come out in a few months but it won't be affordable like the 4 core Sandy Bridge. These will be High-End processors. In a couple of years 8 core will be mainstream.

I can't find an 8 core Ivy Bridge in Intel's Roadmap or on wikipedia's article on Ivy Bridge...do you have a source by any chance? An 8 core IB sounds amazing!

Yeah, 4 & 6 core but 8 core or more is the high-end Xeon processors.

It may be a couple years before we see it in the mainstream. Sandy Bridge-E has 8 cores but only uses 6 with two cores disabled.

Agreed here....
Dr.D

There may be further delays as Ivy Bridge has had some technical issues involving the new tri-gate 22nm lithography. It will likey ship by the end of June but was slated for release in April.

[Master Om]

Hi Don,

If I want to find the best line which draws( drawish line in a loosing position ) Then what settings should I Use in Komodo 4 ?

[diep]

Small question: how many years did your slow laptop take to play 2000 games at those time controls?

if 00 == 6 1 then level 05 == 192 minutes and some increment?

6 00
12 01
24 02
48 03
96 04
192 05

Better use a big cluster with hundreds of cores for the testing if you want to finish playing 2000 games like that within a few weeks.

If you were asking about SP version then I would pick Komodo at those time controls on modern i7 or AMD hardware. Also, I would pick Houdini 1.5 over Houdini 2.0 at long time controls.

DOn why do u say that about 1.5 over 2.0, optimized better for longer time controls?

If you look at the lists you will notice that at the shorter time controls Houdini 2.0 is a big improvement, but at the longer time controls you see Houdini 1.5 catching up.

Anyone that pays attention can easily see that every program has different scaling characteristics. The Ippo clones clearly are dominant at fast time controls but stockfish starts to catch up at longer time controls. Our own internal testing makes this completely obvious - depending on the levels some program are sure to be stronger or weaker than others and it's consistent.

Here is an interesting study I'm doing, which shows Komodo scaling relative to Houdini 1.5:

Code: Select all

                                                                                                           
Level where 00 is 6 + 0.1 and each successive level is double.                                             
                                                                                                           
                     Komodo                                                                                
            HOUDINI   gains                                                                                
            -------  ------                                                                                
 Level 00 -  +143.3                                                                                        
 Leval 01 -   +97.0   +46.3                                                                                
 Leval 02 -   +74.6   +22.4                                                                                
 Level 03 -   +52.8   +21.8                                                                                
 Level 04 -   +39.5   +13.3                                                                                
 Level 05 -   +27.0   +12.5                                                                                

This is a long running test on a slow laptop. Komodo gains several ELO relatively to Houdini for each successful doubling. There is still significant error in 2000 games (each level is 2000 games) so it's hard to be precise, but after 3 more doubling's Komodo will be winning against Houdini 1.5 if it picks up 10 more ELO each time. But the amount it gains per doubling appears to drop a bit with each doubling too so it's really difficult to predict the level at which Komodo is superior.

This is a development version of Komodo which is a little bit stronger than our release version.[/quote]