Explanation for no big improvement in long time control

[Uri Blass]

I looked in the CCRL list and I found no strong program that earn significantly from long time control(by significantly I mean that if most programs at similiar level earn average rating of X elo from doubling the speed the specific program earns X+20 elo rating points from doubling the speed.

My guess is that the reason is simply diminishing returns.
It is easier to earn rating from small elo

Rybka may have an algorithm that allow to increase her rating from
2600 to 2700 by doubling the speed when other programs only incease their rating from 2600 to 2670 by doubling the speed but unfortunately rybka is at the level of 2900 and improving from 2900 is harder so rybka can improve only from 2900 to 2970 when other programs with time control that is long enough can only improve from 2900 to 2950 by doubling the speed.

If you start by playing games with unequal time control when you care to choose for every program specific time limit when you care to choose time limit that programs always score 40-60% in long matches then you will find that the top programs earn more than 20 elo from doubling the speed relative to weak oppponents.

Unfortunately testers usually do not test with unequal time control so we cannot see this effect in rating lists.

[Don]

Uri,

Chess programming is all about how scalable your program is. Getting the most out of each doubling is how you play the game.

Less important are constant factors such as having a good compiler or squeezing a little extra performance (speed) out of the code. Those are important, but no matter how good these things are, the more scalable program will eventually beat you - given long enough time controls or powerful enough hardware.

This is worth studying in your own program by comparing to other programs. For instance I have determined that my own program loses about 10 ELO per doubling against a strong program that I like to test against. That tells me something is wrong with my program, perhaps my move ordering is just a bit inferior somehow, or my LMR conditions or something else. Or it could just be some bug that impacts deeper searches more.

I looked in the CCRL list and I found no strong program that earn significantly from long time control(by significantly I mean that if most programs at similiar level earn average rating of X elo from doubling the speed the specific program earns X+20 elo rating points from doubling the speed.

My guess is that the reason is simply diminishing returns.
It is easier to earn rating from small elo

Rybka may have an algorithm that allow to increase her rating from
2600 to 2700 by doubling the speed when other programs only incease their rating from 2600 to 2670 by doubling the speed but unfortunately rybka is at the level of 2900 and improving from 2900 is harder so rybka can improve only from 2900 to 2970 when other programs with time control that is long enough can only improve from 2900 to 2950 by doubling the speed.

If you start by playing games with unequal time control when you care to choose for every program specific time limit when you care to choose time limit that programs always score 40-60% in long matches then you will find that the top programs earn more than 20 elo from doubling the speed relative to weak oppponents.

Unfortunately testers usually do not test with unequal time control so we cannot see this effect in rating lists.

[Uri Blass]

Don,
I find based on the CCRL that the strong programs do not earn more from time relative to the weak programs and I tried to explain this fact.

The only explanation that I can find is diminishing returns.


from the ccrl 40/40 list

Rybka 3 64-bit 3158 +14 −14 73.4% −168.5 33.5% 1946
Homer 2.01 2540 +24 −24 50.3% −2.5 32.3% 592


from the ccrl 40/4 list

Rybka 3 64-bit 3173 +9 −9 78.5% −226.1 25.5% 6966
Homer 2.01 2466 +24 −24 53.9% −31.3 20.0% 629


If you look at the number you find 707 elo difference at blitz and 618 elo difference at long time control

People may get a wrong impression based on the numbers that strong program do not get more from doubling the time.

I believe that the truth is that they get more from doubling the time if you start from constant rating level but rybka starts from significantly higher level than homer when it plays 40/4 and at higher level it is harder to improve.

Note that when I compare movei to other programs of similiar strength I find that movei earn more from doubling based on the CCRL data.

This fact is surprising because I never thought that movei is stronger at long time control and I consider my order of moves to be relatively primitive(for example I have no special function for better order of moves when the king is under threat).

Uri

[Don]

I'm not convinced that the top 5-10 programs scale the same. It could be that Rybka is strongest at 40/40 but would not be at some really super long time control like game in 1 week. In other words it COULD be the constant factors I am talking about that will explain the differences between programs that are within 200 ELO of each other.

There has been speculation that Rybka is deep down just a really well coded program - very Fritz-like in speed and heavily optimized. And otherwise it's just a very strong program but not super special. I don't know if I completely buy that, but it's possible that Rybka is just 2-3 times faster than the other programs that are (relatively) close to it in strength.

Another issue is that you can optimize a program for a particular level - either on purpose or by accident. I suspect my program is optimized to play well at game in 15 seconds or levels around that, because I test at those levels in order to get the thousands of games I need. I don't like it, but I have no real choice.

Don,
I find based on the CCRL that the strong programs do not earn more from time relative to the weak programs and I tried to explain this fact.

The only explanation that I can find is diminishing returns.


from the ccrl 40/40 list

Rybka 3 64-bit 3158 +14 −14 73.4% −168.5 33.5% 1946
Homer 2.01 2540 +24 −24 50.3% −2.5 32.3% 592


from the ccrl 40/4 list

Rybka 3 64-bit 3173 +9 −9 78.5% −226.1 25.5% 6966
Homer 2.01 2466 +24 −24 53.9% −31.3 20.0% 629


If you look at the number you find 707 elo difference at blitz and 618 elo difference at long time control

People may get a wrong impression based on the numbers that strong program do not get more from doubling the time.

I believe that the truth is that they get more from doubling the time if you start from constant rating level but rybka starts from significantly higher level than homer when it plays 40/4 and at higher level it is harder to improve.

Note that when I compare movei to other programs of similiar strength I find that movei earn more from doubling based on the CCRL data.

This fact is surprising because I never thought that movei is stronger at long time control and I consider my order of moves to be relatively primitive(for example I have no special function for better order of moves when the king is under threat).

Uri

[Uri Blass]

I agree that the top 5-10 programs may not scale the same but
I tend to believe that in most cases rybka3 scale better
with the possible exception of naum4

The difference from rybka to other programs cannot be explained by
speed factor of 2 or 3.

The claim that you have no choice except optimizing your program for fast time control is not correct.
You have a choice not to play thousands of games and do only changes that you believe in them even if you have no evidence that they help in games.

Uri

[Don]

I agree that the top 5-10 programs may not scale the same but
I tend to believe that in most cases rybka3 scale better
with the possible exception of naum4

The difference from rybka to other programs cannot be explained by
speed factor of 2 or 3.

Like I say, I don't buy that it's just speed - I think there is something special about Rybka and that it probably scales better. It certainly scales at least as well or better than the top 3 or 4 programs.

The claim that you have no choice except optimizing your program for fast time control is not correct.

Of course. I do have a choice, I could test at really long time controls and just take much longer to develop.

You have a choice not to play thousands of games and do only changes that you believe in them even if you have no evidence that they help in games
Uri

That works fine unless the right hemisphere of your brain is half missing like in my case.

So I have to do it left brained if I hope to have any success.

[Don]

Hey,

According to this test which you can take in 2 seconds, I am really right brained!

http://www.news.com.au/heraldsun/story/ ... 61,00.html

It took me a long time to see the dancer rotating counter-clockwise.

- Don

I agree that the top 5-10 programs may not scale the same but
I tend to believe that in most cases rybka3 scale better
with the possible exception of naum4

The difference from rybka to other programs cannot be explained by
speed factor of 2 or 3.

Like I say, I don't buy that it's just speed - I think there is something special about Rybka and that it probably scales better. It certainly scales at least as well or better than the top 3 or 4 programs.

The claim that you have no choice except optimizing your program for fast time control is not correct.

Of course. I do have a choice, I could test at really long time controls and just take much longer to develop.

You have a choice not to play thousands of games and do only changes that you believe in them even if you have no evidence that they help in games
Uri

That works fine unless the right hemisphere of your brain is half missing like in my case.

So I have to do it left brained if I hope to have any success.

[adieguez]

interesting.. me too, but as the shadow is anti-clockwise, I could see it anti-clock wise too starting from there.

[Jan Brouwer]

I also see her rotating clockwise, I can't see her turning counter-clockwise however hard I try!
Only when reading the text can I see her rotating counter-clockwise in the corner of my eye.
Is this really an accurate test? I think I should be right-brained.

Jan

[Bo Persson]

I also see her rotating clockwise, I can't see her turning counter-clockwise however hard I try!
Only when reading the text can I see her rotating counter-clockwise in the corner of my eye.
Is this really an accurate test? I think I should be right-brained.

Jan

I get this effect too - amazing.

I'm defintiely left brained though.