Pure Increment matches

[lkaufman]

We have some indications, both from our own testing and from some reports, that Komodo performs better when tested with increment only or with increment being dominant. An example would be 1' + 5" increment. If anyone would like to run matches between Komodo 4 64 bit sse and other top programs at this or similar time limits, please post the results here. If this is confirmed we can try to figure out why it is so.
Also, if anyone wants to run Ponder On matches with other top programs, first in default mode, then with the relevant Divisor reduced by 20% (from 40 to 32 or 30 to 24), the comparison would also be helpful to us.
Thanks in advance to anyone who posts relevant results here.

[Tomcass]

Larry, I have got the following results, which confirm your hypotheses:

GUI Fritz 12
Book Perfect 2012
1 minutes + 5 sec/game (About 9 to 10 minutes per engine and game).
Intel X980 3.33 Mhz. 24 MEM RAM

Komodo64 SSE Version 4 - Deep Rybka 4.1 SSE42 x64(x1) 27.5 - 22.5 +16/=23/-11 55.00%
Komodo64 SSE Version 4 - Houdini 2.0c Pro x64(x1) 24.0 - 26.0 +11/=26/-13 48.00%
Komodo64 SSE Version 4 - Strelka 5.1(X1) 24.0 - 26.0 +12/=24/-14 48.00%
Komodo64 SSE Version 4 - Critter 1.4 64-bit SSE4(x1) 24.0 - 26.0 +14/=20/-16 48.00%

TOTAL 99,5 – 100.5 = 49.75%

GAMES:

http://www.megaupload.com/?d=EYPRI2IB

Previous Test at 25 min / game (December 2011)
GUI Fritz 12
Book Perfect 2012
Intel X980 3.33 Mhz. 24 MEM RAM

Komodo64 SSE Version 4 - Critter 1.2 64-bit SSE4(x1) 8.5 - 11.5 +2/=13/-5 42.50%
Komodo64 SSE Version 4 - Deep Rybka 4.1 SSE42 x64(x1) 10.0 - 10.0 +2/=16/-2 50.00%
Komodo64 SSE Version 4 - Houdini 2.0c Pro x64(x1) 9.5 - 10.5 +4/=11/-5 47.50%
Komodo64 SSE Version 4 - Strelka 5.1(X1) 9.0 - 11.0 +5/=8/-7 45.00%

TOTAL 37.0 – 43.0 = 46,25%

At the incremental time control Komodo 1.4 performs better than at 25 minutes against each rival. An excellent result (48%) against Houdini 2.0c PRO.

Regards from Barcelona.

Tom.

[lkaufman]

I think what it means is that our time control management for time controls with substantial main time is not good. We have already improved it noticeably since the release. So presumably the next version should play more like the level shown in your test here, even if we make no changes to the program itself.
By the way, the Rybka forum reports a 53-47 victory for Komodo 4 over Houdini 2 at 40 moves in 40 minutes on fast hardware. More evidence that we "scale" better than Houdini.

[MM]

I think what it means is that our time control management for time controls with substantial main time is not good. We have already improved it noticeably since the release. So presumably the next version should play more like the level shown in your test here, even if we make no changes to the program itself.
By the way, the Rybka forum reports a 53-47 victory for Komodo 4 over Houdini 2 at 40 moves in 40 minutes on fast hardware. More evidence that we "scale" better than Houdini.

Hi Larry, how much do you think to gain in elo with MP using a single core?
I mean, do you think that MP, for some reasons, will perform better than SP, using one single core?
And another question..do you think Komodo MP will gain again using 4 cores? For example, if using 1 core Komodo 4 MP will be +-30 elo to Houdini, Komodo 4 MP using all cores (eg 4) will be +-?

Thank you

Regards

Regards

[lkaufman]

I think what it means is that our time control management for time controls with substantial main time is not good. We have already improved it noticeably since the release. So presumably the next version should play more like the level shown in your test here, even if we make no changes to the program itself.
By the way, the Rybka forum reports a 53-47 victory for Komodo 4 over Houdini 2 at 40 moves in 40 minutes on fast hardware. More evidence that we "scale" better than Houdini.

Hi Larry, how much do you think to gain in elo with MP using a single core?
I mean, do you think that MP, for some reasons, will perform better than SP, using one single core?
And another question..do you think Komodo MP will gain again using 4 cores? For example, if using 1 core Komodo 4 MP will be +-30 elo to Houdini, Komodo 4 MP using all cores (eg 4) will be +-?

Thank you

Regards

Regards

MP won't perform any better on 1 core than the same version SP, but perhaps when we release MP the SP version will already be stronger.
We have no data on MP speedup yet, but if it is the same as Houdini then I expect a larger Elo gain than Houdini gets just because the evidence is that we scale better.

[beram]

Hi Larry,

does the same effect counts for Komodo 3 ? because I am testing now with Komodo 3 as I don't have Komodo 4

kind regards Bram

[lkaufman]

I would expect the same to be true of Komodo 3, because I don't recall time control changes between Komodo 3 and Komodo 4. Our next release will have totally revised (improved, I believe) time management so this should no longer be an issue.

[beram]

Hi Larry,

I have found mixed results for Komodo3 w32 and x64 as it goes about better results at more incremental TC.

First I have found no better results for Komodo 3 w32 when using longer incremental time controls.
At TC 2m5s, on dualcore T8100 (6,3 Fritzmark)Komodo3 scored 36% and at (doubled) TC 4m10s it scored 37%

As for Komodo3 x64, in matches against Houdini 1.5a it is quite a different story.
On my AMD X6 T1090 at TC 2m5s, Komodo3 scored 38% but at double time it leads after 38 games with 55,3 % !
I found that quite remarkable
Before that match I played a 39 games against H1.5 (instead of H1.5a)and also there Komodo 3 won with a score of approx 58 %

I will have a closer look on the games to see what happened in general in the play of Komodo3 x64 as far as my humanoid eyes and brains (+1900 ELO) can see.

Code: Select all

T8100, Blitz 2m+5s  Bram Testsuite 2.0
                      
1   Houdini 1.5 w32   +23/=18/-9 64.00%   32.0/50
2   Komodo32 3 32bit  +9/=18/-23 36.00%   18.0/50

T8100, Blitz 4m+10s  Bram Testsuite 2.0
                      
1   Houdini 1.5 w32   +22/=19/-9 63.00%   31.5/50
2   Komodo32 3 32bit  +9/=19/-22 37.00%   18.5/50

[lkaufman]

Hi Larry,

I have found mixed results for Komodo3 w32 and x64 as it goes about better results at more incremental TC.

First I have found no better results for Komodo 3 w32 when using longer incremental time controls.
At TC 2m5s, on dualcore T8100 (6,3 Fritzmark)Komodo3 scored 36% and at (doubled) TC 4m10s it scored 37%

As for Komodo3 x64, in matches against Houdini 1.5a it is quite a different story.
On my AMD X6 T1090 at TC 2m5s, Komodo3 scored 38% but at double time it leads after 38 with 55,3 % !
I found that quite remarkable
Before that match I played a 39 games against H1.5 (instead of H1.5a)and also there Komodo 3 won with a score of approx 58 %

I will have a closer look on the games to see what happened in general in the play of Komodo3 x64 as far as my humanoid eyes and brains (+1900 ELO) can see.

Code: Select all

T8100, Blitz 2m+5s  Bram Testsuite 2.0
                      
1   Houdini 1.5 w32   +23/=18/-9 64.00%   32.0/50
2   Komodo32 3 32bit  +9/=18/-23 36.00%   18.0/50

T8100, Blitz 4m+10s  Bram Testsuite 2.0
                      
1   Houdini 1.5 w32   +22/=19/-9 63.00%   31.5/50
2   Komodo32 3 32bit  +9/=19/-22 37.00%   18.5/50

Regarding the comparison between 4'+10" and 2'+5", this is interesting in terms of my theory about Komodo scaling better than Houdini, and your overall results do confirm this. Probably the fact that the improvement was only seen at 64 bit and not 32 bit is just sample error; fifty game samples are tiny. It is however not the relevant comparison for this topic; both time controls you used have the same base time to increment ratio. The proper comparison would be with a small-increment test; for example comparing your 2'+5" test to a 5'+2" test. In the case of your 64 bit results, they seem to confirm my thesis, because although you did not run tests where the increment was small, other people have done so, and your results do seem to be better than most people have reported for Komodo 3. I'm pretty much convinced by various tests that my thesis is correct, and that it means that our time management for the usual small-increment tests was not good. We have already improved our time allocation rules for the next release, so we should do roughly as well at all time controls as we now seem to do only with big-increment tests.

[beram]

and here final results of my tests
I am impressed by Komodo at this TC
It is the first that beats Houdini 1.5 in my own tests

Code: Select all

Bram-PC, Blitz 4m+10s  Bram Testsuite 2.0
                      
1   Komodo64 3        +18/=17/-15 53.00%   26.5/50
2   Houdini 1.5a x64  +15/=17/-18 47.00%   23.5/50

Bram-PC, Blitz 4m+2s  Bram Testsuite 2.0
                     
1   Houdini 1.5 x64  +16/=19/-15 51.00%   25.5/50
2   Komodo64 3       +15/=19/-16 49.00%   24.5/50