Testing Komodo4

[Tomcass]

Gui: Fritz 12
Time Control: 2 min + 1 sec/game
Book: Perfect 2012
Ponder: Off
BNG (Billion Nodes per Game): about 0,2 for Komodo4

201112Komodo4 2011

Komodo64 SSE Version 4 - Critter 1.2 64-bit SSE4(x1) 12.5 - 12.5 +5/=15/-5 50.00%
Komodo64 SSE Version 4 - Deep Rybka 4.1 SSE42 x64(x1) 9.0 - 16.0 +3/=12/-10 36.00%
Komodo64 SSE Version 4 - Houdini 2.0c Pro x64(x1) 10.5 - 14.5 +6/=9/-10 42.00%
Komodo64 SSE Version 4 - Strelka 5.1(X1) 14.0 - 11.0 +9/=10/-6 56.00%


The games: http://www.megaupload.com/?d=NJ3L6I5A

My feelings (after only 100 games):

- There is a substantial difference in speed between SSE and NO-SSE versions. About 15/16% better SSE than no-SSE.
- Komodo 4 seems better tuned against Houdini and Strelka (Houdini derivative) than against Rybka 4.1
- Perceptible improvement in ELO in relation to Komodo3, but I think not more than 35/40 points. (To be confirmed, my first guess can be obviously wrong).

As a comparation, I add a previous test of Komodo 3 against the same opponents, at a longer time control, so that I can explain better some of my feelings:

201104. Houdini 2.0cKomodo3DR4Criter-2 2011

1 2 3 4 5
1 Houdini 2.0c Pro x64(x1) 11.5 - 8.513.0 - 7.015.0 - 5.012.5 - 6.5** 52.0/79
2 Strelka 5.1(X1) 8.5 - 11.510.0 - 10.011.0 - 9.010.5 - 9.5 ** 40.0/80
3 Critter 1.2 64-bit SSE4(x1) 7.0 - 13.010.0 - 10.09.0 - 11.011.0 - 9.0 ** 37.0/80 0.00
[b]4 Komodo64 3(x1) 5.0 - 15.0 9.0 - 11.0 11.0 - 9.0 12.0 - 8.0 ** 37.0/80 0.00[/b]
5 Deep Rybka 4 SSE42 x64(x1) 6.5 - 12.59.5 - 10.59.0 - 11.08.0 - 12.0 ** 33.0/79

I have just started testing Komodo4 against the same opponents at a longer time control (25 minutes per game). Results in about three days.

Regards from Barcelona.

Tom.

[Tomcass]

The link to the games was wrong. Sorry. Here you have the good one:

http://www.megaupload.com/?d=LU7T7369

Tom

[MM]

Gui: Fritz 12
Time Control: 2 min + 1 sec/game
Book: Perfect 2012
Ponder: Off
BNG (Billion Nodes per Game): about 0,2 for Komodo4

201112Komodo4 2011

Komodo64 SSE Version 4 - Critter 1.2 64-bit SSE4(x1) 12.5 - 12.5 +5/=15/-5 50.00%
Komodo64 SSE Version 4 - Deep Rybka 4.1 SSE42 x64(x1) 9.0 - 16.0 +3/=12/-10 36.00%
Komodo64 SSE Version 4 - Houdini 2.0c Pro x64(x1) 10.5 - 14.5 +6/=9/-10 42.00%
Komodo64 SSE Version 4 - Strelka 5.1(X1) 14.0 - 11.0 +9/=10/-6 56.00%


The games: http://www.megaupload.com/?d=NJ3L6I5A

My feelings (after only 100 games):

- There is a substantial difference in speed between SSE and NO-SSE versions. About 15/16% better SSE than no-SSE.
- Komodo 4 seems better tuned against Houdini and Strelka (Houdini derivative) than against Rybka 4.1
- Perceptible improvement in ELO in relation to Komodo3, but I think not more than 35/40 points. (To be confirmed, my first guess can be obviously wrong).

As a comparation, I add a previous test of Komodo 3 against the same opponents, at a longer time control, so that I can explain better some of my feelings:

201104. Houdini 2.0cKomodo3DR4Criter-2 2011

1 2 3 4 5
1 Houdini 2.0c Pro x64(x1) 11.5 - 8.513.0 - 7.015.0 - 5.012.5 - 6.5** 52.0/79
2 Strelka 5.1(X1) 8.5 - 11.510.0 - 10.011.0 - 9.010.5 - 9.5 ** 40.0/80
3 Critter 1.2 64-bit SSE4(x1) 7.0 - 13.010.0 - 10.09.0 - 11.011.0 - 9.0 ** 37.0/80 0.00
[b]4 Komodo64 3(x1) 5.0 - 15.0 9.0 - 11.0 11.0 - 9.0 12.0 - 8.0 ** 37.0/80 0.00[/b]
5 Deep Rybka 4 SSE42 x64(x1) 6.5 - 12.59.5 - 10.59.0 - 11.08.0 - 12.0 ** 33.0/79

I have just started testing Komodo4 against the same opponents at a longer time control (25 minutes per game). Results in about three days.

Regards from Barcelona.

Tom.

Thanks.

Regards

[lkaufman]

Gui: Fritz 12
Time Control: 2 min + 1 sec/game
Book: Perfect 2012
Ponder: Off
BNG (Billion Nodes per Game): about 0,2 for Komodo4

201112Komodo4 2011

Komodo64 SSE Version 4 - Critter 1.2 64-bit SSE4(x1) 12.5 - 12.5 +5/=15/-5 50.00%
Komodo64 SSE Version 4 - Deep Rybka 4.1 SSE42 x64(x1) 9.0 - 16.0 +3/=12/-10 36.00%
Komodo64 SSE Version 4 - Houdini 2.0c Pro x64(x1) 10.5 - 14.5 +6/=9/-10 42.00%
Komodo64 SSE Version 4 - Strelka 5.1(X1) 14.0 - 11.0 +9/=10/-6 56.00%


The games: http://www.megaupload.com/?d=NJ3L6I5A

My feelings (after only 100 games):

- There is a substantial difference in speed between SSE and NO-SSE versions. About 15/16% better SSE than no-SSE.
- Komodo 4 seems better tuned against Houdini and Strelka (Houdini derivative) than against Rybka 4.1
- Perceptible improvement in ELO in relation to Komodo3, but I think not more than 35/40 points. (To be confirmed, my first guess can be obviously wrong).

As a comparation, I add a previous test of Komodo 3 against the same opponents, at a longer time control, so that I can explain better some of my feelings:

201104. Houdini 2.0cKomodo3DR4Criter-2 2011

1 2 3 4 5
1 Houdini 2.0c Pro x64(x1) 11.5 - 8.513.0 - 7.015.0 - 5.012.5 - 6.5** 52.0/79
2 Strelka 5.1(X1) 8.5 - 11.510.0 - 10.011.0 - 9.010.5 - 9.5 ** 40.0/80
3 Critter 1.2 64-bit SSE4(x1) 7.0 - 13.010.0 - 10.09.0 - 11.011.0 - 9.0 ** 37.0/80 0.00
[b]4 Komodo64 3(x1) 5.0 - 15.0 9.0 - 11.0 11.0 - 9.0 12.0 - 8.0 ** 37.0/80 0.00[/b]
5 Deep Rybka 4 SSE42 x64(x1) 6.5 - 12.59.5 - 10.59.0 - 11.08.0 - 12.0 ** 33.0/79

I have just started testing Komodo4 against the same opponents at a longer time control (25 minutes per game). Results in about three days.

Regards from Barcelona.

Tom.

You report a 37/80 score for Komodo 3 at 25 minutes per game. But was there any increment? The time control mechanism is quite different for increment and no-increment games, so any comparison between the two is suspect. Do you also have fast results against same oppoents for Komodo 3?
Your report of 15% speedup from SSE is surprising. I measured 9-10% on my computers.

Regards,
Larry

[Tomcass]

Hi Larry,

I am conscious about the difference between these two tests. This is the reason I have started a 80 games test (x 2 computers = 160 games) at 25 minutes per game with no increase of time, against the same opponents.

The difference between SSE and no-SSE is about 14% in one of my computers and 16% in the other. When I finish my current tests I will reconfirm that my calculations are ok. But this is not the first time that my computers produce different results. I have no idea why.

Regards, and thanks for your excellent work with Komodo.

Tom.

[lkaufman]

Hi Larry,

I am conscious about the difference between these two tests. This is the reason I have started a 80 test (x 2 computers = 160 games) at 25 minutes per game with no increase of time, against the same opponents.

The difference between SSE and no-SSE is about 14% in one of my computers and 16% in the other. When I finish my current tests I will reconfirm that my calculations are ok. But this is not the first time that my computers produce different results. I have no idea why.

Regards, and thanks for your excellent work with Komodo.

Tom.

Most of my sse vs. no-sse timings were done in Linux. I wouldn't have thought that the speedup would be much different in Windows, but perhaps I'm wrong.
You didn't answer as to whether you ever ran the same pairings at a short time control for Komodo 3. If you did, please post the results.
Thanks, Larry