Testing Stockfish 11-03-13. 480 Games.

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

Sorry, I am not allowed to delete my last post wich is wrong-15 minutes limit- . Here you have the right result of this test.

Tom

... Note to the moderatos of this forum: Please delete my previous post, which is wrong. Thanks!.

Tom.

TESTING STOCKFISH DEVELOPMENT 051014MZ: 480 Games

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

SF051014MZ 64 SSE4.2 LP_ - Houdini 4 x64_st_X6_CT0 24.0 - 16.0 +12/=24/-4 60.00%
SF051014MZ 64 SSE4.2 LP_ - Komodo 8 64-bit_6_NOB 17.5 - 22.5 +5/=25/-10 43.75%
SF051014MZ 64 SSE4.2 LP_ - Gull 3 x64 XP 23.5 - 16.5 +11/=25/-4 58.75%

Time Control= 2+2

SF051014MZ 64 SSE4.2 LP_ - Houdini 4 x64_st_X6_CT0 21.5 - 18.5 +10/=23/-7 53.75%
SF051014MZ 64 SSE4.2 LP_ - Komodo 8 64-bit_6_NOB 21.0 - 19.0 +7/=28/-5 52.50%
SF051014MZ 64 SSE4.2 LP_ - Gull 3 x64 XP 22.5 - 17.5 +10/=25/-5 56.25%

Score using 6 cores: 130.0 – 110.0 = 54.17%
240 Games =
http://www.mediafire.com/view/wu8e5tn9x ... 0Games.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012t –Limit 8 moves. Sedat-
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

SF051014MZ 64 SSE4.2 LP_ - Houdini 4 Pro x64-A_OK 21.0 - 19.0 +9/=24/-7 52.50%
SF051014MZ 64 SSE4.2 LP_ - Komodo 8 64-bit_4_NOB 20.5 - 19.5 +10/=21/-9 51.25%
SF051014MZ 64 SSE4.2 LP_ - Gull 3 x64 XP 27.5 - 12.5 +16/=23/-1 68.75%

Time Control= 2+2
SF051014MZ 64 SSE4.2 LP_ - Houdini 4 Pro x64-A_OK 21.5 - 18.5 +9/=25/-6 53.75%
SF051014MZ 64 SSE4.2 LP_ - Komodo 8 64-bit_4_NOB 18.0 - 22.0 +7/=22/-11 45.00%
SF051014MZ 64 SSE4.2 LP_ - Gull 3 x64 XP 24.0 - 16.0 +11/=26/-3 60.00%

Score using 4 cores: 132.5 – 107.5= 55.21%

240 Games:
http://www.mediafire.com/view/j2a23dq1y ... 0games.pgn

Segmenting by Time Control:

Fixed TC = 134.0 – 106.0 = 55.83%
Incremental TC = 128.5 – 121.5 = 53.54%

GLOBAL SCORE: 262.5 – 217.5 = 54.69%

Against : Houdini 4.0 St. Ct0 (3227) = 55.00% ; Komodo 8 (3266) = 48.12%, Gull 3 XP (3199) = 60.94%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3264

Error bars= +/- 23 EEP

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT 071014: 480 Games

Timestamp: 1412415389 Bench: 7543790

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

Stockfish 071014 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 21.5 - 18.5 +7/=29/-4 53.75%
Stockfish 071014 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 21.0 - 19.0 +8/=26/-6 52.50%
Stockfish 071014 64 SSE4.2_ - Gull 3 x64 XP 20.5 - 19.5 +4/=33/-3 51.25%

Time Control= 2+2

Stockfish 071014 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 22.0 - 18.0 +9/=26/-5 55.00%
Stockfish 071014 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 21.5 - 18.5 +10/=23/-7 53.75%
Stockfish 071014 64 SSE4.2_ - Gull 3 x64 XP 25.0 - 15.0 +13/=24/-3 62.50%

Score using 6 cores: 131.5 – 108.5= 54.79%
240 Games =
http://www.mediafire.com/view/r3avbn1ek ... 0Games.pgn


i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012t –Limit 8 moves. Sedat-
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

Stockfish 071014 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 22.5 - 17.5 +10/=25/-5 56.25%
Stockfish 071014 64 SSE4.2_ - Komodo 8 64-bit_4_NOB 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish 071014 64 SSE4.2_ - Gull 3 x64 XP 22.0 - 18.0 +9/=26/-5 55.00%

Time Control= 2+2
Stockfish 071014 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 25.5 - 14.5 +14/=23/-3 63.75%
Stockfish 071014 64 SSE4.2_ - Komodo 8 64-bit_4_NOB 19.0 - 21.0 +7/=24/-9 47.50%
Stockfish 071014 64 SSE4.2_ - Gull 3 x64 XP 23.0 - 17.0 +11/=24/-5 57.50%

Score using 4 cores: 133.5 – 106.5= 55.62%
240 Games:
http://www.mediafire.com/view/cd379xb3l ... 0games.pgn

Segmenting by Time Control:

Fixed TC = 129.0 – 111.0 = 53.75%
Incremental TC = 136.0 – 104.0 = 56.67%

GLOBAL SCORE: 265.0 – 215.0 = 55.21%

Against : Houdini 4.0 St. Ct0 (3227) = 57.19% ; Komodo 8 (3266) = 51.87%, Gull 3 XP (3199) = 56.56%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3267

Error bars= +/- 23 EEP

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH IPMAN 240914IPx = GAMES 481 to 960

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014t – Sedat – Limit 8 moves
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

SF 240914IPx 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 22.5 - 17.5 +9/=27/-4 56.25%
SF 240914IPx 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 21.0 - 19.0 +10/=22/-8 52.50%
SF 240914IPx 64 SSE4.2_ - Gull 3 x64 XP 23.0 - 17.0 +9/=28/-3 57.50%

Time Control= 2+2

SF 240914IPx 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 20.5 - 19.5 +7/=27/-6 51.25%
SF 240914IPx 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 22.5 - 17.5 +9/=27/-4 56.25%
SF 240914IPx 64 SSE4.2_ - Gull 3 x64 XP 26.0 - 14.0 +13/=26/-1 65.00%

Score using 6 cores: 135.5 – 104.5 = 56.46%


i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014t –Sedat- 8 moves
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

SF 240914IPx 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 22.0 - 18.0 +8/=28/-4 55.00%
SF 240914IPx 64 SSE4.2_ - Komodo 8 64-bit_4_NOB 19.0 - 21.0 +8/=22/-10 47.50%
SF 240914IPx 64 SSE4.2_ - Gull 3 x64 XP 24.5 - 15.5 +12/=25/-3 61.25%

Time Control: 2+2

SF 240914IPx 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 21.0 - 19.0 +8/=26/-6 52.50%
SF 240914IPx 64 SSE4.2_ - Komodo 8 64-bit_4_NOB 21.0 - 19.0 +8/=26/-6 52.50%
SF 240914IPx 64 SSE4.2_ - Gull 3 x64 XP 24.0 - 16.0 +11/=26/-3 60.00%

Score using 4 Cores= 131.5 – 108.5 = 54.79%
240 Games:

http://www.mediafire.com/view/cx9qva...s_2nd_Test.pgn

Segmenting by Time Control:

Fixed TC = 132.0 – 108.0= 55.00%
Incremental TC = 135.0 – 105.0 = 56.25%

Global Score= 267.0 – 213.0= 55.62%

Against : Houdini 4.0 St. Ct0 (3227) = 53.75%; Komodo 8 (3266) = 52.19%, Gull 3 XP (3199) = 60.94%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3270

Error bars= +/- 23 EEP

The second part of the test of SF 240914IPx has been a bit surprising for me: The score has been only 3270, 30 Elo points below the previous one (3300). I have decided to start just now a new 480 games test to clarify whether this compile was very lucky in the first test or very unlucky in the second one. After 1440 games the error bars will be only +/-13 Elo points. Let's see!.

Regards from Barelona,

Tom.
Top
User avatar
Ozymandias
Posts: 1537
Joined: Sun Oct 25, 2009 2:30 am

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Ozymandias »

Tomcass wrote:I have decided to start just now a new 480 games test to clarify whether this compile was very lucky in the first test or very unlucky in the second one.
That would be a good time, to update you top-of-the-class list :wink:
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

Ozymandias wrote:
Tomcass wrote:I have decided to start just now a new 480 games test to clarify whether this compile was very lucky in the first test or very unlucky in the second one.
That would be a good time, to update you top-of-the-class list :wink:
You are right, my friend. I have decided to complete some tests so that every engine or compile in my ranking has non less than 960 games, with error bars of 16 Elo points.

Un fuerte abrazo desde Barcelona, Juan.

Tom
Top
User avatar
Ozymandias
Posts: 1537
Joined: Sun Oct 25, 2009 2:30 am

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Ozymandias »

Tomcass wrote:I have decided to complete some tests so that every engine or compile in my ranking has non less than 960 games
Good initiative. As for the rest, PM.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH IPMAN 240914IPx = GAMES 961 to 1440

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014t – Sedat – Limit 8 moves
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

SF 240914IPx 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 22.0 - 18.0 +9/=26/-5 55.00%
SF 240914IPx 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 20.5 - 19.5 +6/=29/-5 51.25%
SF 240914IPx 64 SSE4.2_ - Gull 3 x64 XP 25.0 - 15.0 +12/=26/-2 62.50%

Time Control= 2+2

SF 240914IPx 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 26.5 - 13.5 +15/=23/-2 66.25%
SF 240914IPx 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 20.0 - 20.0 +6/=28/-6 50.00%
SF 240914IPx 64 SSE4.2_ - Gull 3 x64 XP 23.0 - 17.0 +12/=22/-6 57.50%

Score using 6 cores: 137.0 – 103.0 = 57.08%
240 Games: http://www.mediafire.com/view/gr0hrt...s_3rd_test.pgn


i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014t –Sedat- 8 moves
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

SF 240914IPx 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 27.5 - 12.5 +16/=23/-1 68.75%
SF 240914IPx 64 SSE4.2_ - Komodo 8 64-bit_4_NOB 22.5 - 17.5 +12/=21/-7 56.25%
SF 240914IPx 64 SSE4.2_ - Gull 3 x64 XP 27.0 - 13.0 +15/=24/-1 67.50%

Time Control: 2+2

SF 240914IPx 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 21.0 - 19.0 +4/=34/-2 52.50%
SF 240914IPx 64 SSE4.2_ - Komodo 8 64-bit_4_NOB 20.5 - 19.5 +8/=25/-7 51.25%
SF 240914IPx 64 SSE4.2_ - Gull 3 x64 XP 28.0 - 12.0 +17/=22/-1 70.00%

Score using 4 Cores= 146.5 – 93.5 = 61.04%
240 Games:
http://www.mediafire.com/view/ojp6ax..._23rd_Test.pgn

Segmenting by Time Control:

Fixed TC = 144.5 – 95.5 = 60.21%
Incremental TC = 139.0 – 101.0 = 57.92%

Global Score= 283.5 – 196.5= 59.06%

Against : Houdini 4.0 St. Ct0 (3227) = 60.62%; Komodo 8 (3266) = 52.19%, Gull 3 XP (3199) = 64.37%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3294

Error bars= +/- 23 EEP

----------------------------------------

SUMMARY OF TEST 1 (Games 1 to 480)

Global Score= 287.0 – 193.0 = 59.79%

Against : Houdini 4.0 (3227)= 59.06%; Komodo 8 (3266) = 55.62%, Gull 3 XP (3199) = 64.69%

SUMMARY OF TEST 2 (Games 481 to 960)

Global Score= 267.0 – 213.0= 55.62%

Against : Houdini 4.0 (3227)= 53.75%; Komodo 8 (3266) = 52.19%, Gull 3 XP (3199) = 60.94%

SUMMARY OF TEST 3 (Games 961 to 1440)

Global Score= 283.5 – 196.5= 59.06%

Against : Houdini 4.0 (3227) = 60.62%; Komodo 8 (3266) = 52.19%, Gull 3 XP (3199) = 64.37%

--------------------------------------


GLOBAL SCORE FOR STOCKFISH IPMAN 240914X AFTER 1440 GAMES:

837.5 – 602.5 = 58.16%

Against : Houdini 4.0 (3227)= 57.81%; Komodo 8 (3266) = 53.33%, Gull 3 XP (3199) = 63.33%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3288

Error bars= +/- 13 EEP

Regards from Barcelona,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

I have decided to test again the best SF Development scorer in my tests -280614- to minimize error bars. Here you have the third test:

TESTING STOCKFISH DEVELOPMENT 280614 3rd Test: Games 961 to 1440

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

Stockfish 280614 64 SSE4.2_D - Houdini 4 x64_st_X6_CT0 22.5 - 17.5 +11/=23/-6 56.25%
Stockfish 280614 64 SSE4.2_D - Komodo 8 64-bit_6_NOB 19.5 - 20.5 +7/=25/-8 48.75%
Stockfish 280614 64 SSE4.2_D - Gull 3 x64 XP 23.5 - 16.5 +14/=19/-7 58.75%

Time Control= 2+2

Stockfish 280614 64 SSE4.2_D - Houdini 4 x64_st_X6_CT0 21.0 - 19.0 +13/=16/-11 52.50%
Stockfish 280614 64 SSE4.2_D - Komodo 8 64-bit_6_NOB 18.5 - 21.5 +8/=21/-11 46.25%
Stockfish 280614 64 SSE4.2_D - Gull 3 x64 XP 26.0 - 14.0 +14/=24/-2 65.00%

Score using 6 cores: 131.0 – 109.0= 54.58%
240 Games =
http://www.mediafire.com/view/dx2g5ihyo ... d_test.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012t –Limit 8 moves. Sedat-
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

Stockfish 280614 64 SSE4.2_D - Houdini 4 Pro x64-A_OK 18.0 - 22.0 +6/=24/-10 45.00%
Stockfish 280614 64 SSE4.2_D - Komodo 8 64-bit_4_NOB 17.5 - 22.5 +5/=25/-10 43.75%
Stockfish 280614 64 SSE4.2_D - Gull 3 x64 XP 26.0 - 14.0 +16/=20/-4 65.00%

Time Control= 2+2
Stockfish 280614 64 SSE4.2_D - Houdini 4 Pro x64-A_OK 24.5 - 15.5 +14/=21/-5 61.25%
Stockfish 280614 64 SSE4.2_D - Komodo 8 64-bit_4_NOB 20.0 - 20.0 +9/=22/-9 50.00%
Stockfish 280614 64 SSE4.2_D - Gull 3 x64 XP 27.5 - 12.5 +15/=25/-0 68.75%

Score using 4 cores: 133.5 – 106.5= 55.62%

240 Games:
http://www.mediafire.com/view/q9ix484j4 ... _2ndTest.p

Segmenting by Time Control:

Fixed TC = 127.0 – 113.0 = 52.91%
Incremental TC = 137.5 – 102.5 = 57.29%

GLOBAL SCORE: 264.5 – 215.5 = 55.10%

Against : Houdini 4.0 St. Ct0 (3227) = 53.75% ; Komodo 8 (3266) = 47.19%, Gull 3 XP (3199) = 64.37%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3267

Error bars= +/- 23 EEP

In a few minutes you will be able to see the fourth test and the summary for this SF Dev. 280614 engine after 1920 games.

Regards,

Tom.

... please note that although in the name of the file where games are stored appears the name 2nd test, in fact it is the third test, not the second one.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT 280614 4rth Test: Games 1441 to 1920

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

Stockfish 280614 64 SSE4.2_D - Houdini 4 x64_st_X6_CT0 23.5 - 16.5 +10/=27/-3 58.75%
Stockfish 280614 64 SSE4.2_D - Komodo 8 64-bit_6_NOB 22.0 - 18.0 +8/=28/-4 55.00%
Stockfish 280614 64 SSE4.2_D - Gull 3 x64 XP 25.0 - 15.0 +12/=26/-2 62.50%

Time Control= 2+2

Stockfish 280614 64 SSE4.2_D - Houdini 4 x64_st_X6_CT0 18.0 - 22.0 +7/=22/-11 45.00%
Stockfish 280614 64 SSE4.2_D - Komodo 8 64-bit_6_NOB 24.5 - 15.5 +12/=25/-3 61.25%
Stockfish 280614 64 SSE4.2_D - Gull 3 x64 XP 24.0 - 16.0 +11/=26/-3 60.00%

Score using 6 cores: 137.0 – 103.0= 57.08%
240 Games =
http://www.mediafire.com/view/azb4ynx3d ... rdTest.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012t –Limit 8 moves. Sedat-
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

Stockfish 280614 64 SSE4.2_D - Houdini 4 Pro x64-A_OK 22.0 - 18.0 +10/=24/-6 55.00%
Stockfish 280614 64 SSE4.2_D - Komodo 8 64-bit_4_NOB 19.0 - 21.0 +7/=24/-9 47.50%
Stockfish 280614 64 SSE4.2_D - Gull 3 x64 XP 21.5 - 18.5 +9/=25/-6 53.75%

Time Control= 2+2
Stockfish 280614 64 SSE4.2_D - Houdini 4 Pro x64-A_OK 22.5 - 17.5 +8/=29/-3 56.25%
Stockfish 280614 64 SSE4.2_D - Komodo 8 64-bit_4_NOB 23.5 - 16.5 +11/=25/-4 58.75%
Stockfish 280614 64 SSE4.2_D - Gull 3 x64 XP 27.5 - 12.5 +18/=19/-3 68.75%

Score using 4 cores: 136.0 – 104.0= 56.67%
240 Games:
http://www.mediafire.com/view/7mrjygwfo ... rdTest.pgn

Segmenting by Time Control:

Fixed TC = 133.0 – 107.0 = 55.42%
Incremental TC = 140.0 – 100.0 = 58.33%

GLOBAL SCORE: 273.0 – 207.0 = 56.87%

Against : Houdini 4.0 St. Ct0 (3227) = 53.75% ; Komodo 8 (3266) = 55.62%, Gull 3 XP (3199) = 61.25%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3279

Error bars= +/- 23 EEP

Regards,

Tom.

... please note that although in the file where games are stored appears the name 3rd test, in fact it is the fourth test, not the third one.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH280614 64 SSE4.2_D : Summary after 1920 games.

Since this engine was the best SF Development scorer in my tests, I decided keep testing it until 1920 games to get only +/- 11 error bars. This is the summary of the 4 tests. Please note that in the test 1 and 2 I used Komodo 7 (3206 Elo) and in tests 3 and 4 Komodo 8 (3266 Elo).

Bench : 8095369 Timestamp: 1403978792

SUMMARY OF TEST 1 (Games 1 to 480)
289.5 – 190.5 = 60.31%

Against : Houdini 4.0 St. Ct0 (3227) = 58.12% ; Komodo 7 (3206) =60.62%, Gull 3 XP (3199) = 62.19%

SUMMARY OF TEST 2 (Games 481 to 960)
294.5 – 185.5 = 61.35%

Against : Houdini 4.0 St. Ct0 (3227) = 59.69% ; Komodo 7 (3206) =62.81%, Gull 3 XP (3199) = 61.56%

SUMMARY OF TEST 3 (Games 961 to 1440)
264.5 – 215.5 = 55.10%

Against : Houdini 4.0 St. Ct0 (3227) = 53.75% ; Komodo 8 (3266) = 47.19%, Gull 3 XP (3199) = 64.37%

SUMMARY OF TEST 4 (Games 1441 to 1920)
273.0 – 207.0 = 56.87%

Against : Houdini 4.0 St. Ct0 (3227) = 53.75% ; Komodo 8 (3266) = 55.62%, Gull 3 XP (3199) = 61.25%

TOTAL SCORE AFTER 1920 GAMES: 58,41%

Average Estimated Elo Opponents = 3221
Estimated Elo Performance= 3280
Error bars= +/- 11 EEP

Only as a reference, the two latest SF Development I tested (04/10/14 and 07/10/14) performed at 3279 and 3267 after 480 games each, very similar -within error bars- to SF Dev. 280614.

Regards from Barcelona,

Tom.
Top

Return to “Computer Chess Club: Tournaments and Matches”