Testing Stockfish 11-03-13. 480 Games.

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Sedat Canbaz
Posts: 3018
Joined: Thu Mar 09, 2006 11:58 am
Location: Antalya/Turkey

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Sedat Canbaz »

Hello Tom,

Thanks for your efforts...

Really very good results by AMos's Stockfish compile !

Btw, I think AMos 4EvEr is Marco Zerbinati from Italy, right ?

If so...exception that Marco is a SCCT Top Book Author,
That means, even he is expert in engine programing, congrats again to Marco Zerbinati !


Best,
Sedat
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

Hi, Sedat!

Thanks for your positive comments regarding my tests. Yes, the author of the MZ compile is Marco Zerbinati.

I wish you very good look in your new test. If you think I can help in any way, please let me know. I will be delighted to test the new Komodo 8 against the latest MZ compile in a couple of long time control games.

At my current time controls (4+0 and 2+2) you already know my forecast: The new Komodo 8 will be very close to the top: About 20 ELO below the leading SF MZ compile. But at a longer time control ..., I do not dare to forecast anything. :wink:

I will buy Komodo 8 as soon as it will appear.

Best regards,

Tom.
Sedat Canbaz
Posts: 3018
Joined: Thu Mar 09, 2006 11:58 am
Location: Antalya/Turkey

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Sedat Canbaz »

Tomcass wrote:Hi, Sedat!

Thanks for your positive comments regarding my tests. Yes, the author of the MZ compile is Marco Zerbinati.

I wish you very good look in your new test. If you think I can help in any way, please let me know. I will be delighted to test the new Komodo 8 against the latest MZ compile in a couple of long time control games.

At my current time controls (4+0 and 2+2) you already know my forecast: The new Komodo 8 will be very close to the top: About 20 ELO below the leading SF MZ compile. But at a longer time control ..., I do not dare to forecast anything. :wink:

I will buy Komodo 8 as soon as it will appear.

Best regards,

Tom.

Dear Tom,

Great news !

Yes... I need your help to see you as one of the testers in the current history match !

And for me will be honor to work with you and Wael !


Best,
Sedat
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT 300814

Bench: 7461881 Timestamp: 1409429021



i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 300814 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 25.0 - 15.0 +13/=24/-3 62.50%
Stockfish 300814 64 SSE4.2_ - Komodo 7 64-bitx6 23.5 - 16.5 +12/=23/-5 58.75%
Stockfish 300814 64 SSE4.2_ - Gull 3 x64 XP 25.5 - 14.5 +14/=23/-3 63.75%

Time Control= 2+2

Stockfish 300814 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 22.5 - 17.5 +10/=25/-5 56.25%
Stockfish 300814 64 SSE4.2_ - Komodo 7 64-bitx6 27.0 - 13.0 +16/=22/-2 67.50%
Stockfish 300814 64 SSE4.2_ - Gull 3 x64 XP 26.5 - 13.5 +15/=23/-2 66.25%

Score using 6 cores: 150.0 – 90.0 = 62.50%

240 Games:
http://www.mediafire.com/view/ua2a3j..._240Games_.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

Stockfish 300814 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 24.5 - 15.5 +14/=21/-5 61.25%
Stockfish 300814 64 SSE4.2_ - Komodo 7 64-bitx4 24.5 - 15.5 +13/=23/-4 61.25%
Stockfish 300814 64 SSE4.2_ - Gull 3 x64 XP 23.0 - 17.0 +11/=24/-5 57.50%

Time Control: 2+2

Stockfish 300814 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 21.0 - 19.0 +12/=18/-10 52.50%
Stockfish 300814 64 SSE4.2_ - Komodo 7 64-bitx4 23.0 - 17.0 +12/=22/-6 57.50%
Stockfish 300814 64 SSE4.2_ - Gull 3 x64 XP 26.5 - 13.5 +14/=25/-1 66.25%

Score using 4 Cores= 142.5 – 97.5= 59.37%
240 Games:
http://www.mediafire.com/view/djzuv3...V_240games.pgn

Segmenting by Time Control:

Fixed TC = 146.0 – 94.0= 60.83%
Incremental TC = 146.5 – 93.5= 61.04%

Global Score= 292.5 – 187.5 = 60.94%

Against : Houdini 4.0 St. Ct0 (3227) = 58.12% ; Komodo 7 (3206) = 61.25%, Gull 3 XP (3199) = 63.44%

Average Estimated Elo Opponents = 3211

Estimated Elo Performance= 3288


Error bars= +/- 23 EEP

Best score for a Stockfish Development so far!

SF250814aMZ x64 SSE4.2= 3299 (480 games)
SF250814MZ x64 SSE4.2= 3295 (960 games)
Stockfish 300814 64 SSE4.2= 3288 (480 games)
Stockfish 270714 IPMAN = 3284 (960 games)
Stockfish RockWood 190514 = 3283 (480 games)
Stockfish 5 64 SSE4.2 = 3272 (960 games)
Houdini 4.0 = 3227
Komodo 7 = 3206
Gull 3 XP = 3199
Critter 1.6a = 3104
Deep Rybka 4.1 = 3015

Regards,

Tom.
Sedat Canbaz
Posts: 3018
Joined: Thu Mar 09, 2006 11:58 am
Location: Antalya/Turkey

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Sedat Canbaz »

Dear Tom,

I've sent you a private mail...please check


Best,
Sedat
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH 010914 MZ = 240 GAMES

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c – Sedat – Limit 5 moves
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

SF010914MZ x64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 27.0 - 13.0 +16/=22/-2 67.50%
SF010914MZ x64 SSE4.2_ - Komodo 7 64-bitx6_NOB 24.0 - 16.0 +13/=22/-5 60.00%
SF010914MZ x64 SSE4.2_ - Gull 3 x64 XP 26.0 - 14.0 +15/=22/-3 65.00%

Score using 6 cores: 77.0 – 43.0= 64.17%
240 Games =
http://www.mediafire.com/view/7xnfxf..._120Games_.pgn


i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages= Allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

SF010914MZ x64 SSE4.2_ - Houdini 4 Pro x64-A_OK 23.5 - 16.5 +10/=27/-3 58.75%
SF010914MZ x64 SSE4.2_ - Komodo 7 64-bitx4_NOB_OK 26.0 - 14.0 +13/=26/-1 65.00%
SF010914MZ x64 SSE4.2_ - Gull 3 x64 XP 25.5 - 14.5 +16/=19/-5 63.75%

Score using 4 cores: 75.0 – 45.0= 62.50%
120 Games:
http://www.mediafire.com/view/5aojnu...Z_120games.pgn

GLOBAL SCORE: 152.0 – 88.0 = 63.33%

Against : Houdini 4.0 St. Ct0 (3227) = 63.12% ; Komodo 7 (3206) = 62.50%, Gull 3 XP (3199) = 64.37%

Average Estimated Elo Opponents = 3211
Estimated Elo Performance= 3304


What a great score!!

(Please note that this is only the first half of my standard test of 480 games. The error bars are therefore 32 rather than 23).

Error bars= +/- 32 EEP

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH 090914a MZ

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c – Sedat –
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

SF090914MZ x64 SSE4.2_6_BO - Houdini 4 x64_st_X6_CT0 24.0 - 16.0 +12/=24/-4 60.00%
SF090914MZ x64 SSE4.2_6_BO - Komodo 8 64-bit_6 19.0 - 21.0 +10/=18/-12 47.50%
SF090914MZ x64 SSE4.2_6_BO - Gull 3 x64 XP 22.5 - 17.5 +8/=29/-3 56.25%

Time Control= 2+2

SF090914MZ x64 SSE4.2_6_BO - Houdini 4 x64_st_X6_CT0 22.0 - 18.0 +12/=20/-8 55.00%
SF090914MZ x64 SSE4.2_6_BO - Komodo 8 64-bit_6 22.5 - 17.5 +11/=23/-6 56.25%
SF090914MZ x64 SSE4.2_6_BO - Gull 3 x64 XP 23.0 - 17.0 +8/=30/-2 57.50%

Score using 6 cores: 133.0 – 107.0 = 55.42%
240 Games =
http://www.mediafire.com/view/z519w5pnx ... 0Games.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c –Sedat-
No tablebases. No RTB used.
Large Pages= Allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

SF090914MZ x64 SSE4.2_4_BO - Houdini 4 Pro x64-A_OK 20.0 - 20.0 +8/=24/-8 50.00%
SF090914MZ x64 SSE4.2_4_BO - Komodo 8 64-bit_4 21.5 - 18.5 +8/=27/-5 53.75%
SF090914MZ x64 SSE4.2_4_BO - Gull 3 x64 XP 22.5 - 17.5 +12/=21/-7 56.25%

Time Control= 2+2

SF090914MZ x64 SSE4.2_4_BO - Houdini 4 Pro x64-A_OK 21.0 - 19.0 +11/=20/-9 52.50%
SF090914MZ x64 SSE4.2_4_BO - Komodo 8 64-bit_4 20.0 - 20.0 +6/=28/-6 50.00%
SF090914MZ x64 SSE4.2_4_BO - Gull 3 x64 XP 24.0 - 15.0 +15/=18/-6 61.54%

Score using 4 cores: 240 Games: 129.5 – 110.5
http://www.mediafire.com/view/krqh9nadd ... 0games.pgn

GLOBAL SCORE: 262.5 – 217.5 = 54.69%

Against : Houdini 4.0 St. Ct0 (3227) = 54.37% ; Komodo 8 (3266) = 51.87%, Gull 3 XP (3199) = 57.87%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3264


Error bars= +/- 23 EEP

Please note that the compile tested is the second release of SF 090914 MZ. (The first one had bugs). Not as strong as previous MZ releases. About 40 EEP below the top scorer.

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH 140914 MZ: 480 Games

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c – Sedat –
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

SF140914MZ 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 23.5 - 16.5 +11/=25/-4 58.75%
SF140914MZ 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 25.0 - 15.0 +15/=20/-5 62.50%
SF140914MZ 64 SSE4.2_ - Gull 3 x64 XP 25.0 - 15.0 +14/=22/-4 62.50%

Time Control= 2+2

SF140914MZ 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 23.0 - 17.0 +12/=22/-6 57.50%
SF140914MZ 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 22.0 - 18.0 +8/=28/-4 55.00%
SF140914MZ 64 SSE4.2_ - Gull 3 x64 XP 24.0 - 16.0 +12/=24/-4 60.00%

Score using 6 cores: 142.5 – 97.5 = 59.37%
240 Games =
http://www.mediafire.com/view/5pzqzb...6_240Games.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c –Sedat-
No tablebases. No RTB used.
Large Pages= Allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

SF140914MZ 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 24.0 - 16.0 +14/=20/-6 60.00%
SF140914MZ 64 SSE4.2_ - Komodo 8 64-bit_4_NOB 20.0 - 20.0 +6/=28/-6 50.00%
SF140914MZ 64 SSE4.2_ - Gull 3 x64 XP 26.5 - 13.5 +14/=25/-1 66.25%

Time Control= 2+2

SF140914MZ 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 19.5 - 20.5 +7/=25/-8 48.75%
SF140914MZ 64 SSE4.2_ - Komodo 8 64-bit_4_NOB 24.5 - 15.5 +10/=29/-1 61.25%
SF140914MZ 64 SSE4.2_ - Gull 3 x64 XP 24.0 - 16.0 +12/=24/-4 60.00%

Score using 4 cores: 138.5 – 101.5= 57.71%
240 Games:
http://www.mediafire.com/view/a58i68...Z_240games.pgn

Segmenting by Time Control:

Fixed TC = 144.0 – 96.0 = 60.00%
Incremental TC = 137.0 – 103.0= 57.08%


GLOBAL SCORE: 281.0 – 199.0 = 58.54%

Against : Houdini 4.0 St. Ct0 (3227) = 56.25 % ; Komodo 8 (3266) = 57.19%, Gull 3 XP (3199) = 62.19%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3291


Error bars= +/- 23 EEP

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH 190914 MZ: 480 Games

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c – Sedat –
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

SF190914MZ 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 23.5 - 16.5 +13/=21/-6 58.75%
SF190914MZ 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 24.5 - 15.5 +13/=23/-4 61.25%
SF190914MZ 64 SSE4.2_ - Gull 3 x64 XP 25.0 - 15.0 +15/=20/-5 62.50%

Time Control= 2+2

SF190914MZ 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 22.5 - 17.5 +11/=23/-6 56.25%
SF190914MZ 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 22.5 - 17.5 +9/=27/-4 56.25%
SF190914MZ 64 SSE4.2_ - Gull 3 x64 XP 25.5 - 14.5 +13/=25/-2 63.75%

Score using 6 cores: 143.5 – 96.5 = 59.79%
240 Games =
http://www.mediafire.com/view/8rmxaw1cw ... 0Games.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c –Sedat-
No tablebases. No RTB used.
Large Pages= Allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

SF190914MZ 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 27.0 - 13.0 +18/=18/-4 67.50%
SF190914MZ 64 SSE4.2_ - Komodo 8 64-bit_4_NOB 21.5 - 18.5 +10/=23/-7 53.75%
SF190914MZ 64 SSE4.2_ - Gull 3 x64 XP 27.0 - 13.0 +17/=20/-3 67.50%

Time Control= 2+2

SF190914MZ 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 18.5 - 21.5 +6/=25/-9 46.25%
SF190914MZ 64 SSE4.2_ - Komodo 8 64-bit_4_NOB 23.0 - 17.0 +13/=20/-7 57.50%
SF190914MZ 64 SSE4.2_ - Gull 3 x64 XP 21.5 - 18.5 +6/=31/-3 53.75%
Score using 4 cores: 138.5 – 101.5= 57.71%
240 Games:
http://www.mediafire.com/view/snl287y3o ... 0games.pgn

Segmenting by Time Control:

Fixed TC = 148.5 – 91.5 = 61.87%
Incremental TC = 133.5 – 106.5= 57.08%

GLOBAL SCORE: 282.0 – 198.0 = 58.75%

Against : Houdini 4.0 St. Ct0 (3227) = 57.19% ; Komodo 8 (3266) = 57.19%, Gull 3 XP (3199) = 61.87%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3292


Error bars= +/- 23 EEP

This score is very similar to the one got by SF140914 MZ. (See above). Only 1 Elo point of difference after 960 games (480 + 480).

After my latest tests I think there is some room for improvement in the incremental time control management of Stockfish.

Let’s test now the latest Stockfish Development.

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT 210914: 480 Games

Timestamp: 1411320767 Bench: 8331165

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c – Sedat –
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control = 4+0

Stockfish 210914 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 19.0 - 21.0 +6/=26/-8 47.50%
Stockfish 210914 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 21.5 - 18.5 +7/=29/-4 53.75%
Stockfish 210914 64 SSE4.2_ - Gull 3 x64 XP 24.5 - 15.5 +12/=25/-3 61.25%

Time Control= 2+2

Stockfish 210914 64 SSE4.2_ - Houdini 4 x64_st_X6_CT0 25.0 - 15.0 +14/=22/-4 62.50%
Stockfish 210914 64 SSE4.2_ - Komodo 8 64-bit_6_NOB 22.0 - 18.0 +7/=30/-3 55.00%
Stockfish 210914 64 SSE4.2_ - Gull 3 x64 XP 23.5 - 16.5 +7/=33/-0 58.75%

Score using 6 cores: 135.5 – 104.5 = 56.46%
240 Games =

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c –Sedat-
No tablebases. No RTB used.
Large Pages= Allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

Stockfish 210914 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 22.5 - 17.5 +11/=23/-6 56.25%
Stockfish 210914 64 SSE4.2_ - Komodo 8 64-bit_4 22.5 - 17.5 +10/=25/-5 56.25%
Stockfish 210914 64 SSE4.2_ - Gull 3 x64 XP 26.5 - 13.5 +15/=23/-2 66.25%

Time Control= 2+2

Stockfish 210914 64 SSE4.2_ - Houdini 4 Pro x64-A_OK 20.5 - 19.5 +6/=29/-5 51.25%
Stockfish 210914 64 SSE4.2_ - Komodo 8 64-bit_4 17.5 - 22.5 +6/=23/-11 43.75%
Stockfish 210914 64 SSE4.2_ - Gull 3 x64 XP 28.5 - 11.5 +18/=21/-1 71.25%

Score using 4 cores: 138.0 – 101.0= 57.50%
240 Games:
http://www.mediafire.com/view/t6t9x343r ... 0games.pgn

Segmenting by Time Control:

Fixed TC = 136.5 – 103.5 = 56.87%
Incremental TC = 137.0 – 104.0 = 57.08%

GLOBAL SCORE: 273.5 – 206.5 = 56.98%

Against : Houdini 4.0 St. Ct0 (3227) = 54.37% ; Komodo 8 (3266) = 52.19%, Gull 3 XP (3199) = 64.37%

Average Estimated Elo Opponents = 3231
Estimated Elo Performance= 3280


Error bars= +/- 23 EEP

Unfortunately, my tests do not show any substantial improvement for Stockfish Development in the last three months. :cry:

Stockfish 280614 Development = 3287 (after 960 games)

Regards,

Tom.