Testing Stockfish 11-03-13. 480 Games.

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT 080114 = 480 GAMES.

Bench: 8502826 Timestamp: 1389220684

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
No Large Pages
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 080114 64 SSE4.2X - Houdini 4 x64_st_X6_CT0 22.0 - 18.0 +7/=30/-3 55.00%
Stockfish 080114 64 SSE4.2X - Komodo TCECr 64-bitx6_ 27.5 - 12.5 +19/=17/-4 68.75%
Stockfish 080114 64 SSE4.2X - Critter 1.6a 64-bitX6_NOB 28.0 - 12.0 +20/=16/-4 70.00%

Time Control= 2+2

Stockfish 080114 64 SSE4.2X - Houdini 4 x64_st_X6_CT0 20.5 - 19.5 +7/=27/-6 51.25%
Stockfish 080114 64 SSE4.2X - Komodo TCECr 64-bitx6_ 22.0 - 18.0 +10/=24/-6 55.00%
Stockfish 080114 64 SSE4.2X - Critter 1.6a 64-bitX6_NOB 27.0 - 13.0 +16/=22/-2 67.50%

240 Games = http://www.mediafire.com/view/gem8g2ddo ... 0games.pgn
Score using 6 Cores= 147.0 -93.0 = 61.25%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
No Large Pages
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockfish 080114 64 SSE4.2x - Houdini 4 x64xCT0 18.5 - 21.5 +7/=23/-10 46.25%
Stockfish 080114 64 SSE4.2x - Komodo TCECr 64-bitx4 25.0 - 15.0 +14/=22/-4 62.50%
Stockfish 080114 64 SSE4.2x - Critter 1.6a 64-bitnob_4 28.0 - 12.0 +20/=16/-4 70.00%

Time Control= 2+2

Stockfish 080114 64 SSE4.2x - Houdini 4 x64xCT0 21.0 - 19.0 +6/=30/-4 52.50%
Stockfish 080114 64 SSE4.2x - Komodo TCECr 64-bitx4 23.5 - 16.5 +13/=21/-6 58.75%
Stockfish 080114 64 SSE4.2x - Critter 1.6a 64-bitnob_4 29.5 - 10.5 +20/=19/-1 73.75%

240 Games= http://www.mediafire.com/view/6dc0wyeeu ... 0games.pgn
Scores using 4 cores= 145.5 – 94.5= 60.62%


Segmenting by Time Control:

Fixed TC = 149.0 – 91.0 = 62.08%
Incremental TC = 143.5 – 96.5 = 59.79%

Global Score= 292.5 – 187.5 = 60.94%

Against : Houdini 4.0 St. Ct0 (3233) = 51.25% ; Komodo TCECr (3178) = 61.25% ; Critter 1.6a (3093) = 70.31%

Average Estimated Elo Opponents = 3168
Estimated Elo Performance= 3245.

This is a new best score ever in my tests.

The pace of development of Stockfish is really impressive !. This SF version has performed 12 EEP ahead of the best Houdini 4.0 Standard Contempt 0 in my computer and 2 EEP better than the latest Ipman compile, obviously within error bars.

Regards,

Tom.
Top
Kohflote
Posts: 240
Joined: Wed Sep 19, 2007 11:07 am
Location: Singapore

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Kohflote »

Dear Tom,

"TESTING STOCKFISH DEVELOPMENT 080114 = 480 GAMES.

Bench: 8502826 Timestamp: 1389220684"

- do you mean the version whereby the author is Henri Wiechers, Date: Jan 9 at abrok.eu/stockfish website?

Thank you & regards,
Kah Huat
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

Kohflote wrote:Dear Tom,

"TESTING STOCKFISH DEVELOPMENT 080114 = 480 GAMES.

Bench: 8502826 Timestamp: 1389220684"

- do you mean the version whereby the author is Henri Wiechers, Date: Jan 9 at abrok.eu/stockfish website?

Thank you & regards,
Kah Huat
Hello, Kah Huat!

I mean that the version I tested and reported is the one in abrok that you mentioned. But the author is all the Stockfish team.

By the way Mediafire is reporting a problem with the files containing the games. If it is not solved soon I will upload them again.

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

I have re-uploaded both files of my latest test to Mediafire. They seem to work well now.

http://www.mediafire.com/view/lbr5e21nr ... mes(2).pgn

http://www.mediafire.com/view/ggwln3xiv ... mes(2).pgn

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH IPMAN COMPILE 090114 IP = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 090114IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 23.0 - 17.0 +12/=22/-6 57.50%
Stockfish 090114IP 64 SSE4.2L - Komodo TCECr 64-bitx6 27.0 - 13.0 +17/=20/-3 67.50%
Stockfish 090114IP 64 SSE4.2L - Critter 1.6a 64-bitX6 26.5 - 13.5 +17/=19/-4 66.25%

Time Control= 2+2

Stockfish 090114IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 19.5 - 20.5 +8/=23/-9 48.75%
Stockfish 090114IP 64 SSE4.2L - Komodo TCECr 64-bitx6 20.5 - 19.5 +7/=27/-6 51.25%
Stockfish 090114IP 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 26.5 - 13.5 +16/=21/-3 66.25%

240 Games = http://www.mediafire.com/view/9026ded9q ... 0games.pgn
Score using 6 Cores= 143.0 – 97.0 = 59.58%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockfish 090114IP 64 SSE4.2L - Houdini 4 x64xCT0 20.5 - 19.5 +9/=23/-8 51.25%
Stockfish 090114IP 64 SSE4.2L - Komodo TCECr 64-bitx4 24.5 - 15.5 +13/=23/-4 61.25%
Stockfish 090114IP 64 SSE4.2L - Critter 1.6a 64-bitnob_4 29.0 - 11.0 +20/=18/-2 72.50%

Time Control= 2+2

Stockfish 090114IP 64 SSE4.2L - Houdini 4 x64xCT0 21.0 - 19.0 +10/=22/-8 52.50%
Stockfish 090114IP 64 SSE4.2L - Komodo TCECr 64-bitx4 23.0 - 17.0 +12/=22/-6 57.50%
Stockfish 090114IP 64 SSE4.2L - Critter 1.6a 64-bitnob_4 28.0 - 12.0 +19/=18/-3 70.00%

240 Games= http://www.mediafire.com/view/lm4mz290z ... 0games.pgn
Score using 4 Cores= 146.0 – 94.0 = 60.83%

Segmenting by Time Control:

Fixed TC = 150.5 – 89.5 = 62.71%
Incremental TC = 138.5 – 101.5 = 57.71%

Global Score= 289.0 – 191.0 = 60.21%

Against : Houdini 4.0 St. Ct0 (3233) = 52.50% ; Komodo TCECr (3178) = 59.37% ; Critter 1.6a (3093) = 68.75%

Average Estimated Elo Opponents = 3168
Estimated Elo Performance= 3239

After an excellent score at Fixed Time Control I thought in a possible new record. But the second part of my test, at Incremental TC, the performance was not so good.

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT 120114 = 480 GAMES

Bench: 8502826 Timestamp: 1389467446

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
No Large Pages
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 120114 64 SSE4.2x - Houdini 4 x64_st_X6_CT0 17.0 - 23.0 +4/=26/-10 42.50%
Stockfish 120114 64 SSE4.2x - Komodo TCECr 64-bitx6 25.0 - 15.0 +15/=20/-5 62.50%
Stockfish 120114 64 SSE4.2x - Critter 1.6a 64-bitX6_NOB 24.5 - 15.5 +12/=25/-3 61.25%

Time Control= 2+2

Stockfish 120114 64 SSE4.2x - Houdini 4 x64_st_X6_CT0 19.0 - 21.0 +7/=24/-9 47.50%
Stockfish 120114 64 SSE4.2x - Komodo TCECr 64-bitx6 21.5 - 18.5 +8/=27/-5 53.75%
Stockfish 120114 64 SSE4.2x - Critter 1.6a 64-bitX6_NOB 23.5 - 16.5 +13/=21/-6 58.75%

240 Games = http://www.mediafire.com/view/446my31hf ... 0games.pgn
Score using 6 Cores= 130.5 – 109.5 = 54.37%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
No Large Pages
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockfish 120114 64 SSE4.2x - Houdini 4 x64xCT0 20.0 - 20.0 +7/=26/-7 50.00%
Stockfish 120114 64 SSE4.2x - Komodo TCECr 64-bitx4 25.0 - 15.0 +16/=18/-6 62.50%
Stockfish 120114 64 SSE4.2x - Critter 1.6a 64-bitnob_4 27.5 - 12.5 +18/=19/-3 68.75%

Time Control= 2+2
Stockfish 120114 64 SSE4.2x - Houdini 4 x64xCT0 20.0 - 20.0 +11/=18/-11 50.00%
Stockfish 120114 64 SSE4.2x - Komodo TCECr 64-bitx4 23.0 - 17.0 +10/=26/-4 57.50%
Stockfish 120114 64 SSE4.2x - Critter 1.6a 64-bitnob_4 28.0 - 12.0 +20/=16/-4 70.00%

240 Games=
http://www.mediafire.com/view/bcl91v7ux ... 0games.pgn
Scores using 4 cores= 143.5 – 96.5 = 59.79%

Segmenting by Time Control:

Fixed TC = 139.0 – 101.0= 57.92%
Incremental TC = 135.0 – 105.0 = 56.25%

Global Score= 274.0 – 206.0 = 57.08%

Against : Houdini 4.0 St. Ct0 (3233) = 47.50% ; Komodo TCECr (3178) = 59.06% ; Critter 1.6a (3093) = 64.69%

Average Estimated Elo Opponents = 3168
Estimated Elo Performance= 3218

Not a very good performance for this Development Version. 27 EEP below the best scorer version (080114). Just an statistical error?. Who knows! :roll:

Regards,

Tom.
Top
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: Testing Stockfish 11-03-13. 480 Games.

Post by ouachita »

Tomcass wrote: Time Control= 4+0

Stockfish 080114 64 SSE4.2X - Houdini 4 x64_st_X6_CT0 22.0 - 18.0 +7/=30/-3 55.00%
Tom.

Code: Select all

SF080114-6 v. H4B-6, 1m+1s  0

                                      
1   Stockfish 080114 64 SSE4.2   +35  +28/=54/-18 55.00%   55.0/100
2   Houdini 4 Pro x64 B          -35  +18/=54/-28 45.00%   45.0/100
Similar 6-core result, different TC. No need to test, just rely on Tom's.
SIM, PhD, MBA, PE
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH IPMAN COMPILE 130114 IP = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 130114IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 22.0 - 18.0 +12/=20/-8 55.00%
Stockfish 130114IP 64 SSE4.2L - Komodo TCECr 64-bitx6 25.5 - 14.5 +15/=21/-4 63.75%
Stockfish 130114IP 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 27.5 - 12.5 +18/=19/-3 68.75%

Time Control= 2+2

Stockfish 130114IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 20.0 - 20.0 +7/=26/-7 50.00%
Stockfish 130114IP 64 SSE4.2L - Komodo TCECr 64-bitx6 18.5 - 21.5 +5/=27/-8 46.25%
Stockfish 130114IP 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 27.0 - 13.0 +16/=22/-2 67.50%


240 Games = http://www.mediafire.com/view/9f2aaf8p8 ... 0games.pgn
Score using 6 Cores= 140.5 – 99.5 = 58.54%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockfish 130114IP 64 SSE4.2L - Houdini 4 x64xCT0 21.0 - 19.0 +9/=24/-7 52.50%
Stockfish 130114IP 64 SSE4.2L - Komodo TCECr 64-bitx4 28.0 - 12.0 +19/=18/-3 70.00%
Stockfish 130114IP 64 SSE4.2L - Critter 1.6a 64-bitnob_4 29.0 - 11.0 +19/=20/-1 72.50%

Time Control= 2+2

Stockfish 130114IP 64 SSE4.2L - Houdini 4 x64xCT0 21.5 - 18.5 +11/=21/-8 53.75%
Stockfish 130114IP 64 SSE4.2L - Komodo TCECr 64-bitx4 24.0 - 16.0 +12/=24/-4 60.00%
Stockfish 130114IP 64 SSE4.2L - Critter 1.6a 64-bitnob_4 23.5 - 16.5 +11/=25/-4 58.75%
240 Games= http://www.mediafire.com/view/rxpi2bck2 ... 0games.pgn
Score using 4 Cores= 147.0 – 93.0 = 61.25%

Segmenting by Time Control:

Fixed TC = 153.0 – 87.0 = 63.75%
Incremental TC = 134.5 – 105.5 = 56.04%

Global Score= 287.5 – 192.5 = 59.89%
Against : Houdini 4.0 St. Ct0 (3233) =52.81% ; Komodo TCECr (3178) =60.00% ; Critter 1.6a (3093) = 66.87%

Average Estimated Elo Opponents = 3168
Estimated Elo Performance= 3237

Please note the big difference in the score between Fixed and Incremental Time Control.

Regards,

Tom.


... by the way, thanks for your kind comment, Bobby. :D
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH ROCKWOOD COMPILE 140114 = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockwood 140114 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 22.0 - 18.0 +10/=24/-6 55.00%
Stockwood 140114 64 SSE4.2L - Komodo TCECr 64-bitx6 21.5 - 18.5 +8/=27/-5 53.75%
Stockwood 140114 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 27.5 - 12.5 +19/=17/-4 68.75%

Time Control= 2+2

Stockwood 140114 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 22.5 - 17.5 +9/=27/-4 56.25%
Stockwood 140114 64 SSE4.2L - Komodo TCECr 64-bitx6 21.5 - 18.5 +8/=27/-5 53.75%
Stockwood 140114 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 24.5 - 15.5 +11/=27/-2 61.25%

240 Games = http://www.mediafire.com/view/shvw988rm ... 0games.pgn
Score using 6 Cores= 139.5 – 100.5 = 58.12%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockwood 140114 64 SSE4.2L - Houdini 4 x64xCT0 20.5 - 19.5 +8/=25/-7 51.25%
Stockwood 140114 64 SSE4.2L - Komodo TCECr 64-bitx4 25.5 - 14.5 +17/=17/-6 63.75%
Stockwood 140114 64 SSE4.2L - Critter 1.6a 64-bitnob_4 25.5 - 14.5 +16/=19/-5 63.75%

Time Control= 2+2

Stockwood 140114 64 SSE4.2L - Houdini 4 x64xCT0 23.0 - 17.0 +9/=28/-3 57.50%
Stockwood 140114 64 SSE4.2L - Komodo TCECr 64-bitx4 27.0 - 13.0 +17/=20/-3 67.50%
Stockwood 140114 64 SSE4.2L - Critter 1.6a 64-bitnob 27.5 - 12.5 +15/=25/-0 68.75%
240 Games=
http://www.mediafire.com/view/waya2ic7t ... 0games.pgn
Score using 4 Cores= 149.0 – 91.0 = 62.08%

Segmenting by Time Control:

Fixed TC = 142.5 – 97.5= 59.37%
Incremental TC = 146.0 – 94.9 = 60.83%

Global Score= 288.5 – 191.5 = 60.10

Against : Houdini 4.0 St. Ct0 (3233) = 55.00% ; Komodo TCECr (3178) = 59.69% ; Critter 1.6a (3093) = 65.62%

Average Estimated Elo Opponents = 3168
Estimated Elo Performance= 3.239

This version os SF Rockwood has performed much better that the previous one I tested 10 days ago, which got an EEP of 3211.

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT 180114 = 480 GAMES

Bench: 7804908 Timestamp: 1390063957

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
No Large Pages
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 180114 64 SSE4.2x - Houdini 4 x64_st_X6_CT0 20.5 - 19.5 +10/=21/-9 51.25%
Stockfish 180114 64 SSE4.2x - Komodo TCECr 64-bitx6 22.0 - 18.0 +10/=24/-6 55.00%
Stockfish 180114 64 SSE4.2x - Critter 1.6a 64-bitX6_NOB 26.0 - 14.0 +16/=20/-4 65.00%

Time Control= 2+2

Stockfish 180114 64 SSE4.2x - Houdini 4 x64_st_X6_CT0 19.5 - 20.5 +5/=29/-6 48.75%
Stockfish 180114 64 SSE4.2x - Komodo TCECr 64-bitx6 23.5 - 16.5 +13/=21/-6 58.75%
Stockfish 180114 64 SSE4.2x - Critter 1.6a 64-bitX6_NOB 24.5 - 15.5 +13/=23/-4 61.25%


240 Games = http://www.mediafire.com/view/888c11584 ... 0games.pgn
Score using 6 Cores= 136.0 – 104.0 = 56.67%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
No Large Pages
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockfish 180114 64 SSE4.2x - Houdini 4 x64xCT0 22.5 - 17.5 +9/=27/-4 56.25%
Stockfish 180114 64 SSE4.2x - Komodo TCECr 64-bitx4 24.0 - 16.0 +13/=22/-5 60.00%
Stockfish 180114 64 SSE4.2x - Critter 1.6a 64-bitnob 29.0 - 11.0 +20/=18/-2 72.50%

Time Control= 2+2

Stockfish 180114 64 SSE4.2x - Houdini 4 x64xCT0 20.5 - 19.5 +6/=29/-5 51.25%
Stockfish 180114 64 SSE4.2x - Komodo TCECr 64-bitx4 23.0 - 17.0 +13/=20/-7 57.50%
Stockfish 180114 64 SSE4.2x - Critter 1.6a 64-bitnob_4 27.5 - 12.5 +16/=23/-1 68.75%

240 Games= http://www.mediafire.com/view/4zl7ckprs ... 0games.pgn
Scores using 4 cores= 146.5 – 93.5 = 61.04%

Segmenting by Time Control:

Fixed TC = 144.0 – 96.0= 60.00%
Incremental TC = 138.5 – 101.5 = 57.71%

Global Score= 282.5 – 197.5= 58.85%

Against : Houdini 4.0 St. Ct0 (3233) = 51.87% ; Komodo TCECr (3178) = 57.81% ; Critter 1.6a (3093) = 66.87%

Average Estimated Elo Opponents = 3168
Estimated Elo Performance= 3230

Regards,

Tom.
Top

Return to “Computer Chess Club: Tournaments and Matches”