Testing Stockfish 11-03-13. 480 Games.

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT 050314 = 480 GAMES

Bench: 8430785 Timestamp: 1394711583

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c Sedat
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 130314 64 SSE4.2x - Houdini 4 x64_st_X6_CT0 21.5 - 18.5 +7/=29/-4 53.75%
Stockfish 130314 64 SSE4.2x - Komodo TCECr 64-bitx6 24.5 - 15.5 +13/=23/-4 61.25%
Stockfish 130314 64 SSE4.2x - Gull 2.8 beta x64X6 26.5 - 13.5 +17/=19/-4 66.25%

Time Control= 2+2

201402Stockfish_130314_2+2 2014

Stockfish 130314 64 SSE4.2x - Houdini 4 x64_st_X6_CT0 22.5 - 17.5 +12/=21/-7 56.25%
Stockfish 130314 64 SSE4.2x - Komodo TCECr 64-bitx6 27.0 - 13.0 +19/=16/-5 67.50%
Stockfish 130314 64 SSE4.2x - Gull 2.8 beta x64X6 23.5 - 16.5 +13/=21/-6 58.75%

Score using 6 cores: 145.5 – 94.5 = 60.62%
240 Games: http://www.mediafire.com/view/r6x47s18f ... 0games.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c Sedat
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

Stockfish 130314 64 SSE4.2x - Houdini 4 x64_StA_Ct0_X4 20.5 - 19.5 +9/=23/-8 51.25%
Stockfish 130314 64 SSE4.2x - Komodo TCECr 64-bitx4 20.0 - 20.0 +9/=22/-9 50.00%
Stockfish 130314 64 SSE4.2x - Gull 2.8 beta x64 23.0 - 17.0 +14/=18/-8 57.50%

Time Control: 2+2

Stockfish 130314 64 SSE4.2x - Houdini 4 x64_StA_Ct0_X4 20.0 - 20.0 +7/=26/-7 50.00%
Stockfish 130314 64 SSE4.2x - Komodo TCECr 64-bitx4 24.0 - 16.0 +14/=20/-6 60.00%
Stockfish 130314 64 SSE4.2x - Gull 2.8 beta x64 17.5 - 22.5 +3/=29/-8 43.75%

Score using 4 Cores= 125.0 – 115.0 = 52.08%
240 Games:
http://www.mediafire.com/view/jl39slv9h ... amesx4.pgn

Segmenting by Time Control:

Fixed TC = 136.0 – 104.0 = 56.67%
Incremental TC = 134.5 -105.5 = 56.04%

Global Score= 270.5 – 209.5 = 56.35%

Against : Houdini 4.0 St. Ct0 (3227) = 52.81% ; Komodo TCECr (3181) = 59.69%, Gull2.8Beta (3141) = 56.56%

Average Estimated Elo Opponents = 3183
Estimated Elo Performance= 3227


Error bars: +/- 23 EEP

Not a good score this time for SF. By the way, Gull 2.8a seems to be surprisingly strong in my old 4 cores computer.

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH IPMAN COMPILE 230214 = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c – Sedat – Limit 5 moves
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 28.66
Knodes per second: 13759

Time Control= 4+0

SF 160314IPx 64 SSE4.2LP - Houdini 4 x64_st_X6_CT0 19.5 - 20.5 +6/=27/-7 48.75%
SF 160314IPx 64 SSE4.2LP - Komodo TCECr 64-bitx6_NOB 25.5 - 14.5 +15/=21/-4 63.75%
SF 160314IPx 64 SSE4.2LP - Gull 2.8 beta x64X6 25.5 - 14.5 +14/=23/-3 63.75%

Time Control= 2+2

201403Stockfish_160314IP_LP_2+2 2014

SF 160314IPx 64 SSE4.2LP - Houdini 4 x64_st_X6_CT0 24.0 - 16.0 +14/=20/-6 60.00%
SF 160314IPx 64 SSE4.2LP - Komodo TCECr 64-bitx6_NOB 22.5 - 17.5 +9/=27/-4 56.25%
SF 160314IPx 64 SSE4.2LP - Gull 2.8 beta x64X6 27.5 - 12.5 +16/=23/-1 68.75%

240 Games = http://www.mediafire.com/view/esalln6b9 ... 0games.pgn
Score using 6 Cores= 144.5 – 95.5 = 60.21%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c Sedat –limit 5 moves -
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

SF 160314IPx 64 SSE4.2LP - Houdini 4 x64_StA_Ct0_X4 21.5 - 18.5 +8/=27/-5 53.75%
SF 160314IPx 64 SSE4.2LP - Komodo TCECr 64-bitx4_NOB 24.5 - 15.5 +12/=25/-3 61.25%
SF 160314IPx 64 SSE4.2LP - Gull 2.8 beta x64 24.5 - 15.5 +15/=19/-6 61.25%

Time Control= 2+2

SF 160314IPx 64 SSE4.2LP - Houdini 4 x64_StA_Ct0_X4 18.5 - 21.5 +8/=21/-11 46.25%
SF 160314IPx 64 SSE4.2LP - Komodo TCECr 64-bitx4_NOB 23.0 - 17.0 +14/=18/-8 57.50%
SF 160314IPx 64 SSE4.2LP - Gull 2.8 beta x64 24.5 - 15.5 +13/=23/-4 61.25%

240 Games= http://www.mediafire.com/view/6sp24apzk ... amesx4.pgn
Score using 4 Cores= 136.5 -103.5 = 56.87%

Segmenting by Time Control:

Fixed TC = 141.0 – 99.0 = 58.75%
Incremental TC = 140.0 – 100.0 = 58.33%

Global Score= 281.0 – 199.0 = 58.54%
Against : Houdini 4.0 St. Ct0 (3227)= 52.19% ; Komodo TCECr (3181) = 59.69% ; Gull2.8Beta (3141) = 63.75%

Average Estimated Elo Opponents = 3183
Estimated Elo Performance= 3243


Error bars= +/- 23 EEP

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

Wooops! :oops:

The headline of my previous test is wrong. The right one is:

TESTING STOCKFISH IPMAN COMPILE 160314 = 480 GAMES

Sorry!

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH ROCKWOOD COMPILE 220314 = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c Sedat –Limit 5 moves-
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 30.42
Knodes per second: 14602

Time Control= 4+0

StockfishRW 140322x - Houdini 4 x64_st_X6_CT0 22.0 - 18.0 +10/=24/-6 55.00%
StockfishRW 140322x - Komodo TCECr 64-bitx6_NOB 25.0 - 15.0 +16/=18/-6 62.50%
StockfishRW 140322x - Gull 2.8 beta x64X6 22.0 - 18.0 +7/=30/-3 55.00%

Time Control= 2+2

StockfishRW 140322x - Houdini 4 x64_st_X6_CT0 21.5 - 18.5 +12/=19/-9 53.75%
StockfishRW 140322x - Komodo TCECr 64-bitx6_NOB 25.5 - 14.5 +13/=25/-2 63.75%
StockfishRW 140322x - Gull 2.8 beta x64X6 27.0 - 13.0 +17/=20/-3 67.50%

240 Games = http://www.mediafire.com/view/y7lggte5t ... 0Games.pgn
Score using 6 Cores= 143.0 – 97.0 = 59.58%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c Sedat – Limit 5 moves -
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

StockfishRW 140322 SSE4.2x - Houdini 4 x64_StA_Ct0_X4 17.5 - 22.5 +7/=21/-12 43.75%
StockfishRW 140322 SSE4.2x - Komodo TCECr 64-bitx4 22.5 - 17.5 +10/=25/-5 56.25%
StockfishRW 140322 SSE4.2x - Gull 2.8 beta x64 24.5 - 15.5 +13/=23/-4 61.25%

Time Control= 2+2

StockfishRW 140322 SSE4.2x - Houdini 4 x64_StA_Ct0_X4 23.0 - 17.0 +9/=28/-3 57.50%
StockfishRW 140322 SSE4.2x - Komodo TCECr 64-bitx4 24.0 - 16.0 +15/=18/-7 60.00%
StockfishRW 140322 SSE4.2x - Gull 2.8 beta x64 29.0 - 11.0 +20/=18/-2 72.50%
240 Games=
http://www.mediafire.com/view/s3akj01a3 ... 4cores.pgn

Score using 4 Cores= 140.5 -99.5 = 58.54%

Segmenting by Time Control:

Fixed TC = 133.5 – 106.5 = 55.62%
Incremental TC = 150.0 – 90.0 = 62.50%

Global Score= 283.5 – 196.5 = 59.06%

Against : Houdini 4.0 St. Ct0 (3227) 52.50% ; Komodo TCECr (3181) 60.62% ; Gull 2.8Beta (3141)= 64.06%

Average Estimated Elo Opponents = 3183
Estimated Elo Performance= 3246


Regards,

Tom.
duncan
Posts: 12038
Joined: Mon Jul 07, 2008 10:50 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by duncan »

Tomcass wrote:TESTING STOCKFISH ROCKWOOD COMPILE 220314 = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c Sedat –Limit 5 moves-
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 30.42
Knodes per second: 14602

Time Control= 4+0

StockfishRW 140322x - Houdini 4 x64_st_X6_CT0 22.0 - 18.0 +10/=24/-6 55.00%
StockfishRW 140322x - Komodo TCECr 64-bitx6_NOB 25.0 - 15.0 +16/=18/-6 62.50%
StockfishRW 140322x - Gull 2.8 beta x64X6 22.0 - 18.0 +7/=30/-3 55.00%

Time Control= 2+2

StockfishRW 140322x - Houdini 4 x64_st_X6_CT0 21.5 - 18.5 +12/=19/-9 53.75%
StockfishRW 140322x - Komodo TCECr 64-bitx6_NOB 25.5 - 14.5 +13/=25/-2 63.75%
StockfishRW 140322x - Gull 2.8 beta x64X6 27.0 - 13.0 +17/=20/-3 67.50%

240 Games = http://www.mediafire.com/view/y7lggte5t ... 0Games.pgn
Score using 6 Cores= 143.0 – 97.0 = 59.58%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c Sedat – Limit 5 moves -
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

StockfishRW 140322 SSE4.2x - Houdini 4 x64_StA_Ct0_X4 17.5 - 22.5 +7/=21/-12 43.75%
StockfishRW 140322 SSE4.2x - Komodo TCECr 64-bitx4 22.5 - 17.5 +10/=25/-5 56.25%
StockfishRW 140322 SSE4.2x - Gull 2.8 beta x64 24.5 - 15.5 +13/=23/-4 61.25%

Time Control= 2+2

StockfishRW 140322 SSE4.2x - Houdini 4 x64_StA_Ct0_X4 23.0 - 17.0 +9/=28/-3 57.50%
StockfishRW 140322 SSE4.2x - Komodo TCECr 64-bitx4 24.0 - 16.0 +15/=18/-7 60.00%
StockfishRW 140322 SSE4.2x - Gull 2.8 beta x64 29.0 - 11.0 +20/=18/-2 72.50%
240 Games=
http://www.mediafire.com/view/s3akj01a3 ... 4cores.pgn

Score using 4 Cores= 140.5 -99.5 = 58.54%

Segmenting by Time Control:

Fixed TC = 133.5 – 106.5 = 55.62%
Incremental TC = 150.0 – 90.0 = 62.50%

Global Score= 283.5 – 196.5 = 59.06%

Against : Houdini 4.0 St. Ct0 (3227) 52.50% ; Komodo TCECr (3181) 60.62% ; Gull 2.8Beta (3141)= 64.06%

Average Estimated Elo Opponents = 3183
Estimated Elo Performance= 3246


Regards,

Tom.
so no improvement since 290114. quite normal for most chess engines, but for stockfish with its 16/month average? has this happened before almost 2 months without elo improvement
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

Hi Duncan.

The pace of improvement has not been so strong in the latest weeks ... but I feel that something nice will happen soon. :-)

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH IPMAN COMPILE 240314 = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c – Sedat – Limit 5 moves
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 28.66
Knodes per second: 13759

Time Control = 4+0

SF 240314IPx2 64 SSE4.2LP - Houdini 4 x64_st_X6_CT0 20.5 - 19.5 +7/=27/-6 51.25%
SF 240314IPx2 64 SSE4.2LP - Komodo TCECr 64-bitx6 27.0 - 13.0 +18/=18/-4 67.50%
SF 240314IPx2 64 SSE4.2LP - Gull 2.8 beta x64X6 28.5 - 11.5 +21/=15/-4 71.25%

Time Control= 2+2

SF 240314IPx2 64 SSE4.2LP - Houdini 4 x64_st_X6_CT0 24.5 - 15.5 +16/=17/-7 61.25%
SF 240314IPx2 64 SSE4.2LP - Komodo TCECr 64-bitx6 22.0 - 18.0 +11/=22/-7 55.00%
SF 240314IPx2 64 SSE4.2LP - Gull 2.8 beta x64X6 26.0 - 14.0 +18/=16/-6 65.00%


240 Games = http://www.mediafire.com/view/ha7fvlugu ... 0Games.pgn
Score using 6 Cores= 148.5 – 91.5 = 61.87%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c Sedat –limit 5 moves -
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

SF 240314IPx2 64 SSE4.2LP - Houdini 4 x64_StA_Ct0_X4 20.0 - 20.0 +11/=18/-11 50.00%
SF 240314IPx2 64 SSE4.2LP - Komodo TCECr 64-bitx4 24.0 - 16.0 +12/=24/-4 60.00%
SF 240314IPx2 64 SSE4.2LP - Gull 2.8 beta x64 27.5 - 12.5 +20/=15/-5 68.75%

Time Control= 2+2

SF 240314IPx2 64 SSE4.2LP - Houdini 4 x64_StA_Ct0_X4 17.0 - 23.0 +6/=22/-12 42.50%
SF 240314IPx2 64 SSE4.2LP - Komodo TCECr 64-bitx4 23.0 - 17.0 +14/=18/-8 57.50%
SF 240314IPx2 64 SSE4.2LP - Gull 2.8 beta x64 25.5 - 14.5 +15/=21/-4 63.75%
240 Games=
http://www.mediafire.com/view/z5894h0w4 ... 4cores.pgn

Score using 4 Cores= 137.0 – 103.0 = 57.08%

Segmenting by Time Control:

Fixed TC = 147.5 -92.5 = 61.46%
Incremental TC = 138.0 – 102.0 = 57.50%

Global Score= 285.5 – 194.5 = 59.48%
Against : Houdini 4.0 St. Ct0 (3227)= 51.25% ; Komodo TCECr (3181) = 60.00% ; Gull2.8Beta (3141) = 67.19%

Average Estimated Elo Opponents = 3183
Estimated Elo Performance= 3249


Error bars= +/- 23 EEP

Let's test now the latest Development SF.

Regards,

Tom.
User avatar
Ozymandias
Posts: 1537
Joined: Sun Oct 25, 2009 2:30 am

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Ozymandias »

Just goes to show that "nunca llueve a gusto de todos" (one man's meat is another man's poison). I've been crossing my fingers these past two months, hoping it would finally stall it's improvement.
duncan
Posts: 12038
Joined: Mon Jul 07, 2008 10:50 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by duncan »

Ozymandias wrote:Just goes to show that "nunca llueve a gusto de todos" (one man's meat is another man's poison). I've been crossing my fingers these past two months, hoping it would finally stall it's improvement.
why. who do you back ?
User avatar
Ozymandias
Posts: 1537
Joined: Sun Oct 25, 2009 2:30 am

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Ozymandias »

No one, I just want chess programmers to find new hobbies/enterprises. Stockfish being the one with the more alarming improvement rate, should be first to stop. :wink: