Testing Stockfish 11-03-13. 480 Games.

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT 290114 = 480 GAMES

Bench: 6875743 Timestamp: 1391014933

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 290114 64 SSE4.2x - Houdini 4 x64_st_X6_CT0 20.5 - 19.5 +10/=21/-9 51.25%
Stockfish 290114 64 SSE4.2x - Komodo TCECr 64-bitx6 21.0 - 19.0 +12/=18/-10 52.50%
Stockfish 290114 64 SSE4.2x - Critter 1.6a 64-bitX6_NOB 27.5 - 12.5 +17/=21/-2 68.75%

Time Control= 2+2

Stockfish 290114 64 SSE4.2x - Houdini 4 x64_st_X6_CT0 22.5 - 17.5 +10/=25/-5 56.25%
Stockfish 290114 64 SSE4.2x - Komodo TCECr 64-bitx6 25.0 - 15.0 +17/=16/-7 62.50%
Stockfish 290114 64 SSE4.2x - Critter 1.6a 64-bitX6_NOB 26.0 - 14.0 +14/=24/-2 65.00%

Score using 6 cores: 142,5 -97,5 = 59.37%
240 Games: http://www.mediafire.com/view/a14kotk9k ... 0games.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

Stockfish 290114 64 SSE4.2x - Houdini 4 x64xCT0 22.5 - 17.5 +11/=23/-6 56.25%
Stockfish 290114 64 SSE4.2x - Komodo TCECr 64-bitx4 23.0 - 17.0 +13/=20/-7 57.50%
Stockfish 290114 64 SSE4.2x - Critter 1.6a 64-bitnob4 29.0 - 11.0 +21/=16/-3 72.50%

Time Control: 2+2

Stockfish 290114 64 SSE4.2x - Houdini 4 x64xCT0 22.0 - 18.0 +9/=26/-5 55.00%
Stockfish 290114 64 SSE4.2x - Komodo TCECr 64-bitx4 27.5 - 12.5 +18/=19/-3 68.75%
Stockfish 290114 64 SSE4.2x - Critter 1.6a 64-bitnob_4 29.5 - 10.5 +19/=21/-0 73.75%
Score using 4 Cores= 153.5 – 86.5= 63.96% (!!!)
240 Games:
http://www.mediafire.com/view/xzyns75q3 ... amesx4.pgn

Segmenting by Time Control:

Fixed TC = 143.5 – 96.5 = 59.79%
Incremental TC = 152.5 – 87.5 = 63.54%

Global Score= 296.0 – 184.0 = 61.67%

Against : Houdini 4.0 St. Ct0 (3227) = 54.69%; Komodo TCECr (3181) = 60.31%; Critter 1.6a (3104) = 70.00%

Average Estimated Elo Opponents = 3171
Estimated Elo Performance= 3253


By a significative margin this is a new best score in my tests so far

12 Elo points above the previous leader (Stockwood 210114 SSE4.2L = 3241) and 26 Estimated Elo Points above the best Houdini 4.0 (Standard Contempt 0 = 3227). Brilliant!.

Regards,

Tom.
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: Testing Stockfish 11-03-13. 480 Games.

Post by ouachita »

4-Core Tournament Conditions

W7 Ultimate
BIOS Hyperthreading: disabled
GUI: DF13
Tournament: Round Robin
Time Control: 1+1
Processors: Dual E5-2687W CPU (16 physical cores)
Match Engine Threads: 4 physical cores/threads each engine
CPU Usage Avg.: ~30%
Memory: ~37%
Hashtable size: 8192 MB
PB: disabled
Opening Database: 100 Pivotal Bobby Johnson ICCF Game Positions/reversed colors.
Book Learning: disabled
Engine Book: disabled
Engine EGTB: Yes, both engines.
Non-default Engine Parameters: SF, No; H, Yes.

H4ProB v. 290114 64 SSE4.2, 1m+1s

Code: Select all

                                      
1   Houdini 4 Pro x64B           +14  +52/=104/-44 52.00%  104.0/200
2   Stockfish 290114 64 SSE4.2   -14  +44/=104/-52 48.00%   96.0/200
Versus Tom's, this match used shorter TC, both used TB, and tweaked H engine parameters.
SIM, PhD, MBA, PE
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

Thanks for this nice match, Bobby. In fact Houdini 4.0 is still a very hard opponent at short time control.

In the settings you used for this match there is one which IMHO is not optimal : The Hashtable size of 8192 Mb is too big for a time control of 1+1. I would suggest you to set no more than 1024 Mb hashtable for short Time Control tests. In fact I use 256 and 512 for my 4 cores and 6 cores computers, with longer Time Controls.

Regards,

Tom.
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: Testing Stockfish 11-03-13. 480 Games.

Post by ouachita »

Tomcass wrote:The Hashtable size of 8192 Mb is too big for a time control of 1+1.Regards,Tom.
When I use 16 cores I always use 8192 because it's fastest. However, I used 4 cores and should have used 512mb. thx
SIM, PhD, MBA, PE
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH ROCKWOOD COMPILE 290114 = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

StockfishRW290114 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 23.5 - 16.5 +13/=21/-6 58.75%
StockfishRW290114 64 SSE4.2L - Komodo TCECr 64-bitx6 24.5 - 15.5 +14/=21/-5 61.25%
StockfishRW290114 64 SSE4.2L - Critter 1.6a 64-bitX6 25.0 - 15.0 +16/=18/-6 62.50%

Time Control= 2+2

StockfishRW290114 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 23.0 - 17.0 +13/=20/-7 57.50%
StockfishRW290114 64 SSE4.2L - Komodo TCECr 64-bitx6 23.5 - 16.5 +13/=21/-6 58.75%
StockfishRW290114 64 SSE4.2L - Critter 1.6a 64-bitX6 27.0 - 13.0 +17/=20/-3 67.50%

240 Games = http://www.mediafire.com/view/sblm3dz6m ... 0games.pgn
Score using 6 Cores= 141.5 – 98.5 = 58.96%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

StockfishRW290114 64 SSE4.2X - Houdini 4 x64xCT0 23.0 - 17.0 +13/=20/-7 57.50%
StockfishRW290114 64 SSE4.2X - Komodo TCECr 64-bit 23.0 - 17.0 +13/=20/-7 57.50%
StockfishRW290114 64 SSE4.2X - Critter 1.6a 64-bitnob 23.0 - 17.0 +11/=24/-5 57.50%

Time Control= 2+2
StockfishRW290114 64 SSE4.2X - Houdini 4 x64xCT0 23.0 - 17.0 +8/=30/-2 57.50%
StockfishRW290114 64 SSE4.2X - Komodo TCECr 64-bit 25.5 - 14.5 +15/=21/-4 63.75%
StockfishRW290114 64 SSE4.2X - Critter 1.6a 64-bitnob 28.0 - 12.0 +18/=20/-2 70.00%

240 Games=
http://www.mediafire.com/view/6w6qq232t ... amesx4.pgn

Score using 4 Cores= 145.5 – 94.5 = 60.62%

Segmenting by Time Control:

Fixed TC = 142.0 – 98.0 = 59.17%
Incremental TC = 150.0 – 90.0 = 62.50%

Global Score= 292.0 – 188.0 = 60.83%

Against : Houdini 4.0 St. Ct0 (3233) = 57.81% ; Komodo TCECr (3178) 60.31% = ; Critter 1.6a (3093) = 64.37%

Average Estimated Elo Opponents = 3171
Estimated Elo Performance= 3247


This is the second best score ever in my tests. I think that this test confirms that a unusualy big step ahead has happened with this SF compile. Or perhaps there is some statistical noise?. Who knows!. :wink:
Let’s test Ipman’s compile for this magnificient 290114 Stockfish.

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH IPMAN COMPILE 290114 IP = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 290114IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 25.0 - 15.0 +14/=22/-4 62.50%
Stockfish 290114IP 64 SSE4.2L - Komodo TCECr 64-bitx6 23.0 - 17.0 +11/=24/-5 57.50%
Stockfish 290114IP 64 SSE4.2L - Critter 1.6a 64-bitX6_ 28.5 - 11.5 +18/=21/-1 71.25%

Time Control= 2+2

Stockfish 290114IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 25.0 - 15.0 +12/=26/-2 62.50%
Stockfish 290114IP 64 SSE4.2L - Komodo TCECr 64-bitx6 27.0 - 13.0 +19/=16/-5 67.50%
Stockfish 290114IP 64 SSE4.2L - Critter 1.6a 64-bitX6 28.5 - 11.5 +19/=19/-2 71.25%


240 Games =
http://www.mediafire.com/view/zngztbstt ... 0games.pgn

Score using 6 Cores= 157.0 – 83.0 = 65.42%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockfish 290114IP 64 SSE4.2L - Houdini 4 x64xCT0 22.5 - 17.5 +9/=27/-4 56.25%
Stockfish 290114IP 64 SSE4.2L - Komodo TCECr 64-bitx4 26.0 - 14.0 +16/=20/-4 65.00%
Stockfish 290114IP 64 SSE4.2L - Critter 1.6a 64-bitnob 24.5 - 15.5 +12/=25/-3 61.25%



Time Control= 2+2

Stockfish 290114IP 64 SSE4.2L - Houdini 4 x64xCT0 22.0 - 18.0 +9/=26/-5 55.00%
Stockfish 290114IP 64 SSE4.2L - Komodo TCECr 64-bit 25.0 - 15.0 +14/=22/-4 62.50%
Stockfish 290114IP 64 SSE4.2L - Critter 1.6a 64-bitnob 25.5 - 14.5 +14/=23/-3 63.75%



240 Games=
http://www.mediafire.com/view/tcw1d56mt ... amesx4.pgn

Score using 4 Cores= 145.5 – 94.5 = 60.62%

Segmenting by Time Control:

Fixed TC = 149.5 – 90.5 = 62.29%
Incremental TC = 153.0 – 87.0 = 63.75%

Global Score= 302.5 – 177.5 = 63.02%

Against : Houdini 4.0 St. Ct0 (3227) = 59.06% ; Komodo TCECr (3181) = 63.12% ; Critter 1.6a (3104) = 66.87%

Average Estimated Elo Opponents = 3171
Estimated Elo Performance= 3261


This is a clear new best score so far in my tests.

After my latest three tests I have no doubt that Stockfish has made a big step ahead with this extremely strong 290114 development engine.

Regards,

Tom.
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by mwyoung »

Tomcass wrote:TESTING STOCKFISH IPMAN COMPILE 290114 IP = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 290114IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 25.0 - 15.0 +14/=22/-4 62.50%
Stockfish 290114IP 64 SSE4.2L - Komodo TCECr 64-bitx6 23.0 - 17.0 +11/=24/-5 57.50%
Stockfish 290114IP 64 SSE4.2L - Critter 1.6a 64-bitX6_ 28.5 - 11.5 +18/=21/-1 71.25%

Time Control= 2+2

Stockfish 290114IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 25.0 - 15.0 +12/=26/-2 62.50%
Stockfish 290114IP 64 SSE4.2L - Komodo TCECr 64-bitx6 27.0 - 13.0 +19/=16/-5 67.50%
Stockfish 290114IP 64 SSE4.2L - Critter 1.6a 64-bitX6 28.5 - 11.5 +19/=19/-2 71.25%


240 Games =
http://www.mediafire.com/view/zngztbstt ... 0games.pgn

Score using 6 Cores= 157.0 – 83.0 = 65.42%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockfish 290114IP 64 SSE4.2L - Houdini 4 x64xCT0 22.5 - 17.5 +9/=27/-4 56.25%
Stockfish 290114IP 64 SSE4.2L - Komodo TCECr 64-bitx4 26.0 - 14.0 +16/=20/-4 65.00%
Stockfish 290114IP 64 SSE4.2L - Critter 1.6a 64-bitnob 24.5 - 15.5 +12/=25/-3 61.25%



Time Control= 2+2

Stockfish 290114IP 64 SSE4.2L - Houdini 4 x64xCT0 22.0 - 18.0 +9/=26/-5 55.00%
Stockfish 290114IP 64 SSE4.2L - Komodo TCECr 64-bit 25.0 - 15.0 +14/=22/-4 62.50%
Stockfish 290114IP 64 SSE4.2L - Critter 1.6a 64-bitnob 25.5 - 14.5 +14/=23/-3 63.75%



240 Games=
http://www.mediafire.com/view/tcw1d56mt ... amesx4.pgn

Score using 4 Cores= 145.5 – 94.5 = 60.62%

Segmenting by Time Control:

Fixed TC = 149.5 – 90.5 = 62.29%
Incremental TC = 153.0 – 87.0 = 63.75%

Global Score= 302.5 – 177.5 = 63.02%

Against : Houdini 4.0 St. Ct0 (3227) = 59.06% ; Komodo TCECr (3181) = 63.12% ; Critter 1.6a (3104) = 66.87%

Average Estimated Elo Opponents = 3171
Estimated Elo Performance= 3261


This is a clear new best score so far in my tests.

After my latest three tests I have no doubt that Stockfish has made a big step ahead with this extremely strong 290114 development engine.

Regards,

Tom.
My testing shows the same with this version of Stockfish. That is why I wanted to test this version at long time controls.

Thanks you for the results.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

Hi Mark, thanks for your comments and for your tests at long time controls. I follow them very carefully.

Kind regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT 030214 = 480 GAMES

Bench: 6875743 Timestamp: 1391459834

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0

Stockfish 030214 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 21.0 - 19.0 +8/=26/-6 52.50%
Stockfish 030214 64 SSE4.2L - Komodo TCECr 64-bitx6 23.0 - 17.0 +13/=20/-7 57.50%
Stockfish 030214 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 26.0 - 14.0 +18/=16/-6 65.00%

Time Control= 2+2

Stockfish 030214 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 18.0 - 22.0 +8/=20/-12 45.00%
Stockfish 030214 64 SSE4.2L - Komodo TCECr 64-bitx6 22.5 - 17.5 +10/=25/-5 56.25%
Stockfish 030214 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 28.0 - 12.0 +16/=24/-0 70.00%

Score using 6 cores: 138.5 – 101.5 = 57.71%
240 Games: http://www.mediafire.com/view/99bmjh418 ... 0games.pgn


i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control: 4+0

Stockfish 030214 64 SSE4.2x - Houdini 4 x64xCT0 16.5 - 23.5 +6/=21/-13 41.25%
Stockfish 030214 64 SSE4.2x - Komodo TCECr 64-bitx4 22.0 - 18.0 +12/=20/-8 55.00%
Stockfish 030214 64 SSE4.2x - Critter 1.6a 64-bitnob 27.0 - 13.0 +19/=16/-5 67.50%

Time Control: 2+2
Stockfish 030214 64 SSE4.2x - Houdini 4 x64xCT0 22.5 - 17.5 +13/=19/-8 56.25%
Stockfish 030214 64 SSE4.2x - Komodo TCECr 64-bitx4 24.5 - 15.5 +13/=23/-4 61.25%
Stockfish 030214 64 SSE4.2x - Critter 1.6a 64-bitnob_4 28.0 - 12.0 +20/=16/-4 70.00%
Score using 4 Cores= 140.5 – 99.5= 58.54%
240 Games:
http://www.mediafire.com/view/0fzzajzxw ... amesx4.pgn

Segmenting by Time Control:

Fixed TC = 135.5 – 104.5 = 56.46%
Incremental TC = 143.5 – 96.5 = 59.79%

Global Score= 279.0 – 201.0 = 58.12%

Against : Houdini 4.0 St. Ct0 (3227) = 48.75 %; Komodo TCECr (3181) = 57.50 %; Critter 1.6a (3104) = 68.12%

Average Estimated Elo Opponents = 3171
Estimated Elo Performance= 3228


Not a brilliant performance this time. SF 030214 has performed around 25 Elo points below the best Development Stockfish (290114 = 3253).

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

Re-reading my post I realize that my comment has not been fair with Stockfish 030214. I said: The score has not been so brilliant. I should have added: ... just at the best Houdini 4.0 level!!. :wink:

Regards,

Tom.