Testing Stockfish 11-03-13. 480 Games.

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: no more tests?

Post by Tomcass »

Sorry again. I think there is a problem with Mediafire...

This will work , hopefully. :)

http://www.mediafire.com/view/4790aoxon ... 0Games.pgn

Tom
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: no more tests?

Post by Tomcass »

TESTING STOCKFISH 101213 = 480 GAMES.

Timestamp: 1386655506 Bench: 7767864

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Stockfish 101213 64 SSE4.2X6 - Houdini 4 x64_st_X6CT0 19.0 - 21.0 +10/=18/-12 47.50%
Stockfish 101213 64 SSE4.2X6 - Komodo 6 64-bitNOBx6 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish 101213 64 SSE4.2X6 - Critter 1.6a 64-bitX6 25.0 - 15.0 +14/=22/-4 62.50%

Time Control= 2+2

201311Stockfish101213_2+2 2013

Stockfish 101213 64 SSE4.2X6 - Houdini 4 x64_st_X6_CT0 20.5 - 19.5 +10/=21/-9 51.25%
Stockfish 101213 64 SSE4.2X6 - Critter 1.6a 64-bitX 30.5 - 9.5 +22/=17/-1 76.25%
Stockfish 101213 64 SSE4.2X6 - Komodo 6 64-bitNOBx6 23.0 - 17.0 +12/=22/-6 57.50%

Score using 6 cores= 139.5 – 100.5 = 58.12%
240 Games= http://www.mediafire.com/view/r8x5bte0i ... 0games.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

Stockfish 101213 64 SSE4.2X4 - Houdini 4 x64xCT0 17.0 - 23.0 +6/=22/-12 42.50%
Stockfish 101213 64 SSE4.2X4 - Komodo 6 64-bitx4 23.0 - 17.0 +15/=16/-9 57.50%
Stockfish 101213 64 SSE4.2X4 - Critter 1.6a 64-bitnob 29.5 - 10.5 +19/=21/-0 73.75%

Time Control_ 2+2

Stockfish 101213 64 SSE4.2X4 - Houdini 4 x64xCT0 17.0 - 23.0 +3/=28/-9 42.50%
Stockfish 101213 64 SSE4.2X4 - Critter 1.6a 64-bitnob 27.0 - 13.0 +18/=18/-4 67.50%
Stockfish 101213 64 SSE4.2X4 - Komodo 6 64-bitx4_NOB 22.0 - 18.0 +11/=22/-7 55.00%
Score using 4 cores= 135.5 – 104.5 = 56.46%
http://www.mediafire.com/view/tqgj96zro ... 0games.pgn

Fixed = 135.0 – 105.0 = 56.25%
Incremental = 140.0 – 100.0 = 58.33%

Global Score: 275.0-205.0 = 57.29%

Against : Houdini 4.0 St. Ct0 (3233) = 45.94% ; Komodo 6 (3162) = 55.94% ; Critter 1.6a (3093) = 70.00%


Average Estimated Elo Opponents = 3.163
Estimated Elo Performance= 3.214

Regards,

Tom
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: no more tests?

Post by Tomcass »

TESTING STOCKFISH 151213 IPMAN LARGE PAGES= 480 GAMES


i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages= Allowed
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Stockfish 151213IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish 151213IP 64 SSE4.2L - Komodo 6 64-bitNOBx6 21.5 - 18.5 +10/=23/-7 53.75%
Stockfish 151213IP 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 26.0 - 14.0 +16/=20/-4 65.00%

Time Control= 2+2

Stockfish 151213IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 20.5 - 19.5 +9/=23/-8 51.25%
Stockfish 151213IP 64 SSE4.2L - Komodo 6 64-bitNOBx6 28.0 - 12.0 +19/=18/-3 70.00%
Stockfish 151213IP 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 25.5 - 14.5 +13/=25/-2 63.75%

Score using 6 cores= 143.0 – 97.0 = 59.58%
240 Games= http://www.mediafire.com/view/yykurjjpy ... 0games.pgn

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages= Allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

Stockfish 151213IP 64 SSE4.2L - Houdini 4 x64xCT0 23.0 - 17.0 +13/=20/-7 57.50%
Stockfish 151213IP 64 SSE4.2L - Komodo 6 64-bitx4 25.5 - 14.5 +16/=19/-5 63.75%
Stockfish 151213IP 64 SSE4.2L - Critter 1.6a 64-bitnob 22.5 - 17.5 +11/=23/-6 56.25%

201312SF151213_LP_IP_55_2+2 2013

Stockfish 151213IP 64 SSE4.2L - Houdini 4 x64xCT0 22.5 - 17.5 +12/=21/-7 56.25%
Stockfish 151213IP 64 SSE4.2L - Komodo 6 64-bitx4 25.5 - 14.5 +15/=21/-4 63.75%
Stockfish 151213IP 64 SSE4.2L - Critter 1.6a 64-bitnob 24.0 - 16.0 +10/=28/-2 60.00%
Score using 4 Cores= 143.0 – 97.0 = 59.58%
240 Games= http://www.mediafire.com/view/tch539otn ... 0games.pgn

Segmenting by Time Control:

Fixed TC = 140.0 – 100.0 = 58.33%
Incremental TC = 146.0 – 94.0 = 60.83%

Global Score= 286.0 – 194.0 = 59.58%

Against : Houdini 4.0 St. Ct0 (3233) = 54.69% ; Komodo 6 (3162) = 60.31% ; Critter 1.6a (3093) = 61.25%

Average Estimated Elo Opponents = 3163
Estimated Elo Performance= 3230


This is clearly the best Stockfish I have tested so far. Very similar strength to Houdini 4.0 Standard A Contempt 0, the best Houdini 4.0 in my computer.

Regards,

Tom.
duncan
Posts: 12038
Joined: Mon Jul 07, 2008 10:50 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by duncan »

Tomcass wrote:TESTING STOCKFISH 110313: 480 GAMES.

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases

Stockfish 110313 – Criter 1.6 +8/=26/-6 21.0 – 19.0 52.50%
Stockfish 110313 – Deep Rybka 4.1 +12/=21/-7 22.5 – 17,5 56.25%
Stockfish 110313 – Houdini 3.0Pro +5/=16/-19 13.0 – 27.0 32.50%

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2 min +2 sec/ game
No tablebases

Stockfish 110313 – Criter 1.6 +13/=22/-5 24.0 – 16.0 60.0%
Stockfish 110313 – Deep Rybka 4.1 +13/=23/-4 24.5 – 15.5 61.25%
Stockfish 110313 – Houdini 3.0Pro +7/=15/-18 14.5 – 25.5 36.25%

240 games x 6 cores http://www.mediafire.com/?cu97cn269a6rdfs

Overall Average with 6 cores (119.5 – 120.5) = 49.58%



i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

Stockfish 110313 - Critter 1.6a +7/=23/-10 18.5 - 21.5 46.25%
Stockfish 110313 - Deep Rybka +12/=20/-8 22.0 -18.0 55.00%
Stockfish 110313 - Houdini3.0Pro +3/=21/-16 13.5 – 26.5 33.75%
i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min + 2 sec/ game
Ponder: Off
No tablebases

Stockfish 110313 - Critter 1.6a +6/=29/-5 20.5 – 19.5 51.25%
Stockfish 110313 - Deep Rybka +13/=22/-5 24.0 – 16.0 60.00%
Stockfish 110313 - Houdini3.0Pro +9/=24/-7 21.0 – 19.0 52.50%

240 games x 4cores http://www.mediafire.com/?xmprtbf7bf4c5mv

Overall average with 4 cores ( 119.5 – 120.5) : 49.58%

… and after 480 games: 239.0 – 241.0 : 49.58%

Just two comments:

- This version of Stockfish has got the best result ever in my tests.
- Please note the substantial difference between 4+0 and 2+2 time control.

Regards,

Tom.
so we can see stockfish elo progress in 9 months, do you have the elo score for this version ?
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

Hi, Duncan.

You can not take my ratings very seriously, because I make a very simple approach, not reliable at all. But only as a mere reference, here you have a vision of the amazing improvement of Stockfish in the latest 8 months:

STOCKFISH 07-04-13 default : 3099 Estimated Elo Points.

STOCKFISH IPMAN WITH LP 15-12-13 : 3230 Estimated Elo Points
.


That is 131 Estimated Elo Points more. Brilliant!!.

Tom.
User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 8:44 pm
Location: Amman,Jordan

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Dr.Wael Deeb »

Tomcass wrote:Hi, Duncan.

You can not take my ratings very seriously, because I make a very simple approach, not reliable at all. But only as a mere reference, here you have a vision of the amazing improvement of Stockfish in the latest 8 months:

STOCKFISH 07-04-13 default : 3099 Estimated Elo Points.

STOCKFISH IPMAN WITH LP 15-12-13 : 3230 Estimated Elo Points
.


That is 131 Estimated Elo Points more. Brilliant!!.

Tom.
So after 2-3 months,Houdini will eat its dust I guess :D
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
duncan
Posts: 12038
Joined: Mon Jul 07, 2008 10:50 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by duncan »

Tomcass wrote:Hi, Duncan.

You can not take my ratings very seriously, because I make a very simple approach, not reliable at all. But only as a mere reference, here you have a vision of the amazing improvement of Stockfish in the latest 8 months:

STOCKFISH 07-04-13 default : 3099 Estimated Elo Points.

STOCKFISH IPMAN WITH LP 15-12-13 : 3230 Estimated Elo Points
.


That is 131 Estimated Elo Points more. Brilliant!!.

Tom.
yes brilliant and if he continues it will be almost 200 points a year. thanks for all your hard work testing.

has anyone else improved at such a rate. rybka 2 or 3 ?
User avatar
Ajedrecista
Posts: 2144
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Ajedrecista »

Hello Duncan:
duncan wrote:yes brilliant and if he continues it will be almost 200 points a year. thanks for all your hard work testing.

has anyone else improved at such a rate. rybka 2 or 3 ?
Quazar comes to my mind:

Version History

IIRC, those estimates of +500 Elo in less than five months were confirmed by rating lists. Quazar 0.4 is a bit stronger than Zappa Mexico II (Quazar 0.4 had something like 2730 rating at IPON, writing from memory. It means around 70 Elo weaker than Shredder 12).

Dendograms do not show that Quazar 0.4 is a clone. All I can say is that I find a similar behaviour with SF in infinite analysis: run fast into depth, similar fail highs/fail lows (I mean, when the engine finds a good move and there are little jumps in evaluation at the same depth of 0.08, 0.16, etc.). Please note that I am not a programmer, it is only my POV.

Regards from Spain.

Ajedrecista.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Testing Stockfish 11-03-13. 480 Games.

Post by Tomcass »

TESTING STOCKFISH 181213 IPMAN LARGE PAGES= 480 GAMES


i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages= Allowed
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Stockfish 181213IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 21.0 - 19.0 +10/=22/-8 52.50%
Stockfish 181213IP 64 SSE4.2L - Komodo 6 64-bitNOBx6 25.0 - 15.0 +15/=20/-5 62.50%
Stockfish 181213IP 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 25.5 - 14.5 +16/=19/-5 63.75%

Time Control= 2+2

Stockfish 181213IP 64 SSE4.2L - Houdini 4 x64_st_X6_CT0 24.5 - 15.5 +14/=21/-5 61.25%
Stockfish 181213IP 64 SSE4.2L - Komodo 6 64-bitNOBx6 22.5 - 17.5 +11/=23/-6 56.25%
Stockfish 181213IP 64 SSE4.2L - Critter 1.6a 64-bitX6_NOB 27.5 - 12.5 +18/=19/-3 68.75%

Score using 6 cores= 146.0 – 94.0 =
240 Games= http://www.mediafire.com/download/dw3nec12l348d47/

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Large Pages= Allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

Stockfish 181213IP 64 SSE4.2L - Houdini 4 x64xCT0 22.0 - 18.0 +9/=26/-5 55.00%
Stockfish 181213IP 64 SSE4.2L - Komodo 6 64-bitx4_NOB 24.5 - 15.5 +12/=25/-3 61.25%
Stockfish 181213IP 64 SSE4.2L - Critter 1.6a 64-bitnob_4 24.5 - 15.5 +13/=23/-4 61.25%

Time Control= 2+2

Stockfish 181213IP 64 SSE4.2L - Houdini 4 x64xCT0 17.0 - 23.0 +6/=22/-12 42.50%
Stockfish 181213IP 64 SSE4.2L - Komodo 6 64-bitx4_NOB 25.0 - 15.0 +17/=16/-7 62.50%
Stockfish 181213IP 64 SSE4.2L - Critter 1.6a 64-bitnob_4 26.5 - 13.5 +16/=21/-3 66.25%

Score using 4 Cores: 139.5-100.5 = 58.12%
240 Games= http://www.mediafire.com/download/oz1ru1j6wyqbx69/

Segmenting by Time Control:

Fixed TC = 142.5 – 97.5 = 59.37%
Incremental TC = 143.0 – 97.0 = 59.58%

Global Score= 285.5 – 194.5 = 59.48%

Against : Houdini 4.0 St. Ct0 (3233) = 52.81% ; Komodo 6 (3162) = 60.62% ; Critter 1.6a (3093) = 65.00%

Average Estimated Elo Opponents = 3163
Estimated Elo Performance= 3229


Let’s test now the latest and promising SF development version.

Regards and Merry X’mas from Barcelona,

Tom

… with my right leg broken playing golf (!!!). At my age I should not practice these high risk sports. :wink:
fauzi
Posts: 61
Joined: Wed Nov 20, 2013 10:42 am

Re: Testing Stockfish 11-03-13. 480 Games.

Post by fauzi »

the second file is set to private, cannot download