Testing Stockfish 130216. 480 games.

[Tomcass]

TESTING STOCKFISH 130216: 480 GAMES.

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min + 0 sec/ game
No tablebases

Stockfish 130216 – Criter 1.6a 18.5 – 21.5 +4/=29/-7 46.25%
http://www.mediafire.com/?a5mou47898ngasa

Stockfish 130216 – Deep Rybka 4.1 21.5 – 18.5 +12/=19/-9 53.75%
http://www.mediafire.com/?ncse3ig78gec7ag

Stockfish 130216 – Houdini 3.0Pro 15.5 – 24.5 +5/=21/-14 38.25%
http://www.mediafire.com/?cusdb42499l8dah

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2 min +2 sec/ game
No tablebases

Stockfish 130216 – Criter 1.6a 21.5 – 18.5 +10/=23/-7 53.75%
http://www.mediafire.com/?ulqquaqkxdqn3gk

Stockfish 130216 – Deep Rybka 4.1 19.5 – 20.5 +7/=25/-8 48.75%
http://www.mediafire.com/?8z4vj8vjrvwiwkw

Stockfish 130216 – Houdini 3.0Pro 15.5 – 24.5 +7/-17/-16 38.25%
http://www.mediafire.com/?6rkorpw9cg09hkh

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

Stockfish 130216 - Critter 1.6a 22.5 - 17.5 +12/=21/-7 56.25%
http://www.mediafire.com/?vambf1a769oizot

Stockfish 130216 - Deep Rybka 4 22.0 - 18.00 +8/=28/-4 55.00%
http://www.mediafire.com/?fb28bpuc5e66822

Stockfish 130216 - Houdini3.0Pro 18.00 - 22.00 +6/=24/-10 45.00%
http://www.mediafire.com/?z63bvn4cpi9uz4j

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min +2 sec/ game
Ponder: Off
No tablebases

Stockfish 130216 - Critter 1.6a 19.5 - 20.5 +7/=25/-8 48.75%
http://www.mediafire.com/?6czc0j5twjc9mwe

Stockfish 130216 - Deep Rybka 4 22.5 - 17.5 +13/=19/-8 56.25%
http://www.mediafire.com/?hrp1i618jhpg6zc

Stockfish 130216 - Houdini3.0Pro 15.5 - 24.5 +5/=21/-14 38.25%
http://www.mediafire.com/?4djr9t4df1a5l69

SUMMARY AFTER 480 GAMES:

Stockfish 130216 – Criter 1.6 82.0 – 78.0 51.25%
Stockfish 130216 – Deep Rybka 4 85.5 – 74.5 53.44%
Stockfish 130216 – Houdini 3.0Pro 64.5 – 95.5 40.31%

Overall average score: 48.33%

Regards from Barcelona.

Tom.

[gladius]

Thanks Tom! This seems to be one of the better results. There was a change applied to master today that should be helpful.

[Tomcass]

Thanks to you, Gary. I will test the version that includes this change.

Tom.

[Jouni]

One question about autocompiles at http://abrok.eu/stockfish/. Are they as fast as JA compiles? If there is minor improvement it is very hard to detect if compiles is slower.

[Tomcass]

Hi Jouni, I can not answer your question. Lately I am testing only the compiles in abrok.eu.

Here you have the result after 480 additional games for the 130220 version.

TESTING STOCKFISH 130220. 480 games:

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min + 0 sec/ game
No tablebases

Stockfish 130220 – Criter 1.6a +5/=22/-13 16.0 – 24.0 40.00%
Stockfish 130220 – Deep Rybka 4.1 +13/=18/-9 22.0 – 18.0 55.00%
Stockfish130220 – Houdini 3.0Pro +6/=22/-12 17.0 – 23.0 42.50%

120 Games: http://www.mediafire.com/?czv73kh4191f7dd

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2 min +2 sec/ game
No tablebases

Stockfish 130220 – Criter 1.6 +9/=23/-8 20.5 – 19.5 51.25%
Stockfish 130220 – Deep Rybka 4.1 +15/=15/-10 22.5 – 17.5 56.25%
Stockfish 130220 – Houdini 3.0Pro +5/=24/-11 17.0 – 23.0 42.50%

Games: http://www.mediafire.com/?mzqr6oixpoisrdx

Overall average with 6 cores: (115.0 – 125.0). 47.92%

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

Stockfish 130220 - Critter 1.6a +6/=25/-9 18.5 – 21.5 46.25%
Stockfish 130220 - Deep Rybka +13/=19/-8 22.5 – 17.5 56.25%
Stockfish 130220 - Houdini3.0Pro +11/=24/-5 23.0 – 17.0 57.50%

Games: http://www.mediafire.com/?cfdpw089b4gj77i

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min +2 sec/ game
Ponder: Off
No tablebases

Stockfish 130220 - Critter 1.6a +10/=25/-5 22.5 – 17.5 56.25%
Stockfish 130220 - Deep Rybka 4 +6/=28/-6 20.0 – 20.0 50.00%
Stockfish 130220 - Houdini3.0Pro +7/=18/-15 16.0 – 24.0 40.00%
Games: http://www.mediafire.com/?8u7fdqlxhrmpodf
Overall average with 4 cores: ( 122.5 – 117.5 ) 51.05%

GLOBAL AVERAGE AFTER 480 GAMES: 49.48%

Only as a reference, the GLOBAL AVERAGE SCORE AFTER 480 for Stockfish 130216 was: 48.33%. Effect of the x-ray Gary’s bishop or statistical effect? I don’t know. Anyway, a big WELL DONE for the Stockfish team!.

Regards,

Tom.

[Tomcass]

TESTING STOCKFISH 130223:

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min + 0 sec/ game
No tablebases

Stockfish 23-02-13 – Criter 1.6a +4/=26/-10 17.0 – 23.0 42.50%
Stockfish 23-02-13 – Deep Rybka 4.1 +11/=22/-7 22.0 -18.0 55.00%
Stockfish 23-02-13 – Houdini 3.0Pro +10/=16/-14 18.0 – 22.0 45.00%

Games: http://www.mediafire.com/?y73s77z1l5olyi1

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2 min +2 sec/ game
No tablebases

Stockfish 23-02-13 – Criter 1.6 +10/=21/-9 20.5 – 19.5 51.25%
Stockfish 23-02-13 – Deep Rybka 4.1 +9/=23/-8 20.5 – 19.5 51.25%
Stockfish 23-02-13 – Houdini 3.0Pro +5/=21/-14 16.5 – 23.5 41.25%

Games: http://www.mediafire.com/?i0ccsd75hxc835c

Overall Average with 6 cores: (114.5 -125.5) 47.71%

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

Stockfish 23-02-13 - Critter 1.6a +4/=30/-6 19.0 – 21.0 47.50%
Stockfish 23-02-13 - Deep Rybka +6/=29/-5 20.5 -19.5 51.25%
Stockfish 23-02-13 - Houdini3.0Pro +5/=21/-14 16.5 – 23.5 41.25%
Games: http://www.mediafire.com/?o27s7b5qjzoax87

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min +2 sec/ game
Ponder: Off
No tablebases

Stockfish 23-02-13 - Critter 1.6a +6/=24/-10 18.0 – 22.0 45.00%
Stockfish 23-02-13 - Deep Rybka 4 +10/=25/-5 22.5 – 17.5 56.25%
Stockfish 23-02-13 - Houdini3.0Pro +9/=15/-16 16.5 – 23.5 41.25%

Games: http://www.mediafire.com/?f00aty2r22yo5wb

Overall Average using 4 cores: (113.0 – 127.0) 47.08%

Global Average Score after 480 games: (227.5 – 252.5) = 47.40%

The result has been a bit below the two previous tests. Anyway now it is more reliable from statistical point of view with 1.440 games in three tests.

Regards,

Tom

[Tomcass]

And finally, segmenting the result by opponent:

against Critter 1.6 = 234.0 - 246.0 48.75%
against Deep Rybka 4 = 258.5 - 221.5 53.85%
against Houdini 3.0 = 203-0 - 277.0 42.29%

This test is finished ... until new Stockfish developments, of course!.

Tom.