Testing Stockfish 11-03-13. 480 Games.

[Tomcass]

Thanks for sharing this information, Bill. Well done!. Anyway I will finish the second leg of my test at Incremental Time Control.

Regards,

Tom.

[voyagerOne]

Ok sounds good.

After that test and if you don't mind can you run a quick test on the prior dev:

Author: mstembera
Date: Wed Jul 15 20:17:16 2015 +0100
Timestamp: 1436987836

Consistent TT replace policy
Bench: 8248164

[Tomcass]

I will test it, Bill. 480 Games.

Tom.

[Tomcass]

TESTING STOCKFISH DEVELOPMENT 160715 = 1440 GAMES

Timestamp: 1437027460 Bench: 6943812

SECOND LEG OF 720 GAMES AT INCREMENTAL TIME CONTROL 2+2.

6 real cores

Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control 2+2

Stockfish 160715 64 POPCNTx6 - Komodo 9.1 64-bit_x6 61.5 - 58.5 +19/=85/-16 51.25%
Stockfish 160715 64 POPCNTx6 - Houdini 4 x64_st_X6_CT0 83.5 - 36.5 +53/=61/-6 69.58%
Stockfish 160715 64 POPCNTx6 - Gull 3 x64 XP 79.5 - 40.5 +44/=71/-5 66.25%

360 Games:
http://www.mediafire.com/download/1hbabayg2spv32z

8 real cores

Ponder: Off
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 44.92
Knodes per Second: 21.562

Time Control: 2+2

Stockfish 160715 64 BMI2x8 - Komodo 9.1 64-bit_x8 55.0 - 65.0 +12/=86/-22 45.83%
Stockfish 160715 64 BMI2x8 - Houdini 4 Pro x64_Ct0_8 75.5 - 44.5 +37/=77/-6 62.92%
Stockfish 160715 64 BMI2x8 - Gull 3 x64 XPx8 77.0 - 43.0 +42/=70/-8 64.17%

360 Games:
http://www.mediafire.com/download/7qg9msg9y17q1wu

GLOBAL SCORE AFTER 720 GAMES AT INCREMENTAL TIME CONTROL: 432.0 – 288.0 = 60.00%

Average Elo of Oponents= 3.162

Estimated Elo Performance for Stockfish Development 160715 after 720 games= 3.232

As a reference, the score of Stockfish Development 270615 (Timestamp: 1435394759 Bench: 8646407) after 720 games at Incremental Time Control 4+0 was 436.0 – 284.0 = 60.56% . And its estimated Elo Score: 3.236

In Summary,after 1440 games for each version:

SF DEV270615 886.5- 553.5 61.56% 3243
SF DEV160715 863.5-576.5 59.96% 3232


Error bars: 13 Elo points.

Regards,

Tom.

[Tomcass]

TESTING STOCKFISH DEVELOPMENT 150715 = 480 GAMES

Timestamp: 1436987836 Bench: 8248164

6 real cores

Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time control 4+0

Stockfish 150715 64 POPCNT_ - Komodo 9.1 64-bit_x6 19.5 - 20.5 +4/=31/-5 48.75%
Stockfish 150715 64 POPCNT_ - Houdini 4 x64_st_X6_CT0 22.5 - 17.5 +9/=27/-4 56.25%
Stockfish 150715 64 POPCNT_ - Gull 3 x64 XP 29.5 - 10.5 +20/=19/-1 73.75%

Time Control 2+2

Stockfish 150715 64 POPCNT_ - Komodo 9.1 64-bit_x6 19.5 - 20.5 +6/=27/-7 48.75%
Stockfish 150715 64 POPCNT_ - Houdini 4 x64_st_X6_CT0 24.0 - 16.0 +11/=26/-3 60.00%
Stockfish 150715 64 POPCNT_ - Gull 3 x64 XP 23.5 - 16.5 +11/=25/-4 58.75%

240 Games:
http://www.mediafire.com/download/9e11o63vq35o8ot

8 real cores

Ponder: Off
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 44.92
Knodes per Second: 21.562

Time control: 4+0

Stockfish 150715 64 BMI2_ - Komodo 9.1 64-bit_x8 21.0 - 19.0 +8/=26/-6 52.50%
Stockfish 150715 64 BMI2_ - Houdini 4 Pro x64_Ct0_8 28.0 - 12.0 +18/=20/-2 70.00%
Stockfish 150715 64 BMI2_ - Gull 3 x64 XPx8 25.0 - 15.0 +12/=26/-2 62.50%

Time control 2+2

Stockfish 150715 64 BMI2_ - Komodo 9.1 64-bit_x8 17.0 - 23.0 +3/=28/-9 42.50%
Stockfish 150715 64 BMI2_ - Houdini 4 Pro x64_Ct0_8 21.0 - 19.0 +7/=28/-5 52.50%
Stockfish 150715 64 BMI2_ - Gull 3 x64 XPx8 24.0 - 16.0 +8/=32/-0 60.00%

240 Games:
http://www.mediafire.com/download/mh2gfmg3hfzondf

Segmenting by Time Control:
Fixed Time Control: 145.5 - 94.5 = 60.62%
Incremental Time Control: 129.0 – 111.0 = 53.75%

GLOBAL SCORE AFTER 480 GAMES= 274.5 – 205.5 = 57.19%

Against : Komodo 9.1 (3247) = 48.12% Houdini 4 (3136) = 59.69%, Gull 3 XP (3103) = 63.75%

Average Elo of Oponents= 3.162

Estimated Elo for Stockfish Dev. 150715 = 3.212

SF DEV 270615 3243
SF DEV 150715 3212
SF DEV 160715 3232

The regression point seems to be somewhere between SF DEV 270615 (peak) and SF DEV 150715 .

I will disappear for one month, away from my computers. Enjoy your summer!.

Tom.

[ernest]

The regression point seems to be somewhere between SF DEV 270615 (peak) and SF DEV 150715 .

Stefan Pohl's Testrun of Stockfish 150716
either doesn't confirm that regression,
or show that it was corrected precisely with the 150716 version !

[voyagerOne]

Thanks for all your testings...

Enjoy your summer.

[Tomcass]

Thanks for all your testings...

Enjoy your summer.

... thanks to you for following them, Bill.

TESTING STOCKFISH DEVELOPMENT 200815 = 960 GAMES

Timestamp: 1440098826 Bench: 7620871

6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time control 4+0

Stockfish 200815 64 POPCNTx6 - Komodo 9.1 64-bit_x6 41.0 - 39.0 +15/=52/-13 51.25%
Stockfish 200815 64 POPCNTx6 - Houdini 4 x64_st_X6_CT0 50.0 - 30.0 +25/=50/-5 62.50%
Stockfish 200815 64 POPCNTx6 - Gull 3 x64 XP 52.5 - 27.5 +28/=49/-3 65.63%

Time Control 2+2

Stockfish 200815 64 POPCNTx6 - Komodo 9.1 64-bit_x6 36.5 - 43.5 +9/=55/-16 45.63%
Stockfish 200815 64 POPCNTx6 - Houdini 4 x64_st_X6_CT0 54.5 - 25.5 +32/=45/-3 68.13%
Stockfish 200815 64 POPCNTx6 - Gull 3 x64 XP 55.0 - 25.0 +33/=44/-3 68.75%

Score using 6 cores: 289.5 – 190.5 = 60.31%
480 Games:
http://www.mediafire.com/download/c8jcu ... 0Games.pgn

8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 44.92
Knodes per Second: 21.562

Time control: 4+0

Stockfish 200815 64 BMI2x8 - Komodo 9.1 64-bit_x8 40.0 - 40.0 +12/=56/-12 50.00%
Stockfish 200815 64 BMI2x8 - Houdini 4 Pro x64_Ct0_8 55.5 - 24.5 +36/=39/-5 69.38%
Stockfish 200815 64 BMI2x8 - Gull 3 x64 XPx8 58.0 - 22.0 +39/=38/-3 72.50%

Time control 2+2

Stockfish 200815 64 BMI2x8 - Komodo 9.1 64-bit_x8 39.5 - 40.5 +14/=51/-15 49.38%
Stockfish 200815 64 BMI2x8 - Houdini 4 Pro x64_Ct0_8 50.0 - 30.0 +26/=48/-6 62.50%
Stockfish 200815 64 BMI2x8 - Gull 3 x64 XPx8 56.5 - 23.5 +34/=45/-1 70.63%

Score using 8 cores: 299.5 - 180.5= 62.40%

480 Games:
http://www.mediafire.com/download/x48nh ... Games_.pgn

Segmenting by Time Control:
Fixed Time Control: 297.0 – 183.0= 61.87%
Incremental Time Control: 292.0 – 188.0 = 60.83%

GLOBAL SCORE AFTER 960 GAMES= 589.0 – 371.0 = 61.35%

Against : Komodo 9.1 (3247) = 49.06% Houdini 4 (3136) = 65.63%, Gull 3 XP (3103) = 69.38%
Average Elo of Oponents= 3.162
Estimated Elo Performance for Stockfish Development 200815 = 3241
Error bars = +/- 16

This is not a new record in my tests, but Stockfish Development 200815 has performed only 2 points below than the best SF Development scorer so far, SF Development 270615 (3243). Obviously within error bars. The top scorer in my tests under my standard testing conditions is Komodo 9.1, with an Estimated Elo Performance of 3247.

Best regards from Barcelona.

Tom

[JJJ]

And now you have to test Komodo 9.2

[Tomcass]

And now you have to test Komodo 9.2

The 1920 games test of Komodo 9.2 has started 8 hours ago, Jean Baptiste. It will last about five days non-stop in my two computers.

Tom.