Testing Stockfish 11-03-13. 480 Games.

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: For Tom- or Maybe Anyone Who Knows!!

Post by Tomcass »

beram wrote:hi Tom,

Where can we get this 'garricar' SF DEV 021115 KP compile ?
I don't know if I can post it here, Bram. I send the link to you.

Regards,

Tom.

... edited: Woops! The link I have is broken. I will ask another one to GarryCar.
Krgp
Posts: 20
Joined: Mon Nov 04, 2013 6:18 am

Re: For Tom- or Maybe Anyone Who Knows!!

Post by Krgp »

Here is the link.

https://www.dropbox.com/s/ak7vvd7c43daq ... e.zip?dl=0

It's the same link but now working.
KP
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: For Tom- or Maybe Anyone Who Knows!!

Post by Tomcass »

Krgp wrote:Here is the link.

https://www.dropbox.com/s/ak7vvd7c43daq ... e.zip?dl=0

It's the same link but now working.
Thanks Kiran! Your compile is extremely strong.

Best regards from Barcelona.

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: For Tom- or Maybe Anyone Who Knows!!

Post by Tomcass »

TESTING STOCKFISH DEVELOPMENT KP 021115 = 2.400 GAMES
SECOND LEG: 480 GAMES AT INCREMENTAL TIME CONTROL


6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time control=2 + 2
Stockfish 021115 KP_X6 - Komodo 9.2 64-bit_x6 33.5 - 26.5 +15/=37/-8 55.83%
Stockfish 021115 KP_X6 - Stockfish 190915 64 POPCNT_6 30.0 - 30.0 +5/=50/-5 50.00%
Stockfish 021115 KP_X6 - Houdini 4 x64_st_X6_CT0 41.0 - 19.0 +26/=30/-4 68.33%
Stockfish 021115 KP_X6 - Gull 3 x64 XP 40.0 - 20.0 +25/=30/-5 66.67%

Score after 240 games using 6 cores= 144.5 – 95.5= 60.21%
240 Games = http://www.mediafire.com/download/xhhfdpq8c0ch23x

8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 53.22
Knodes per Second: 25.543

Time Control= 2 + 2
Stockfish 021115 KP_X8 - Komodo 9.2 64-bit_x8 35.5 - 24.5 +18/=35/-7 59.17%
Stockfish 021115 KP_X8 - Stockfish 190915 64 BMI2_8 32.5 - 27.5 +7/=51/-2 54.17%
Stockfish 021115 KP_X8 - Houdini 4 Pro x64_Ct0_8 39.5 - 20.5 +22/=35/-3 65.83%
Stockfish 021115 KP_X8 - Gull 3 x64 XPx8 40.0 - 20.0 +24/=32/-4 66.67%

Score after 240 games using 8 cores= 147.5 – 92.5 = 61.46%
240 Games= http://www.mediafire.com/download/3b3pib89lfka2f3

Against : Komodo 9.2 (3251) = 57.50% Stockfish 190915 (3262) = 52.08% Houdini 4 (3136) = 67.08%, Gull 3 XP (3103) = 66.67%

Average Elo of Oponents= 3.188

GLOBAL SCORE FOR SF DEV 021115 KP AFTER 480 GAMES AT INCREMENTAL TIME CONTROL= 292.0 – 188.0= 60.83%

ESTIMATED ELO FOR SF DEV 021115 KP AT INCREMENTAL TIME CONTROL AFTER 480 GAMES= 3.270 = 3264


---------

GLOBAL ESTIMATED ELO FOR SF DEV 021115 KP AFTER 2.400 GAMES= (3.270 -Fixed- + 3.264 -Incremental-) / 2= 3.267

This is a new best score ever in my tests. 5 ELO points better than the previous top scorer Stockfish Development 190915.

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: For Tom- or Maybe Anyone Who Knows!!

Post by Tomcass »

NEW RANKING OF SOME SELECTED ENGINES IN MY TESTS:

1.- Stockfish 021115 KP= 3.267
2.- Stockfish Development 190915= 3.262
3.- Stockfish Bullet 280915= 3.259
4.- Komodo 9.2 = 3.251.
5.- Komodo 9.1= 3.247
6.- Sugar Pro V.1= 3.246
7.- Stockfish 6= 3.219
8.- Komodo 9= 3.209
9.- Houdini 4= 3.136
10.- Gull 3 XP= 3.103

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: For Tom- or Maybe Anyone Who Knows!!

Post by Tomcass »

TESTING STOCKFISH 121115 IPMAN COMPILE: 1920 GAMES
FIRST LEG: 960 GAMES AT FIXED TIME CONTRO
L


6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time control=4 +0
SF 121115IP 64 POPCNT_X6 - Komodo 9.2 64-bit_x6 69.0 - 51.0 +37/=64/-19 57.50%
SF 121115IP 64 POPCNT_X6 - Stockfish 190915 64 POPCNT_6 62.0 - 58.0 +14/=96/-10 51.67%
SF 121115IP 64 POPCNT_X6 - Houdini 4 x64_st_X6_CT0 85.5 - 34.5 +57/=57/-6 71.25%
SF 121115IP 64 POPCNT_X6 - Gull 3 x64 XP 83.5 - 36.5 +55/=57/-8 69.58%

Score after 480 games using 6 cores= 300.0 – 180.0= 62.50%
480 Games = http://www.mediafire.com/download/lt45382f8s76thq

8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 53.22
Knodes per Second: 25.543

Time Control= 4 + 0
SF 121115IP 64 BMI2_X8 - Komodo 9.2 64-bit_x8 65.5 - 54.5 +28/=75/-17 54.58%
SF 121115IP 64 BMI2_X8 - Stockfish 190915 64 BMI2_8 66.0 - 54.0 +16/=100/-4 55.00%
SF 121115IP 64 BMI2_X8 - Houdini 4 Pro x64_Ct0_8 86.5 - 33.5 +57/=59/-4 72.08%
SF 121115IP 64 BMI2_X8 - Gull 3 x64 XPx8 86.5 - 33.5 +58/=57/-5 72.08%

Score after 480 Games using 8 cores= 304.5 – 175.5= 63.44%
480 Games: http://www.mediafire.com/download/w41mgo20cjt6002


GLOBAL SCORE FOR SF 121115 IPMAN AFTER 960 GAMES AT FIXED TIME CONTROL= 604.5 – 355.5 = 62.97%

ESTIMATED ELO FOR SF 121115 IPMAN AT FIXED TIME CONTROL AFTER 1920 GAMES= 3.279


This is the best performance ever in my tests at fixed time control.

Only as a reference, it has performed:

- 9 ELO points (!!) better than the previous top scorer at fixed time control SF DEV 021115 KP
- 24 ELO better than KOMODO 9.2
- 28 ELO better than STOCKFISH DEVELOPMENT 201015 LAZY.
- 60 ELO better than STOCKFISH 6

Well done, IPMAN!. Let’s see what happen at incremental time control. A 480 games test have just started.

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: For Tom- or Maybe Anyone Who Knows!!

Post by Tomcass »

TESTING STOCKFISH 121115 IPMAN COMPILE: 1920 GAMES

SECOND LEG: 960 GAMES AT INCREMENTAL TIME CONTROL


6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time control = 2 + 2
SF 121115IP 64 POPCNT_X6 - Komodo 9.2 64-bit_x6 66.0 - 54.0 +33/=66/-21 55.00%
SF 121115IP 64 POPCNT_X6 - Stockfish 190915 64 POPCNT 69.0 - 51.0 +20/=98/-2 57.50%
SF 121115IP 64 POPCNT_X6 - Houdini 4 x64_st_X6_CT0 78.0 - 42.0 +42/=72/-6 65.00%
SF 121115IP 64 POPCNT_X6 - Gull 3 x64 XP 89.0 - 31.0 +60/=58/-2 74.17%

Score after 480 games using 6 cores: 302.0 – 178.0 = 62.92%
480 Games= http://www.mediafire.com/download/bdk9ffh7lvife6i/

8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 53.22
Knodes per Second: 25.543

Time control 2 + 2
SF 121115IP 64 BMI2_X8 - Komodo 9.2 64-bit_x8 58.5 - 61.5 +20/=77/-23 48.75%
SF 121115IP 64 BMI2_X8 - Stockfish 190915 64 BMI2_8 63.5 - 56.5 +12/=103/-5 52.92%
SF 121115IP 64 BMI2_X8 - Houdini 4 Pro x64_Ct0_8 85.5 - 34.5 +58/=55/-7 71.25%
SF 121115IP 64 BMI2_X8 - Gull 3 x64 XPx8 90.5 - 29.5 +63/=55/-2 75.42%

Score after 480 games using 8 cores: 298.0 – 182.0= 62.08%
480 Games= http://www.mediafire.com/download/9y6jt1sqe9zcxnu

GLOBAL SCORE FOR SF 121115 IPMAN AFTER 960 GAMES AT INCREMENTAL TIME CONTROL=
600.0 – 360.0 = 62.50%

-------------------

... AND AFTER 1920 GAMES:

Against : Komodo 9.2 (3251) = 53.96% Stockfish 190915 (3262) = 54.27% Houdini 4 (3136) = 69.89%, Gull 3 XP (3103) = 72.81%

Average Elo of Oponents= 3.188

GLOBAL SCORE FOR SF 121115 IPMAN AFTER 1.920 GAMES= 1.204.5 – 715.5= 62.73%

GLOBAL ESTIMATED ELO FOR SF 121115 IPMAN AFTER 1.920 GAMES= 3.277


Error bars= +/- 11

The excellent work of the Stockfish team and the expert tuning by Ipman produced this extremely strong engine, by far the best I have ever tested. It has performed 10 elo points better than the previous top scorer, Stockfish 021115 KP, and 58 elo better than Stockfish 6.

Congratulations to the Stockfish Team and to Ipman for this exceptional performance.

Let’s see if the new and hopefully very strong Komodo 9.3 is able to improve this amazing score.

My new ranking:

1.- Stockfish 121115 IPMAN= 3.277
2.- Stockfish 021115 KP= 3.267
3.- Stockfish Development 190915= 3.262
4.- Komodo 9.2 = 3.251.
5.- Komodo 9.1= 3.247
6.- Sugar Pro V.1= 3.246
7.- Stockfish 6= 3.219
8.- Komodo 9= 3.209
9.- Houdini 4= 3.136
10.- Gull 3 XP= 3.103

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: For Tom- or Maybe Anyone Who Knows!!

Post by Tomcass »

TESTING Stockfish Development 22122015 : 1920 GAMES
Timestamp: 1450778557 Bench: 8355485
FIRST LEG: 960 GAMES AT FIXED TIME CONTROL

6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control= 4+0
Stockfish 221215 64 POPCNT_ - Komodo 9.3 64-bit_ST_NOB_x6 69.0 - 51.0 +33/=72/-15 57.50%
Stockfish 221215 64 POPCNT_ - SugaR PrO v1.3 64 POPCNT_X6 58.0 - 62.0 +4/=108/-8 48.33%
Stockfish 221215 64 POPCNT_ - Houdini 4 Pro x64_Ct0_X6 83.5 - 36.5 +54/=59/-7 69.58%
Stockfish 221215 64 POPCNT_ - Gull 3 x64 XP_X6 83.0 - 37.0 +51/=64/-5 69.17%

Score after 480 games using 6 cores= 293.5 – 186.5 = 61.15%
480 Games = http://www.mediafire.com/download/294wcyyu75ariv1

8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 53.22
Knodes per Second: 25.543

Time Control 4+0
Stockfish 221215 64 BMI2_X8 - Komodo 9.3 64-bit_St_nob_X8 64.5 - 55.5 +20/=89/-11 53.75%
Stockfish 221215 64 BMI2_X8 - SugaR PrO v1.3 64 BMI2_X8 60.5 - 59.5 +9/=103/-8 50.42%
Stockfish 221215 64 BMI2_X8 - Houdini 4 Pro x64_Ct0_8 84.5 - 35.5 +53/=63/-4 70.42%
Stockfish 221215 64 BMI2_X8 - Gull 3 x64 XPx8 91.0 - 29.0 +63/=56/-1 75.83%

Score after 480 games using 8 cores= 300.5 – 179.5 = 62.60%
480 games: http://www.mediafire.com/download/d28g3bb062y8b38

Against Komodo 9.3 (3259) = 55.62% SugaR Pro v1.3 (3280)= 49.37% Houdini 4 Pro (3136)= 70.00% Gull 3XP (3103) = 72.50%

Global score for Stockfish Development 221215 after 960 games at Fixed Time Control=
594.0 – 366.0= 61.87%
Average Elo of Oponents= 3.195
Estimated Elo for SF Development 221215= 3.278


This is a very good score, only 4 ELO points below the leader SugaR Pro v1.3. Let’s see what happens at Incremental Time Control.

Regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: For Tom- or Maybe Anyone Who Knows!!

Post by Tomcass »

TESTING Stockfish Development 22122015 : 1920 GAMES
Timestamp: 1450778557 Bench: 8355485
SECOND LEG: 960 GAMES AT INCREMENTAL TIME CONTROL

6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control= 2+2
Stockfish 221215 64 POPCNT- Komodo 9.3 64-bit_NOBx6 65.0 - 55.0 +25/=80/-15 54.17%
Stockfish 221215 64 POPCNT - SugaR PrO v1.3 64 POPCNT_X6 61.5 - 58.5 +9/=105/-6 51.25%
Stockfish 221215 64 POPCNT - Houdini 4 Pro x64_Ct0_X6 79.0 - 41.0 +43/=72/-5 65.83%
Stockfish 221215 64 POPCNT_ - Gull 3 x64 XP_X6 84.0 - 36.0 +53/=62/-5 70.00%

Score after 480 games using 6 cores= 289.5 – 190.5 = 60.31
480 Games = http://www.mediafire.com/download/aar3jvpai81cs2v

8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 53.22
Knodes per Second: 25.543

Time Control 2+2
Stockfish 221215 64 BMI2_X8 - Komodo 9.3 64-bit_nob_X8 67.5 - 52.5 +27/=81/-12 56.25%
Stockfish 221215 64 BMI2_X8 - SugaR PrO v1.3 64 BMI2_X8 61.5 - 58.5 +7/=109/-4 51.25%
Stockfish 221215 64 BMI2_X8 - Houdini 4 Pro x64_Ct0_8 81.0 - 39.0 +46/=70/-4 67.50%
Stockfish 221215 64 BMI2_X8 - Gull 3 x64 XPx8 87.5 - 32.5 +60/=55/-5 72.92%

Score after 480 games using 8 cores: 297.5 – 182.5 = 61.98%
http://www.mediafire.com/download/pvvitath3ts2y0j

Global score for Stockfish Development 221215 after 960 games at Incremental Time Control=
587.0 – 373.0= 61.15%
Average Elo of Oponents= 3.195
Estimated Elo for SF Development 221215= 3.273

GLOBAL SCORE FOR STOCKFISH DEVELOPMENT 221215 AFTER 1920 GAMES at fixed and incremental time control
1181 – 739 = 61.51%
AVERAGE ELO OF OPONENTS= 3.195
ESTIMATED ELO FOR SF DEVELOPMENT 221215= 3.276


New ranking after this score:

1.- SugaR Pro v1.3= 3.280
2.- Stockfish 121115 IPMAN= 3.277
3.- Stockfish Development 221215 = 3.276
4.- Stockfish 021115 KP= 3.267
5.- Stockfish Development 190915= 3.262
6.- Komodo 9.3 = 3.259
7.- Komodo 9.2 = 3.251.
8.- Komodo 9.1= 3.247
9.- Stockfish 6= 3.219
10.- Komodo 9= 3.209
11.- Houdini 4= 3.136
12.- Gull 3 XP= 3.103

Best regards,

Tom.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: For Tom- or Maybe Anyone Who Knows!!

Post by Tomcass »

TESTING STOCKFISH 7 BETA 1. 480 GAMES
First leg: 240 Games at Fixed Time control.

6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control= 4+0
Stockfish7Beta1 64 POPCNT_6 - Komodo 9.3 64-bit_ST_NOB_x6 19.0 - 11.0 +9/=20/-1 63.33%
Stockfish 7Beta1 64 POPCNT_6- SugaR PrO v1.3 64 POPCNT_X6 13.5 - 16.5 +0/=27/-3 45.00%
Stockfish 7Beta1 64 POPCNT_6 - Houdini 4 Pro x64_Ct0_X6 20.5 - 9.5 +13/=15/-2 68.33%
Stockfish 7Beta1 64 POPCNT_6 - Gull 3 x64 XP_X6 20.5 - 9.5 +11/=19/-0 68.33%

Score after 120 games using 6 cores= 73.5 – 46.5 = 61.25%
120 Games = http://www.mediafire.com/download/qdfamad1nc4yk6z/

8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014c – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 53.22
Knodes per Second: 25.543

Time Control: 4+0
Stockfish 7Beta1 64 BMI2_8 - Komodo 9.3 64-bit_nob_X8 15.5 - 14.5 +7/=17/-6 51.67%
Stockfish 7Beta1 64 BMI2_8 - SugaR PrO v1.3 64 BMI2_X8 16.0 - 14.0 +2/=28/-0 53.33%
Stockfish 7Beta1 64 BMI2_8 - Houdini 4 Pro x64_Ct0_8 18.5 - 11.5 +10/=17/-3 61.67%
Stockfish 7Beta1 64 BMI2_8 - Gull 3 x64 XPx8 20.0 - 10.0 +11/=18/-1 66.67%

Score after 120 games using 8 cores= 70.0 – 50.0 = 58.33%
120 Games= http://www.mediafire.com/download/56z8csn40d5z79o/

GLOBAL SCORE FOR STOCKFISH 7 BETA1 AFTER 240 GAMES AT FIXED TIME CONTROL: 143,5 – 96.5 = 59.79%
AVERAGE ELO OF OPONENTS= 3.195
ESTIMATED ELO FOR SF DEVELOPMENT 221215= 3.264
.


Please note that according to this short test, whose number of games is not big enough to be statistically significant, Stockfish 7 Beta 1 has performed 14 ELO points below Stockfish Development 221215 at Fixed Time Control.

I have started the second leg of this test with 240 games at Incremental Time Control.

Regards,

Tom