Testing Gull 3: 480 Games

[Tomcass]

TESTING GULL 3 = 480 GAMES

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c Sedat
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control= 4+0

Gull 3 x64 - Houdini 4 x64_st_X6_CT0 33.0 - 47.0 +10/=46/-24 41.25%
Gull 3 x64 - Komodo TCECr 64-bitNOB_OK 35.5 - 44.5 +12/=47/-21 44.37%
Gull 3 x64 - Stockfish 290314 64 SSE4.2DV 33.0 - 47.0 +14/=38/-28 41.25%

240 Games = 42.29%
http://www.mediafire.com/view/hoadt3o3a ... 0Games.pgn

Time Control: 2+2

Gull 3 x64 - Houdini 4 x64_st_X6_CT0 37.0 - 43.0 +11/=52/-17 46.25%
Gull 3 x64 - Komodo TCECr 64-bitNOB_OK 42.0 - 38.0 +23/=38/-19 52.50%
Gull 3 x64 - Stockfish 290314 64 SSE4.2DV 33.5 - 46.5 +12/=43/-25 41.88%

240 Games= 46.88%
http://www.mediafire.com/view/mr7yjw0v7 ... 0Games.pgn

Global Score= 214.0 - 266.0 = 44.58%

Against : Houdini 4.0 St. Ct0 (3227) = 43.75% ; Komodo TCECr (3181) = 48.43%, Stockfish Dev. 290314 (3269) = 41.56%

Average Estimated Elo Opponents = 3226
Estimated Elo Performance= 3188

Error bars: +/- 23 EEP

My 4 cores computer is under repair. I have used for this 480 games test only my 6 cores computer. According with this test, Gull 3 has an Estimated Elo Performance of 3188, slightly above than Komodo TCECr (within error bars). The improvement over Gull 2.8 Beta is 47 Elo points, much better than what I expected. Gull 3 is extremely strong at incremental time control.

Thank you very much for this free powerful engine, ThinkingALot!

Regards,

Tom.

[Ozymandias]

You're going to replace Critter, I take it?

[Tomcass]

Hola, Juan!

You are right. In my latest 480 games tests of Stockfish I used Gull 2.8Beta instead of Critter 1.6a. From now I will use Gull 3 instead of Gull 2.8 Beta.

Un abrazo!

Tom.

[Sedat Canbaz]

Dear Tom,

Interesting results and thanks for your updates

I just wonder,
Did you test Gull 3 (and rest engines) with 6 cores or with 1 core ?

For example,
http://www.talkchess.com/forum/viewtopic.php?t=52101

I could not see any improvement in my SCCT results, where all engines (including Gull 3) are tested with 6 cores

Best,
Sedat

[Tomcass]

Hello, Sedat!

I tested Gull 3 using 6 cores. The improvement seemed substantial, but in other tests against Stockfish it has performed slightly below Komodo TCECr. Perhaps Gull 3 was a bit lucky in my first test. In my 6 cores computer and with my time controls, Gull 3.0 seems to be around 29/30 Elo points stronger than Gull 2.8 Beta, rather than 47 Elo better as it appeared to be in my fist test.

Enjoy your week-end!

Tom-

[Sedat Canbaz]

Hello, Sedat!

I tested Gull 3 using 6 cores. The improvement seemed substantial, but in other tests against Stockfish it has performed slightly below Komodo TCECr. Perhaps Gull 3 was a bit lucky in my first test. In my 6 cores computer and with my time controls, Gull 3.0 seems to be around 29/30 Elo points stronger than Gull 2.8 Beta, rather than 47 Elo better as it appeared to be in my fist test.

Enjoy your week-end!

Tom-

Hello Tom,

Yes...our test conditions are almost same, exception I used different openings, probably due to openings we see different performance...

And nice weekend to you too!

Greetings,
Sedat

[Tomcass]

TESTING GULL 3 XP = 640 GAMES

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014c Sedat
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control= 4+0

201405Gull 3 XP_4+0 2014

Gull 3 x64 XP - Houdini 4 x64_st_X6_CT0 16.0 - 24.0 +4/=24/-12 40.00%
Gull 3 x64 XP - Komodo TCECr 64-bitNOB_OK 21.5 - 18.5 +13/=17/-10 53.75%
Gull 3 x64 XP - Critter 1.6a 64-bitX6_NOB_ok 23.5 - 16.5 +15/=17/-8 58.75%
Gull 3 x64 XP - SF 270414IPx 64 SSE4.2_ 17.0 - 23.0 +7/=20/-13 42.50%

Time Control= 2+2

201405Gull 3 XP_2+2 2014

Gull 3 x64 XP - Houdini 4 x64_st_X6_CT0 16.5 - 23.5 +4/=25/-11 41.25%
Gull 3 x64 XP - Komodo TCECr 64-bitNOB_OK 22.5 - 17.5 +9/=27/-4 56.25%
Gull 3 x64 XP - SF 270414IPx 64 SSE4.2x6 18.5 - 21.5 +9/=19/-12 46.25%
Gull 3 x64 XP - Critter 1.6a 64-bitX6_NOB_ok 23.5 - 16.5 +12/=23/-5 58.75%

320 Games= http://www.mediafire.com/view/d4ch1q9w6 ... 0Games.pgn

Score using 6 cores= 159.0 – 161.0 = 49.69%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2014c Sedat –limit 5 moves -
No tablebases. No RTB used.
Large Pages allowed
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Gull 3 x64 XP - Houdini 4 Pro x64_Ct0_OK 17.5 - 22.5 +7/=21/-12 43.75%
Gull 3 x64 XP - Komodo TCECr 64-bit_NOBx4 20.5 - 19.5 +7/=27/-6 51.25%
Gull 3 x64 XP - SF 270414IPx 64 SSE4.2_ 17.5 - 22.5 +7/=21/-12 43.75%
Gull 3 x64 XP - Critter 1.6a 64-bit_NOB 25.5 - 14.5 +14/=23/-3 63.75%
160 games=
http://www.mediafire.com/view/19a0c9v8i ... es_4+0.pgn

Time Control = 2+2

Gull 3 x64 XP - Komodo TCECr 64-bit_NOBx4 20.5 - 19.5 +9/=23/-8 51.25%
Gull 3 x64 XP - Critter 1.6a 64-bit_NOB 23.5 - 16.5 +16/=15/-9 58.75%
Gull 3 x64 XP - SF 270414IPx 64 SSE4.2_ 20.0 - 20.0 +8/=24/-8 50.00%
Gull 3 x64 XP - Houdini 4 Pro x64_Ct0_OK 17.0 - 23.0 +6/=22/-12 42.50%

160 games=
http://www.mediafire.com/view/dhk3d11cg ... es_2+2.pgn

Score using 4 cores= 162.0 – 158.0 = 50.62%

Score at Fixed Time Control= 159.0 – 161.0 = 49.69%
Score at Incremental Time control= 162.0 – 158.0 = 50.62%

Global Score= 321.0 – 319.0 = 50.16%

Against : Houdini 4.0 St. Ct0 (3227)= 41.87% ; Komodo TCECr (3181)= 53.12% ; SF Ipman 270414 (3281)= 45.62% ; Critter 1.6a (3104)= 60.00%

Average Estimated Elo Opponents = 3198
Estimated Elo Performance= 3199

Error bars= +/- 18 EEP

Gull 3 XP is an extremely strong engine. In my test it has been able to win all four legs against the powerful Komodo TCECr and offered a hard opposition to the best scorer SF270414 Ipman. Thanks for this precious gift, ThinkingALot!.

Regards from Barcelona.

Tom.

[Ozymandias]

Did you check if the BYO compiles give you better performance? They are really fast on machines with AVX.

[Tomcass]

Did you check if the BYO compiles give you better performance? They are really fast on machines with AVX.

Hi, Juan. I have not tried BYO compiles yet. I will explore if they work well in my no-AVX computers.

Un abrazo!

Tom.