Testing Stockfish 11-03-13. 480 Games.

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: no more tests?

Post by Tomcass »

Hi, Bobby.

I don't know what other testers do. What I do is simply:

- For Stockfish development versions I respect default values.
- For other compiles allowing Large Pages I tick the box: 'Try Large Pages'.

In both cases I set the number of threads appropriate to my computers.

Regards,

Tom.
Top
User avatar
beachknight
Posts: 3533
Joined: Tue Jan 09, 2007 8:33 pm
Location: Antalya, Turkey

Re: no more tests?

Post by beachknight »

Tomcass wrote:Hello,

Here you have the latest Ipman's compile, allowing the use of Large Pages. (you only have to tick 'Tyr Large Pages'). If I am not wrong, it does include the source code:

http://www.mediafire.com/download/xe6n3 ... L+Ipman.7z

Regards,

Tom.
Thank you Tom and Ipman.

Yes, I checked the file, includes the source code.

Best,
hi, merhaba, hallo HT
Top
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: no more tests?

Post by ouachita »

set to the number of cpu
SIM, PhD, MBA, PE
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: no more tests?

Post by Tomcass »

TESTING STOCKFISH DD = 640 Games.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Stockfish DD 64 SSE4.2x6 - Houdini 4 Pro x64x6 21.5 - 18.5 +10/=23/-7 53.75%
Stockfish DD 64 SSE4.2x6 - Houdini 3 Pro x64_6 20.0 - 20.0 +8/=24/-8 50.00%
Stockfish DD 64 SSE4.2x6 - Komodo 6 64-bitNOBx6 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish DD 64 SSE4.2x6 - Stockfish 111113SL 64 SSE4.2L 21.5 - 18.5 +5/=33/-2 53.75%

Time Control= 2+2

Stockfish DD 64 SSE4.2x6 - Houdini 4 Pro x64x6 22.5 - 17.5 +10/=25/-5 56.25%
Stockfish DD 64 SSE4.2x6 - Houdini 3 Pro x64_6 23.0 - 17.0 +11/=24/-5 57.50%
Stockfish DD 64 SSE4.2x6 - Komodo 6 64-bitNOBx6 24.0 - 16.0 +11/=26/-3 60.00%
Stockfish DD 64 SSE4.2x6 - Stockfish 111113SL 64 SSE4.2L 23.0 - 17.0 +9/=28/-3 57.50%

320 Games = http://www.mediafire.com/download/b1mykws89dm8l61/
Score using 6 cores= 177.0 – 143.0 = 55.31%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockfish DD 64 SSE4.2x - Houdini 4 Pro x64x4 20.0 - 20.0 +11/=18/-11 50.00%
Stockfish DD 64 SSE4.2x - Houdini 3 Pro x64_4 22.5 - 17.5 +14/=17/-9 56.25%
Stockfish DD 64 SSE4.2x - Komodo 6 64-bitx4_NOB 18.0 - 22.0 +8/=20/-12 45.00%
Stockfish DD 64 SSE4.2x - Stockfish 111113SL 64 SSE4.2I 18.0 - 22.0 +3/=30/-7 45.00%

Time Control = 2+2

Stockfish DD 64 SSE4.2x - Houdini 4 Pro x64x4 21.5 - 18.5 +13/=17/-10 53.75%
Stockfish DD 64 SSE4.2x - Houdini 3 Pro x64_4 21.0 - 19.0 +12/=18/-10 52.50%
Stockfish DD 64 SSE4.2x - Komodo 6 64-bitx4_NOB 23.5 - 16.5 +13/=21/-6 58.75%
Stockfish DD 64 SSE4.2x - Stockfish 111113SL 64 SSE4.2I 18.5 - 21.5 +4/=29/-7 46.25%

320 games= http://www.mediafire.com/view/2c5vg7j4p ... 0games.pgn
Score using 4 cores = 163.0 – 157.0 = 50.94%

Fixed = 163.0 – 157.0 = 50.94%
Incremental = 177.0 – 143.0 = 55.31%

SUMMARY:

Global Score: 340.0 – 300.0 = 53.12%

Stockfish DD 64 SSE4.2x6 - Houdini 4 Pro x64x6 53.44%
Stockfish DD 64 SSE4.2x6 - Houdini 3 Pro x64_6 54.06%
Stockfish DD 64 SSE4.2x6 - Komodo 6 64-bitNOBx6 54.37%
Stockfish DD 64 SSE4.2x6 - Stockfish 111113SL 64 SSE4.2L 50.62%

Average Elo Opponents = 3190.
Estimated ELO Performance = 3212

-----------------------------------------------

After the latest tests, this is the Estimated Elo Performance I have found in my computers with 4 and 6 cores under my testing conditions:

1.- Stockfish 111113SL Large Pages Ipman = 3220
2.- Stockfish DD = 3212
3.- Houdini 4.0 Pro B = 3210
4.- Stockfish 191113 64 SSE4.2x = 3204
5.- Houdini 3.0 Pro = 3172
6.- Komodo 6.0 = 3166
7.- Amitis 16-10-2013 = 3.160
8.- Stockfish 4= 3119
9.- Critter 1.6a = 3093
10.- Deep Rybka 4.1 = 3015

Regards from Barcelona,

Tom.
Top
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: no more tests?

Post by ouachita »

Nice; Looks fairly easy to do but probably isnt.

When Komodo 1142 is released, I plan to run an engine v engine test involving:

Komodo 1142
Stockfish DD
Houdini Pro 4B

i7-2687W @3.10ghz
16 real cores
Ponder: on or off?
GuI: DF13
Book: Perfect 2012C
No TB or RTB
Hash Size: ?

Time Controls: 90 + 0

Might need your help setting this up correctly.
SIM, PhD, MBA, PE
Top
Vinvin
Posts: 5228
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: no more tests?

Post by Vinvin »

ouachita wrote:Nice; Looks fairly easy to do but probably isnt.

When Komodo 1142 is released, I plan to run an engine v engine test involving:

Komodo 1142
Stockfish DD
Houdini Pro 4B

i7-2687W @3.10ghz
16 real cores
Ponder: on or off?
GuI: DF13
Book: Perfect 2012C
No TB or RTB
Hash Size: ?

Time Controls: 90 + 0

Might need your help setting this up correctly.
Great !
16 threads (ponder off) or 8 threads (ponder on), I'd prefer 1st choice.
Hash Size : 1/4 of memory for each engine.
TB : Syzygy 6 pieces (Stockfish Syzygy have to be used)
Book : may be there's a better book. Length ? 4, 5 or 6 moves ?
Time Controls : I'd prefer a small increment, 5 or 10 seconds.
Nice tournament again ! :twisted:
Top
User avatar
beachknight
Posts: 3533
Joined: Tue Jan 09, 2007 8:33 pm
Location: Antalya, Turkey

Re: no more tests?

Post by beachknight »

Tomcass wrote:TESTING STOCKFISH DD = 640 Games.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Stockfish DD 64 SSE4.2x6 - Houdini 4 Pro x64x6 21.5 - 18.5 +10/=23/-7 53.75%
Stockfish DD 64 SSE4.2x6 - Houdini 3 Pro x64_6 20.0 - 20.0 +8/=24/-8 50.00%
Stockfish DD 64 SSE4.2x6 - Komodo 6 64-bitNOBx6 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish DD 64 SSE4.2x6 - Stockfish 111113SL 64 SSE4.2L 21.5 - 18.5 +5/=33/-2 53.75%

Time Control= 2+2

Stockfish DD 64 SSE4.2x6 - Houdini 4 Pro x64x6 22.5 - 17.5 +10/=25/-5 56.25%
Stockfish DD 64 SSE4.2x6 - Houdini 3 Pro x64_6 23.0 - 17.0 +11/=24/-5 57.50%
Stockfish DD 64 SSE4.2x6 - Komodo 6 64-bitNOBx6 24.0 - 16.0 +11/=26/-3 60.00%
Stockfish DD 64 SSE4.2x6 - Stockfish 111113SL 64 SSE4.2L 23.0 - 17.0 +9/=28/-3 57.50%

320 Games = http://www.mediafire.com/download/b1mykws89dm8l61/
Score using 6 cores= 177.0 – 143.0 = 55.31%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control = 4+0

Stockfish DD 64 SSE4.2x - Houdini 4 Pro x64x4 20.0 - 20.0 +11/=18/-11 50.00%
Stockfish DD 64 SSE4.2x - Houdini 3 Pro x64_4 22.5 - 17.5 +14/=17/-9 56.25%
Stockfish DD 64 SSE4.2x - Komodo 6 64-bitx4_NOB 18.0 - 22.0 +8/=20/-12 45.00%
Stockfish DD 64 SSE4.2x - Stockfish 111113SL 64 SSE4.2I 18.0 - 22.0 +3/=30/-7 45.00%

Time Control = 2+2

Stockfish DD 64 SSE4.2x - Houdini 4 Pro x64x4 21.5 - 18.5 +13/=17/-10 53.75%
Stockfish DD 64 SSE4.2x - Houdini 3 Pro x64_4 21.0 - 19.0 +12/=18/-10 52.50%
Stockfish DD 64 SSE4.2x - Komodo 6 64-bitx4_NOB 23.5 - 16.5 +13/=21/-6 58.75%
Stockfish DD 64 SSE4.2x - Stockfish 111113SL 64 SSE4.2I 18.5 - 21.5 +4/=29/-7 46.25%

320 games= http://www.mediafire.com/view/2c5vg7j4p ... 0games.pgn
Score using 4 cores = 163.0 – 157.0 = 50.94%

Fixed = 163.0 – 157.0 = 50.94%
Incremental = 177.0 – 143.0 = 55.31%

SUMMARY:

Global Score: 340.0 – 300.0 = 53.12%

Stockfish DD 64 SSE4.2x6 - Houdini 4 Pro x64x6 53.44%
Stockfish DD 64 SSE4.2x6 - Houdini 3 Pro x64_6 54.06%
Stockfish DD 64 SSE4.2x6 - Komodo 6 64-bitNOBx6 54.37%
Stockfish DD 64 SSE4.2x6 - Stockfish 111113SL 64 SSE4.2L 50.62%

Average Elo Opponents = 3190.
Estimated ELO Performance = 3212

-----------------------------------------------

After the latest tests, this is the Estimated Elo Performance I have found in my computers with 4 and 6 cores under my testing conditions:

1.- Stockfish 111113SL Large Pages Ipman = 3220
2.- Stockfish DD = 3212
3.- Houdini 4.0 Pro B = 3210
4.- Stockfish 191113 64 SSE4.2x = 3204
5.- Houdini 3.0 Pro = 3172
6.- Komodo 6.0 = 3166
7.- Amitis 16-10-2013 = 3.160
8.- Stockfish 4= 3119
9.- Critter 1.6a = 3093
10.- Deep Rybka 4.1 = 3015

Regards from Barcelona,

Tom.
Thank you for the 4cpu and 6cpu tests.
Wooowww. SFDD version is almost 100 points better than SF4.
Time is up for SF6!
Best,
hi, merhaba, hallo HT
Top
ouachita
Posts: 454
Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: no more tests?

Post by ouachita »

I agree SF DD has closed the gap at these TC, but we need LTC (30, 60, 90), which I will try soon.
SIM, PhD, MBA, PE
Top
fauzi
Posts: 61
Joined: Wed Nov 20, 2013 10:42 am

Re: no more tests?

Post by fauzi »

the first link is put to private we can't download
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: no more tests?

Post by Tomcass »

fauzi wrote:the first link is put to private we can't download
Sorry!. Try this.

http://www.mediafire.com/view/b1mykws89 ... 0games.pgn

Tom.
Top

Return to “Computer Chess Club: Tournaments and Matches”