Testmatch SF 141130 (TCEC) - SF 141112

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
fastgm
Posts: 492
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Testmatch SF 141130 (TCEC) - SF 141112

Post by fastgm » Thu Dec 11, 2014 9:09 pm

Dual AMD Opteron 6376, 2.3 GHz

Cutechess, 1 Core, 128 MB Hash, no ponder, 8moves_v3.pgn, 120s+1.2s

Code: Select all

    Program                Elo    +   -   Games   Score   Av.Op.  Draws
-------------------------------------------------------------------------------------------
  1 Stockfish 141130 T1  : 3000   4   4   6002    50.0 %   3000   74.4 %  (+771,=4464,-767)
  2 Stockfish 141112 T1  : 3000   4   4   6002    50.0 %   3000   74.4 %  (+767,=4464,-771)
Total equal after 6000 games.

Carlos777
Posts: 645
Joined: Sun Dec 13, 2009 5:09 pm

Re: Testmatch SF 141130 (TCEC) - SF 141112

Post by Carlos777 » Thu Dec 11, 2014 11:52 pm

fastgm wrote:Dual AMD Opteron 6376, 2.3 GHz

Cutechess, 1 Core, 128 MB Hash, no ponder, 8moves_v3.pgn, 120s+1.2s

Code: Select all

    Program                Elo    +   -   Games   Score   Av.Op.  Draws
-------------------------------------------------------------------------------------------
  1 Stockfish 141130 T1  : 3000   4   4   6002    50.0 %   3000   74.4 %  (+771,=4464,-767)
  2 Stockfish 141112 T1  : 3000   4   4   6002    50.0 %   3000   74.4 %  (+767,=4464,-771)
Total equal after 6000 games.
I think it would be more interesting if both versions play against a set of strong engines like Komodo, Gull and Houdini.

IMO this kind of testing used by the Stockfish team is wrong, I mean testing Stockfish with a previous version of itself.

JJJ
Posts: 1295
Joined: Sat Apr 19, 2014 11:47 am

Re: Testmatch SF 141130 (TCEC) - SF 141112

Post by JJJ » Fri Dec 12, 2014 10:22 am

Funny, I was running the almost same test. Ok, we prove they equal against each others. What about versus others engines ?

fastgm
Posts: 492
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Re: Testmatch SF 141130 (TCEC) - SF 141112

Post by fastgm » Sun Dec 14, 2014 11:04 pm

Here the results against Komodo 8, same conditions:

Code: Select all

    Program                Elo    +   -   Games   Score   Av.Op.  Draws
 --------------------------------------------------------------------------------------------
  1 Stockfish 141112 T1  : 3027   6   6   6000    57.6 %   2973   51.2 %  (+1920,=3071,-1009) 3455.5/6000
  2 Komodo 8 T1          : 2973   6   6   6000    42.4 %   3027   51.2 %  (+1009,=3071,-1920)

    Program                Elo    +   -   Games   Score   Av.Op.  Draws
 --------------------------------------------------------------------------------------------
  1 Stockfish 141130 T1  : 3026   6   6   6000    57.4 %   2974   51.4 %  (+1904,=3082,-1014) 3445.0/6000
  2 Komodo 8 T1          : 2974   6   6   6000    42.6 %   3026   51.4 %  (+1014,=3082,-1904)
Andreas

fenchel
Posts: 36
Joined: Thu Dec 04, 2014 5:01 am

Re: Testmatch SF 141130 (TCEC) - SF 141112

Post by fenchel » Mon Dec 15, 2014 2:47 am

fastgm wrote:Here the results against Komodo 8, same conditions:

Code: Select all

    Program                Elo    +   -   Games   Score   Av.Op.  Draws
 --------------------------------------------------------------------------------------------
  1 Stockfish 141112 T1  : 3027   6   6   6000    57.6 %   2973   51.2 %  (+1920,=3071,-1009) 3455.5/6000
  2 Komodo 8 T1          : 2973   6   6   6000    42.4 %   3027   51.2 %  (+1009,=3071,-1920)

    Program                Elo    +   -   Games   Score   Av.Op.  Draws
 --------------------------------------------------------------------------------------------
  1 Stockfish 141130 T1  : 3026   6   6   6000    57.4 %   2974   51.4 %  (+1904,=3082,-1014) 3445.0/6000
  2 Komodo 8 T1          : 2974   6   6   6000    42.6 %   3026   51.4 %  (+1014,=3082,-1904)
Andreas
Andreas, can you confirm that you compiled these stockfish versions yourself, and moreover compiled them both in the same way (compiler version, compiler flags, stc.)?

(If you follow SF's mailing list, you'll know why I bring this up...)

User avatar
M ANSARI
Posts: 3423
Joined: Thu Mar 16, 2006 6:10 pm

Re: Testmatch SF 141130 (TCEC) - SF 141112

Post by M ANSARI » Mon Dec 15, 2014 8:07 am

Where are you downloading your SF versions? Is it from the main site? Hard to keep up with all the SF versions and figure out which one is which and if it has TB support or not.

fastgm
Posts: 492
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Re: Testmatch SF 141130 (TCEC) - SF 141112

Post by fastgm » Mon Dec 15, 2014 8:26 am

No, that's the offical versions from http://abrok.eu/stockfish/ downloaded on 10th December.

Jouni
Posts: 2108
Joined: Wed Mar 08, 2006 7:15 pm

Re: Testmatch SF 141130 (TCEC) - SF 141112

Post by Jouni » Mon Dec 15, 2014 11:48 am

There is also test in SF forum, which shows definitely NO PROGRESS in last month! It was 178200 games for each version !! Was there bug in syzygy addition and/or compiler change maybe? Tomorrow is last day to make TCEC final compile :!:.
Jouni

User avatar
M ANSARI
Posts: 3423
Joined: Thu Mar 16, 2006 6:10 pm

Re: Testmatch SF 141130 (TCEC) - SF 141112

Post by M ANSARI » Mon Dec 15, 2014 4:54 pm

I think people have to be careful when testing different compilers as the improvement or regression could simply be because the compiler was tuned for certain hardware. It is very easy to lose track of what is going on or if there actually is an improvement if testing is done with different hardware and different compilers without specifying such differences.

JJJ
Posts: 1295
Joined: Sat Apr 19, 2014 11:47 am

Re: Testmatch SF 141130 (TCEC) - SF 141112

Post by JJJ » Mon Dec 15, 2014 7:58 pm

I think SF 141112 is still the best Stockfish so far.

Post Reply