Program Elo + - Games Score Av.Op. Draws
--------------------------------------------------------------------------------------------
1 Stockfish 141112 T1 : 3027 6 6 6000 57.6 % 2973 51.2 % (+1920,=3071,-1009) 3455.5/6000
2 Komodo 8 T1 : 2973 6 6 6000 42.4 % 3027 51.2 % (+1009,=3071,-1920)
Program Elo + - Games Score Av.Op. Draws
--------------------------------------------------------------------------------------------
1 Stockfish 141130 T1 : 3026 6 6 6000 57.4 % 2974 51.4 % (+1904,=3082,-1014) 3445.0/6000
2 Komodo 8 T1 : 2974 6 6 6000 42.6 % 3026 51.4 % (+1014,=3082,-1904)
Andreas
Andreas, can you confirm that you compiled these stockfish versions yourself, and moreover compiled them both in the same way (compiler version, compiler flags, stc.)?
(If you follow SF's mailing list, you'll know why I bring this up...)
Where are you downloading your SF versions? Is it from the main site? Hard to keep up with all the SF versions and figure out which one is which and if it has TB support or not.
There is also test in SF forum, which shows definitely NO PROGRESS in last month! It was 178200 games for each version !! Was there bug in syzygy addition and/or compiler change maybe? Tomorrow is last day to make TCEC final compile .
I think people have to be careful when testing different compilers as the improvement or regression could simply be because the compiler was tuned for certain hardware. It is very easy to lose track of what is going on or if there actually is an improvement if testing is done with different hardware and different compilers without specifying such differences.