New Stockfish 5 to be released this weekend

[fern]

another example of this kind of tester logic.
Once in a decade I win a game to a top software, BUT if I multiply that by one million I am the chess world champion, no doubt....and without, with such number, margin of error.

Of course we can also divide.
Divided my 1000 the SF victory comes to nothing.

Maths regards
Fern

[gotogo]

the real score should be 9 to 2... 4 games were give-me to both sides. that means komodo only earned 2 and stockfish 9

[NATIONAL12]

You cannot deny that Komodo won last game in brilliant fashion though.

[aturri]

the real score should be 9 to 2... 4 games were give-me to both sides. that means komodo only earned 2 and stockfish 9

Please, could you elaborate that answer? which 4 games are you talking about?

[aturri]

I regard 13 wins to 6 as comprehensively beaten. But second place is no shame, Komodo is still a superb engine, and the Komodo team can be proud of their efforts.

It's like winning 12.5-11.5, in a match decided by the first to reach 12.5.

Yes, comprehensively beaten, i.e. according to the stipulation.

Very nice point. According to that, Kasparov systematically crushed Karpov in 3 of 4 of their matches to 24 games:

1985: 13-11 (Karpov lost the last game in a must-win situation, so 12-11 up to that point)
1986: 12.5-11.5
1987: 12-12 (hum, Karpov was this time no "crushed" by Kasparov!!)
1990: 12.5-11.5

Adding the 4 matches, it makes 50-46.

I remember those days, [Irony ON] and it was like nowadays (Carlsen vs Anand rematch): why to see a rematch between Karpov and Kasparov if Karpov is going to crush Kasparov again and again by 12.5-11.5? [Irony OFF]

[jpqy]

Stockfish 5 is out!

http://abrok.eu/stockfish/

Author: Marco Costalba
Date: Sat May 31 09:16:54 2014 +0200
Timestamp: 1401520614

Stockfish 5

Stockfish bench signature is: 8732553

[gotogo]

the real score should be 9 to 2... 4 games were give-me to both sides. that means komodo only earned 2 and stockfish 9

Please, could you elaborate that answer? which 4 games are you talking about?

first set games 27 and 28,
second set 33 and 34,
third and fourth set 51, 52, 53 and 54
all these games black could not defend the opening.

[BeyondCritics]

Let's look at the results. I use gratefully a small cpp program, someone else posted here earlier:

$ ./elodiff 13 6 45
Number of games: 64
Winning fraction: 0.554688
Elo difference: +38.1534
LOS: 0.945853

LOS is calculated using the methods shown by remi coloum.
Thus it is quite certain that stockfish is better, _but_ LOS does not say how much better. It could by better by one Elo point with the same confidence, LOS simply does not know that.
The estimated Elo difference is 38, gathered with 64 tournament games.

They used a special book, to expose misevaluations and to avoid drawish positions and this suceeded obviously extrenly well. _But_ you should keep in mind that the Elo difference is exaggerated due to this.

Openings chosen for there entertainment value over there chess testing value. As was done in tcec. Will always favor the weaker engines. Not the other way around as you claim. Inferior openings always help the weaker engine win or draw games. When the openings are played by both programs from each side of the board. A weaker engine can win or draw games against a superior program just because of the inferior opening that that the superior program was forced to play. Resulting in a contraction of the true elo difference.

For the Superfinal, there was an attempt to find openings that were less drawish without favoring White or Black too much. These type of positions help differentiate the engines as well as creating more entertaining games. Obviously, the selection process was not a complete success, but I think it worked well. And I do not think the TCEC results underestimate the strength difference between Stockfish and Komodo. A random selection of openings typically used in engine testing would tend to have a higher draw rate and W - L would be lower IMO.

If openings chosen for TCEC affected one game in 64 because of its entertainment value was more important then the fair testing value of the opening. And there was, then it did effect the strength difference between Stockfish and Komodo. It would only take one inferior opening to contract the results. In a 64 game match. But that is beside the point.

The main point of my reply is to rebut the notion that inferior openings some how help the superior program. And that is pure and utter nonsense.

_What_ is your main point? Do you talk to me or what?? I have no idea what you are talking about, when you mention "inferior" openings. If you want to rebut me, you should learn to base your arguments on given facts, not on your own preconceptions of what i might have in mind. I for my part had never the notion, that "inferior" postions do help the superior program.
The pretence, that even one inferior opening can contract the result is also just a capricious exaggeration of your part. You could say that about nearly any match in chess history.

The "inferior" openings in your picture are the result of very carefully attempts to create complex, fighting positions. They knew exactly that they wanted and completely explained it to the public:

On one hand you do not want one-sided opening positions where the same color will almost always win. On the other hand you do not want extremely draw-prone opening positions, particularly in Stages when very evenly-matched, high-quality programs are playing. What you are looking for are hundreds of openings that are consistently in-between.

So again, nobody believes that the inferior positions helps the stronger side.

[BKTormey]

If worthwhile, I think that the effect of Contempt given the opening selections could be interesting.

For example, if settings were adjusted for Komodo specific to Stockfish, such that Komodo would play for a draw given an inferior opening position, I'm not sure that we can say that that strategy worked given the Superfinal results. More work may be needed for Komodo to work harder for a draw. Or perhaps such an effort would be useless. Perhaps the strategy depends on or assumes the use of its own book; it would not expect the inferior position in the first place, and so planning for it would be largely unproductive except for pre-fabricated openings.

Non-the-less, I think that TCEC accomplishes its aims wonderfully, and the only change I would make is to find a more independent way of choosing the openings, rather than using the candidates themselves for the opening analysis and selection.

[aturri]

Aha, thank you, it would be interesting to analise such opening positions.