Alex54 wrote:My concept "crush" arises from the other point of view, in terms of percentages of win and lost games, MORE THAN DOUBLE.
THAT is a crush, at this level of playing strength, where the situation of the first programs an their playing levels is supposed to be most balanced and very , very equalized.
And as far as we can see after the Superfinal, this is NOT the case.
Disregarding draws, when pondering about the outcome of a match, is just plain wrong. Just two examples: +1000 =0 -500 is "crushing", between top programs, +2 =1000 -1 isn't.
Alex54 wrote:Now his situation is rather difficult ,as it is not possible to justify the 60 dollars price of the Komodo 7, after such a disastrous result in the TCEC Superfinal, opposite to a FREE Stockfish.
I'm a K7 customer, and the only quibble I have about it is poor 6-men support. Otherwise, $60 is a steal for such a good engine.
BeyondCritics wrote:Let's look at the results. I use gratefully a small cpp program, someone else posted here earlier:
$ ./elodiff 13 6 45
Number of games: 64
Winning fraction: 0.554688
Elo difference: +38.1534
LOS: 0.945853
LOS is calculated using the methods shown by remi coloum.
Thus it is quite certain that stockfish is better, _but_ LOS does not say how much better. It could by better by one Elo point with the same confidence, LOS simply does not know that.
The estimated Elo difference is 38, gathered with 64 tournament games.
They used a special book, to expose misevaluations and to avoid drawish positions and this suceeded obviously extrenly well. _But_ you should keep in mind that the Elo difference is exaggerated due to this.
Openings chosen for there entertainment value over there chess testing value. As was done in tcec. Will always favor the weaker engines. Not the other way around as you claim. Inferior openings always help the weaker engine win or draw games. When the openings are played by both programs from each side of the board. A weaker engine can win or draw games against a superior program just because of the inferior opening that that the superior program was forced to play. Resulting in a contraction of the true elo difference.
For the Superfinal, there was an attempt to find openings that were less drawish without favoring White or Black too much. These type of positions help differentiate the engines as well as creating more entertaining games. Obviously, the selection process was not a complete success, but I think it worked well. And I do not think the TCEC results underestimate the strength difference between Stockfish and Komodo. A random selection of openings typically used in engine testing would tend to have a higher draw rate and W - L would be lower IMO.
If openings chosen for TCEC affected one game in 64 because of its entertainment value was more important then the fair testing value of the opening. And there was, then it did effect the strength difference between Stockfish and Komodo. It would only take one inferior opening to contract the results. In a 64 game match. But that is beside the point.
The main point of my reply is to rebut the notion that inferior openings some how help the superior program. And that is pure and utter nonsense.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
BeyondCritics wrote:Let's look at the results. I use gratefully a small cpp program, someone else posted here earlier:
$ ./elodiff 13 6 45
Number of games: 64
Winning fraction: 0.554688
Elo difference: +38.1534
LOS: 0.945853
LOS is calculated using the methods shown by remi coloum.
Thus it is quite certain that stockfish is better, _but_ LOS does not say how much better. It could by better by one Elo point with the same confidence, LOS simply does not know that.
The estimated Elo difference is 38, gathered with 64 tournament games.
They used a special book, to expose misevaluations and to avoid drawish positions and this suceeded obviously extrenly well. _But_ you should keep in mind that the Elo difference is exaggerated due to this.
Openings chosen for there entertainment value over there chess testing value. As was done in tcec. Will always favor the weaker engines. Not the other way around as you claim. Inferior openings always help the weaker engine win or draw games. When the openings are played by both programs from each side of the board. A weaker engine can win or draw games against a superior program just because of the inferior opening that that the superior program was forced to play. Resulting in a contraction of the true elo difference.
And that's why I let engines with own opening books play with them....
A brilliant example related to this issue is the Hiarcs chess engine regards,
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
Modern Times wrote:I regard 13 wins to 6 as comprehensively beaten. But second place is no shame, Komodo is still a superb engine, and the Komodo team can be proud of their efforts.
It's like winning 12.5-11.5, in a match decided by the first to reach 12.5.
Yes, comprehensively beaten, i.e. according to the stipulation.
Crush is too hard and adventured expression. Not only lot of draws, but 19-6 is not that big a difference. You rest just a few games from one and add them to the other and you have another draw.
Maybe still SF is superior that far, but not overwheelmingly so.
Alex54 wrote:My concept "crush" arises from the other point of view, in terms of percentages of win and lost games, MORE THAN DOUBLE.
THAT is a crush, at this level of playing strength, where the situation of the first programs an their playing levels is supposed to be most balanced and very , very equalized.
And as far as we can see after the Superfinal, this is NOT the case.
Disregarding draws, when pondering about the outcome of a match, is just plain wrong. Just two examples: +1000 =0 -500 is "crushing", between top programs, +2 =1000 -1 isn't.
Alex54 wrote:Now his situation is rather difficult ,as it is not possible to justify the 60 dollars price of the Komodo 7, after such a disastrous result in the TCEC Superfinal, opposite to a FREE Stockfish.
I'm a K7 customer, and the only quibble I have about it is poor 6-men support. Otherwise, $60 is a steal for such a good engine.
I consider 13-6 a crushing for this tournament. The conditions were known, so for an engine to win by that much is crushing in my opinion.
Many draws are inevitable, it's the wins that separate them, and SF managed 6 more wins. Multiply the results by 10 and you'd have +130 =450 -60. Easier to see the difference.
Yes, and if you multiply by 1000 the difference is greater.
What kind of logic is that?
Crushing is something like 20-1 or so.
Crushing is no draws or almost.
Crushing is what happens to me -and maybe to you- when playing top engines.
I would say that SF had a clear victory this time, not more than that.
Yes, and if you multiply by 1000 you get 64000 games.
64000 games give you small error bars and a reliable Elo rating.
That is all what counts.
That is called the "testers logic".
BTW, do you have a ERROR bar in your area around?
Go there and have a beer.
Alex54 wrote:My concept "crush" arises from the other point of view, in terms of percentages of win and lost games, MORE THAN DOUBLE.
THAT is a crush, at this level of playing strength, where the situation of the first programs an their playing levels is supposed to be most balanced and very , very equalized.
And as far as we can see after the Superfinal, this is NOT the case.
Disregarding draws, when pondering about the outcome of a match, is just plain wrong. Just two examples: +1000 =0 -500 is "crushing", between top programs, +2 =1000 -1 isn't.
Alex54 wrote:Now his situation is rather difficult ,as it is not possible to justify the 60 dollars price of the Komodo 7, after such a disastrous result in the TCEC Superfinal, opposite to a FREE Stockfish.
I'm a K7 customer, and the only quibble I have about it is poor 6-men support. Otherwise, $60 is a steal for such a good engine.
I consider 13-6 a crushing for this tournament. The conditions were known, so for an engine to win by that much is crushing in my opinion.
Many draws are inevitable, it's the wins that separate them, and SF managed 6 more wins. Multiply the results by 10 and you'd have +130 =450 -60. Easier to see the difference.
