Rating Lists and double standards for clones

[Madeleine Birchfield]

Why is it that most rating lists for computer chess, CCRL, CGET, FGRL, et cetera, list Houdini, Fire, and Fat Fritz as their own separate engines, while listing Shashchess and Sugar as Stockfish derivatives and outright refusing to test the likes of Corchess and Crystal? Regarding GPL violating engines, why does Houdini and Fat Fritz get to stay on the lists while Eman is excluded? This shows really big double standards in their treatment of Stockfish clones and derivatives, and tells people that they could get away with passing off a Stockfish clone as their own unique creation so long as they sell it on Chessbase first.

The rating lists need to have one standard for all Stockfish derivatives and ckones: either 1. label them as independent engines, like Houdini and Fat Fritz, 2. Label them as members of the Stockfish family, like Shashchess and Sugar, or 3. outright not put them on the list, like Corchess or Crystal.

[gonzochess75]

Very much a duplicate thread to Andrew's, but it is a good question. An even *better* question is do these various rating lists actually provide any value to the chess software development community? If they don't provide any thing of value, then they can be safely ignored. If they are *just* used to promote crap/unethical products they should be mocked, shunned and clowned in direct proportion to the harm they do in scamming chess enthusiasts who don't follow chess software development.

[Madeleine Birchfield]

Very much a duplicate thread to Andrew's, but it is a good question.

Andrew's thread is about boycotting Chessbase products, while this is about double standards in the rating lists of including clones on the lists. Two separate topics.

My preferred solution is listing Houdini and Fat Fritz as Stockfish derivatives, but if the rating lists include Houdini and Fat Fritz as separate engines, then they should do the same for Eman, Bluefish, and AlphaSubZero. Otherwise, there are double standards involved.

[hgm]

Developers of engines typically love to see their engine on the rating lists. And knowing the ratings of other free engines is also of great value for them. Because it enables them to easily get an objective assessment of the strength of their own engine themselves, by playing it against opponents of known rating. So the rating lists fulfill a very useful function.

The testing teams working on the creation of such lists put an incredible amount of effort in it, completely voluntarily. They don't have to account for their actions to anyone. That means they can also apply their own standards for what they want to test and what not.

You call the standard they have chosen to currently apply 'double', I would call it 'pragmatic'. There is a lot of interest in commercial engines, because these are actively marketed, and many people will buy them. Those people would be interested in knowing where the engine stands, and should be interested to know that even before they decide to buy it. For free Stockfish clones that seems much less important: people can download those any time they want, and try those themselves. This is not possible for commercial engines.

[Madeleine Birchfield]

You call the standard they have chosen to currently apply 'double', I would call it 'pragmatic'. There is a lot of interest in commercial engines, because these are actively marketed, and many people will buy them. Those people would be interested in knowing where the engine stands, and should be interested to know that even before they decide to buy it. For free Stockfish clones that seems much less important: people can download those any time they want, and try those themselves. This is not possible for commercial engines.

Even if they restrict it to commercial Stockfish derivatives, Eman is also a commercial Stockfish derivative, yet it is not on the ratings list, while Houdini and Fat Fritz are. And if the rating lists restrict it to commercial Stockfish derivatives, then they should get rid of Fire, which is a Stockfish derivative that is freely available to downloal, but none of the rating lists are willing to get rid of Fire. This is what I mean by double standards.

[hgm]

I agree that it is an inconsistency, but it is not obvious that this is the result of any 'standards'. It might just be historical accident, or personal preference of one member of the testing team. It doesn't require much imagination to understand why they don't want to test each and every Stockfish derivative (many only differing from the real Stockfish by having edited the name...). It seems also logical they would not be very keen on testing an engine they would have to pay for, when they know it is hardly different from what they can download for free.

[gonzochess75]

Developers of engines typically love to see their engine on the rating lists. And knowing the ratings of other free engines is also of great value for them. Because it enables them to easily get an objective assessment of the strength of their own engine themselves, by playing it against opponents of known rating. So the rating lists fulfill a very useful function.

I've never cared about seeing any of my engines on rating lists for anything other than curiosity. Doubt that top engine devs care all that much either for SF or Leela. For them, I don't think there is much value at all. So they are free to be ignored.

[hgm]

You must be kidding. Top-engine authors go through an enormous hassle with contempt to exploit a systematic error in the rating measurements to gain a few Elo over the competition.

But even if you were right: who cares what the authors of top-engines think? The number of top engines is completely negligible.

[gonzochess75]

You must be kidding. Top-engine authors go through an enormous hassle with contempt to exploit a systematic error in the rating measurements to gain a few Elo over the competition.

But even if you were right: who cares what the authors of top-engines think? The number of top engines is completely negligible.

Noted. Thanks for making clear how much/little TalkChess mods care for the concerns or opinions of top engine devs or their projects.

[carldaman]

Some of the derivatives/forks like Bluefish, SugaR and ShashChess are legal and do bring value in the form of an enhanced style of play. I don't mind seeing them listed on rating lists.