Ethereal's 66.64% and RubiChess's 64.42% similarity with Stockfish 12

[willmorton]

I've read in the Prodeo forum the Ethereal (66.64%) and RubiChess (64.42%) similarity (1s test) with Stockfish 12. Rebel says that in his opinion the high similarity is due to the playing strength but RubiChess is about 350/400 points weaker than SF 12. what am I missing here? what's the result for Komodo Dragon?

[AndrewGrant]

Probably not missing anything. Simtests are sometimes a good metric to say something might be up, sometimes not. A while back I pushed a patch which started pruning root nodes at depth 1. As a result, Ethereal's similarity tanked from 60%+ to ~30%. Not because Ethereal changed greatly, but because the sole metric that the Simtest was comparing was depth 1 search results. I would question any high similarity between engines, and take further inspection. And at the same time I would not see a low similarity is proof of a lack of cloning.

Its interesting to note the few clones that have been proven lately have had high sim scores with the cloned engine. Namely Fire has very high sim scores against Stockfish's earlier versions. Additionally, Houdini had moderate sim scores with both Stockfish and Komodo. Komodo authors claim that some of Houdini's eval features were reverse engineered from Komodo. But maybe this is a red herring; humans see patterns if they want to I suppose.

I don't have the data on hand, but are there simtests comparing early Fire versions with Gull and Roc? Would be interesting to see if the pattern holds with more examples. As an aside: Were those sims using NNUE for all 3 engines? Or maybe for Rubichess? I imagine NNUE would have high similarity at low depths,

[Guenther]

I've read in the Prodeo forum the Ethereal (66.64%) and RubiChess (64.42%) similarity (1s test) with Stockfish 12. Rebel says that in his opinion the high similarity is due to the playing strength but RubiChess is about 350/400 points weaker than SF 12. what am I missing here? what's the result for Komodo Dragon?

You are missing the time control for the simtest. Ed seems a bit bored and frustrated about NNUE non-similarities...
As nothing helped so far, he tried testing at much higher tc the simtest was meant for.
(higher tc means closer to same moves, if programs have reached a certain strengh and programs are much stronger then 10 years ago, the arrival of the simtest - much stronger hardware multiplies this effect)

[Rebel]

I've read in the Prodeo forum the Ethereal (66.64%) and RubiChess (64.42%) similarity (1s test) with Stockfish 12. Rebel says that in his opinion the high similarity is due to the playing strength but RubiChess is about 350/400 points weaker than SF 12. what am I missing here? what's the result for Komodo Dragon?

You forgot to mention that Ethereal and Rubichess were tested with the NNUE of SF12. The research is about if the sim-test still makes sense with NNUE engines. And the conclusion is that there is no sense, for NNUE engines the sim-test is dead.

[RubiChess]

I've read in the Prodeo forum the Ethereal (66.64%) and RubiChess (64.42%) similarity (1s test) with Stockfish 12. Rebel says that in his opinion the high similarity is due to the playing strength but RubiChess is about 350/400 points weaker than SF 12. what am I missing here? what's the result for Komodo Dragon?

Thank you for putting RubiChess (and Ethereal) in a headline together with "Similarity" and "Stockfish" not even giving a link where you found this "information" (meanwhile I found the thread myself).
@Rebel: Thank you (without sarcasm this time) for clarification that this was NOT RubiChess but RubiChess playing a Stockfish net.

[Wilson]

You forgot to mention that Ethereal and Rubichess were tested with the NNUE of SF12. The research is about if the sim-test still makes sense with NNUE engines. And the conclusion is that there is no sense, for NNUE engines the sim-test is dead.

The guy obviously can't read but feeding different engines the same NNUE means you are giving them the same eval and thus testing their search similarity, doesn't it? Have you tested other engines with SF-NNUE?

I've found the thread as well: Nemorino 59.63%