Sim-score

[Rebel]

Collected the results of 25 NNUE engines at 100ms. The simex result can be found here (no single surprise), to check the sim-score similarity download nnue.7z , go to the nnue folder on the command line and compare each engine with each other. Example : sim-score sf13.epd FatFritz_2.epd will produce:

Code: Select all

Score comparison between 'sf13.epd' and 'fatfritz_2.epd'

dev0    dev1    dev2    dev3    dev4    dev5    RMS     SIM
2775    2113    1400    845     507     583     54.20   67.87

Please.... carefully read the introduction page, especially - Let's begin to say that the (above) data is far too less to draw conclusions, let alone final conclusions, more data is needed but the current available data is good enough for an initial discussion.

http://rebel13.nl/home/nnue.html

[matejst]

Ed, I had a look at the sim table. Do your results imply that the similarity with NNs' based evaluation is in general lower than it was before? That's the impression I get when I look at numbers.

As an aside, Nemorino's and Ethereal's results look unreal -- I guess they are. Then, could you have presented in a table the RMS data?

[Rebel]

Ed, I had a look at the sim table. Do your results imply that the similarity with NNs' based evaluation is in general lower than it was before? That's the impression I get when I look at numbers.

As an aside, Nemorino's and Ethereal's results look unreal -- I guess they are. Then, could you have presented in a table the RMS data?

Crucial to understand, SIMEX checks the similarity of moves and from the HMTL you can see how unrealistic SIMEX has become when it's about engines with NNUE. On the other hand there is SIM-SCORE, it doesn't test moves but the similarity of scores because that's what NNUE basically is, a set of scores.

I am busy with other things, for the moment no desire to create a 2 dimensional table of the data.