FCP Tourney-KI started January 02nd, 2022

[Frank Quisinsky]

Tomorrow I will looking what Ferdinand made with his new PGNHelper tool.
forum3/viewtopic.php?f=2&t=80118

I hope of more silence with the a entry I added on my site:
I changed Rules, Note 2:

Rules, Note 2:
Please be informed that several programmers used NN (Neural Network) files based on Stockfish!
For more information have a look on the sites by the programmers!
Since March 09th, 2022 you can detect these group of engines with my hint "NNSf".

NNSf = Neural Network Stockfish
NNRe = Neural Network Rebel

Controversial is Fire 8 MC.3 NN.
In my humble opinion the programmer developed an own Neural Network.

June 16th, 2022
A sentence programmers not like!

The Fire programmer Norman Schmidt wrote in TalkChess:
- June 15th, 2022
- June 16th, 2022

"Fire includes 'Raptor', a top reinforcement learning network trained by Sergio Vieri
https://www.comp.nus.edu.sg/~sergio-v/nnue/"

It is not comparable to a copy and pasted SF eval function, it plays uniquely.

Some other available engines (for the moment not tested by me) used NNSf.
Keeping an overview is difficult.

[connor_mcmonigle]

...
Perhaps unsurprisingly, such engines' evaluations are consequently highly correlated (in the statistical sense) with Stockfish's evaluations relative to engines which don't copy previous versions of Stockfish's evaluation function verbatim. I'll share some data demonstrating this phenomenon later today.

Sorry for hijacking Frank's thread, but following up on this, the results are pretty definitive:

Code: Select all

corr(berserk, berserk) = 1.0
corr(berserk, stockfish_12) = 0.5356738825368556
corr(berserk, fire) = 0.562181605852763
corr(stockfish_12, berserk) = 0.5356738825368556
corr(stockfish_12, stockfish_12) = 0.9999999999999997
corr(stockfish_12, fire) = 0.9273272205303431
corr(fire, berserk) = 0.562181605852763
corr(fire, stockfish_12) = 0.9273272205303431
corr(fire, fire) = 1.0

Here's the repo: https://github.com/connormcmonigle/engine-similarity. I can provide exact binaries and epds if you wish to exactly reproduce the provided results.

[connor_mcmonigle]

...
It is not comparable to a copy and pasted SF eval function

However, that's exactly what it is.

[Rebel]

It is not comparable to a copy and pasted SF eval function, it plays uniquely.

https://tests.stockfishchess.org/nns?page=10

1. Download net - nn-82215d0fd0df.nnue
2. Do a file compare with nn.bin from Norman
3. They are 100% equal.

nn-82215d0fd0df.nnue = Stockfish 12 release.

That it plays differently is because of a different search.

[kranium]

It is not comparable to a copy and pasted SF eval function, it plays uniquely.

https://tests.stockfishchess.org/nns?page=10

1. Download net - nn-82215d0fd0df.nnue
2. Do a file compare with nn.bin from Norman
3. They are 100% equal.

nn-82215d0fd0df.nnue = Stockfish 12 release.

That it plays differently is because of a different search.

Yes, Connors python script measures the exact score given each position, not move choice...

From his data we can see that comparing the exact same net scores a perfect 1.0
corr(berserk, berserk) = 1.0
corr(stockfish_12, stockfish_12) = 0.9999999999999997

corr(fire, stockfish_12) = 0.9273272205303431

Therefore, I believe Raptor = a previous net before the release version SF 12
That's why it's not perfect match with SF12

I can't verify the exact version, Sergio's private post-grad space on the Singapore University server
(which contained every net he developed) no longer exists.
https://www.comp.nus.edu.sg/~sergio-v/nnue/

[Rebel]

Your nn.bin is a perfect match with SF12 nn-82215d0fd0df.nnue

Just type UCI.

[kranium]

Your nn.bin is a perfect match with SF12 nn-82215d0fd0df.nnue

Just type UCI.

nn.bin? that was the nnue from an earlier version (Fire 8 NN.1),
That explains the 92% instead of 100%

Connor very likely tested raptor.bin from 8.NN.3...

[connor_mcmonigle]

Your nn.bin is a perfect match with SF12 nn-82215d0fd0df.nnue

Just type UCI.

If that's the case, I don't understand the 92% instead of 100%

nn.bin was the nnue from an earlier version (Fire 8 NN.1 I think), have you tried raptor.bin from 8.NN.3?

The network is actually an exact match which I didn't realize when I first ran this correlation test. Fire is currently using SF 12's network (raptor.bin == SF 12 network). Those correlations coefficients were computed using low depth search results. Presumably the slight difference originates from the search not being quite identical.

I used this network for Fire: https://github.com/FireFather/fire-NN/b ... raptor.bin

[Frank Quisinsky]

I am outside ...
powerless ...

To a topic engine-programming = Long time project without project manager.
Programmers should working hand in hand without defamation.

Engine programming started for many years!
More as 1000 persons gave time and his own contribution over generations!
A long time project with a lot of milestones.

All the work the persons do is a present for the future, for the still running project ... a project never ended.
We are a part of it.

I like more other discussions and not the narrow-view inside such a long-time-project.

I think my examples were bad!

I am outside ...
powerless ...

And now season 583 in programmers war!
In the middle Ed Schröder we know him from example: Season 211 ... Rybka / Fruit topic!

[Modern Times]

Your nn.bin is a perfect match with SF12 nn-82215d0fd0df.nnue

Just type UCI.

I'm not aware of the full history, so correct me if I'm wrong. The net was developed by Sergio and submitted to the Stockfish project for consideration, and was rejected. It is not exclusive to Stockfish, is not Stockfish property and was not "copied" by Norman. It was some 12+ months later simply picked up "off the shelf" by Norman for use in Fire. Is that all correct ? Is there something illegal or immoral about that ? Just because some authors think that the only acceptable approach is for network to be created from scratch by engine authors does not mean that they should try and impose those views on others.