RubiChess NNUE player implemented

[xr_a_y]

Under linux I am using full path.
Under windows I think I only tried putting the net in the executable dir and set NNUEFile to just the name of the net.

[voffka]

Also, I tried Igel, but there is no popcount version on github.

You mean this? (From https://github.com/vshcherbyna/igel/releases/tag/2.7.0)

Untitled.png

[Rebel]

Under linux I am using full path.
Under windows I think I only tried putting the net in the executable dir and set NNUEFile to just the name of the net.

Alright, I managed...

Code: Select all

Positions 10000 Minic sf-nn 
Minic 2.48      ----- 86.38 
sf-nnue         86.38 ----- 

 1.         Engine Minic 2.48 and sf-nnue show a similarity of 86.38%


Done...

[Rebel]

Also, I tried Igel, but there is no popcount version on github.

You mean this? (From https://github.com/vshcherbyna/igel/releases/tag/2.7.0)

Untitled.png

Tried, did not work, no avx support on my PC.

[chrisw]

Under linux I am using full path.
Under windows I think I only tried putting the net in the executable dir and set NNUEFile to just the name of the net.

Alright, I managed...

Code: Select all

Positions 10000 Minic sf-nn 
Minic 2.48      ----- 86.38 
sf-nnue         86.38 ----- 

 1.         Engine Minic 2.48 and sf-nnue show a similarity of 86.38%


Done...

I don't follow Minic so have no idea its relationship to SF-NNUE, but it is worth pointing out for sake of reducing misunderstandings that the SimTest does NOT measure eval(sf-nnue) against eval(minic), it measures whether or not they (tend to) select the same best move. So what counts is not any absolute eval output, but that the relative standing of one move (the selected best) is greater than any of the other moves.
The SimTest best moves are generated at d=1, but it's apparent that most engines are doing some sort of search even at depth 1, and those searches affect move order and effectively add some noise to the results over and above the eval.
When we did the SimTest across a wide range of engines, we got a range of Similarity percentages, and the cutoff figures 65% or whatever were based on two observations.
1. That known similar programs (eg close releases of the same engine) gave those sorts of high scores and above.
2. High scores between assumed different engines were relatively rare.

The other caveat is that the cutoff figures were all based on AB engine behaviours. It's not clear if they hold in the same way for NNs.

Anyway, I guess what this thread is about, is whether or not the NN paradigm in itself produces similar engine behaviour and thus means we are generating a lot of very similar products differing mostly in 'name'. Same question could probably be asked about Texel tuned engines.

[xr_a_y]

So Minic standard eval is 36% similar to SF and NNUE 86% ! wow

[xr_a_y]

Just please, do not use "Minic" as name when a net is given, please display MinicNNUE.

One might think here that standard Minic is 86% similar with SF which is of course not the case.

[Rebel]

So Minic standard eval is 36% similar to SF and NNUE 86% ! wow

Yep.

[Rebel]

Under linux I am using full path.
Under windows I think I only tried putting the net in the executable dir and set NNUEFile to just the name of the net.

Alright, I managed...

Code: Select all

Positions 10000 Minic sf-nn 
Minic 2.48      ----- 86.38 
sf-nnue         86.38 ----- 

 1.         Engine Minic 2.48 and sf-nnue show a similarity of 86.38%


Done...

I don't follow Minic so have no idea its relationship to SF-NNUE, but it is worth pointing out for sake of reducing misunderstandings that the SimTest does NOT measure eval(sf-nnue) against eval(minic), it measures whether or not they (tend to) select the same best move. So what counts is not any absolute eval output, but that the relative standing of one move (the selected best) is greater than any of the other moves.
The SimTest best moves are generated at d=1, but it's apparent that most engines are doing some sort of search even at depth 1, and those searches affect move order and effectively add some noise to the results over and above the eval.
When we did the SimTest across a wide range of engines, we got a range of Similarity percentages, and the cutoff figures 65% or whatever were based on two observations.
1. That known similar programs (eg close releases of the same engine) gave those sorts of high scores and above.
2. High scores between assumed different engines were relatively rare.

The other caveat is that the cutoff figures were all based on AB engine behaviours. It's not clear if they hold in the same way for NNs.

Anyway, I guess what this thread is about, is whether or not the NN paradigm in itself produces similar engine behaviour and thus means we are generating a lot of very similar products differing mostly in 'name'. Same question could probably be asked about Texel tuned engines.

What the result addresses is the concern of Andrew and Andreas.

Secondly NN engines show similar high percentages, tested that 1.5-2 years ago.

[Rebel]

Just please, do not use "Minic" as name when a net is given, please display MinicNNUE.

I will add MinicNNUE to the OKE test, curious if it will give the early opening a boast as well.