Igel vs Igel NNUE

[Orhan.Ozturk]

Dear All,

Recently, i started to have some interest to NNUE evaluation.
Before understanding the concept and integrating it to my own engine, as a starting point i wanted to observe its performance.
I took Igel, i compiled it to two versions, the version "Igel" that is having the classical evaluation function
and "IgelNN" that is having "NNUE" evaluation.

I set up a tournament (40moves/2min), including these two IGEL versions and some other sparring partners,
my own engine YNODE has also participated.

The results are not as good as i expected for IgelNN.
IgelNN did really bad. I checked some of the games, "IgelNN" was doing really dummy mistakes.

Standings :
Program Elo + - Games Score Av.Op. Draws
----------------------------------------------------------------------------------------------------
1 Igel : 211 103 96 69 81.2 % -43 8.7 %
2 Zurichess : 20 79 78 68 53.7 % -6 13.2 %
3 C_0116 : 19 79 78 69 53.6 % -6 11.6 %
4 MrBob : -33 77 78 68 44.9 % 3 16.2 %
5 C_0121 : -45 75 76 69 42.8 % 6 18.8 %
6 IgelNN : -172 87 91 69 23.9 % 29 13.0 %

Igel Stats
6 IgelNN : -172 69 (+ 12,= 9,- 48), 23.9 %
----------------------------------------------------------------------------------------------------
MrBob : 14 (+ 3,= 4,- 7), 35.7 %
Igel : 13 (+ 0,= 0,- 13), 0.0 %
Zurichess : 14 (+ 5,= 1,- 8), 39.3 %
C_0116 : 14 (+ 3,= 1,- 10), 25.0 %
C_0121 : 14 (+ 1,= 3,- 10), 17.9 %

Igel Version : 3.0.0
The network file was "ign-1-d593efbd" which is the most recent one.
Tournament Tool : cutechess-cli

I could not get what i am doing wrong here. I was expecting IgelNN to have a clear win over others, but it did happen so, and it came as the last.
Any comment, idea on this case ?

[Fabio Gobbato]

I don't know what is the problem with Igel but if you want to see how nnue perform against classical evaluation you can use stockfish that has an UCI option to disable nnue.

[Scally]

Hi all,

I just ran a quick tournament between Igel v3.0.10 and IgelNN v3.0.10

IgelNN trounced Igel by 288 elo:

# PLAYER : RATING POINTS PLAYED (%)
1 igelNN-v3.0.10 : 3622.9 429.0 512 84
2 igel-v3.0.10 : 3335.0 83.0 512 16


Cheers,

Al.