Can the sardine! NNUE clobbers SF.

[Nay Lin Tun]

Thank you, I see list of epd files where can I find the best moves ?

[Laskos]

Thank you, I see list of epd files where can I find the best moves ?

Ah, you mean the opening test suite?

http://s000.tinyupload.com/?file_id=427 ... 6636434517

[cdani]

I don't have much endgame suites, but positionally the result seems to confirm a lesser accent on openings, unlike Lc0 zero nets.

Yes. Stockfish evaluation of openings is not specially good. So another nice try can be to mix somehow Lco evaluations to train the nets.

[Pio]

I don't have much endgame suites, but positionally the result seems to confirm a lesser accent on openings, unlike Lc0 zero nets.

Yes. Stockfish evaluation of openings is not specially good. So another nice try can be to mix somehow Lco evaluations to train the nets.

I don’t believe it matters so much what/who you have as trainer to be good in the openings. I think it is the MCTS-similar Search that mainly makes LCO good in the openings. Of course by using LCO as a trainer the nets might learn openings better but I don’t believe it will make it overall better since learning specific openings might take away some memory/generalisation of the Nets that could be of better use later in the game.

I really believe that it is the MCTS-similar way of maximising the possibilities for you that makes the difference. Alpha-Beta is a lot more sensitive to errors. One wrong leaf evaluation could be disastrous if you are unlucky with alpha-beta and in the beginning there are lots of possibilities to choose a really bad leaf position. I believe that alpha-beta is way more sensitive to big errors in the eval function than mcts-similar search.

/Pio

[Pio]

I don't have much endgame suites, but positionally the result seems to confirm a lesser accent on openings, unlike Lc0 zero nets.

Yes. Stockfish evaluation of openings is not specially good. So another nice try can be to mix somehow Lco evaluations to train the nets.

I don’t believe it matters so much what/who you have as trainer to be good in the openings. I think it is the MCTS-similar Search that mainly makes LCO good in the openings. Of course by using LCO as a trainer the nets might learn openings better but I don’t believe it will make it overall better since learning specific openings might take away some memory/generalisation of the Nets that could be of better use later in the game.

I really believe that it is the MCTS-similar way of maximising the possibilities for you that makes the difference. Alpha-Beta is a lot more sensitive to errors. One wrong leaf evaluation could be disastrous if you are unlucky with alpha-beta and in the beginning there are lots of possibilities to choose a really bad leaf position. I believe that alpha-beta is way more sensitive to big errors in the eval function than mcts-similar search.

/Pio

It would be really interesting to see how Stockfish would play the openings using LCO-nets. My guess is that it wouldn’t play the openings great.

/Pio

[cdani]

Hello.
I have done a test between SFNNUE GK 27-6 against Sergio's network. More games needed to differentiate between them.
200+1

Code: Select all

   # PLAYER              : RATING  ERROR   POINTS  PLAYED    (%)
   1 stnnuesergio2007    : 2857.8    4.7    303.5     601   50.5%
   2 stnnuegk2706        : 2854.2    4.7    297.5     601   49.5%

Games:
https://1drv.ms/u/s!Aj8wlSmr_Hq3irYEYDT ... A?e=5A1x1m

May be sergio's net is stronger due to this test:
https://discord.com/channels/4359437104 ... 6715986984

[Laskos]

Hello.
I have done a test between SFNNUE GK 27-6 against Sergio's network. More games needed to differentiate between them.
200+1

Code: Select all

   # PLAYER              : RATING  ERROR   POINTS  PLAYED    (%)
   1 stnnuesergio2007    : 2857.8    4.7    303.5     601   50.5%
   2 stnnuegk2706        : 2854.2    4.7    297.5     601   49.5%

Games:
https://1drv.ms/u/s!Aj8wlSmr_Hq3irYEYDT ... A?e=5A1x1m

May be sergio's net is stronger due to this test:
https://discord.com/channels/4359437104 ... 6715986984

I have something similar to the discord result, also LOS of 100% for Sergio 20.07 net:

Code: Select all

Games Completed = 1000 of 1000 (Avg game length = 25.058 sec)
Settings = RR/128MB/6000ms+100ms/M 1500cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\3M_08_10.epd(395)
Time = 6436 sec elapsed, 0 sec remaining
 1.  Sergio 20.07              	534.5/1000	303-234-463  	(L: m=12 t=0 i=0 a=222)	(D: r=249 i=94 f=36 s=4 a=80)	(tpm=172.6 d=17.55 nps=1090570)
 2.  GK                       	465.5/1000	234-303-463  	(L: m=1 t=0 i=0 a=302)	(D: r=249 i=94 f=36 s=4 a=80)	(tpm=174.1 d=17.60 nps=1097271)


   # PLAYER          : RATING  ERROR    POINTS  PLAYED     (%)   CFS(next)
   1 Sergio 20.07    :  27.88   7.81     534.5    1000    53.5     100    
   2 GK              :   0.00   7.81     465.5    1000    46.5     ---    

White advantage = 136.18 +/- 8.10
Draw rate (equal opponents) = 54.76 % +/- 1.86

Now testing Sergio 21.07.

[mehmet123]

Good tests. But i think test them against Stockfish or a strong engine give us more quality results.

[Laskos]

Hello.
I have done a test between SFNNUE GK 27-6 against Sergio's network. More games needed to differentiate between them.
200+1

Code: Select all

   # PLAYER              : RATING  ERROR   POINTS  PLAYED    (%)
   1 stnnuesergio2007    : 2857.8    4.7    303.5     601   50.5%
   2 stnnuegk2706        : 2854.2    4.7    297.5     601   49.5%

Games:
https://1drv.ms/u/s!Aj8wlSmr_Hq3irYEYDT ... A?e=5A1x1m

May be sergio's net is stronger due to this test:
https://discord.com/channels/4359437104 ... 6715986984

I have something similar to the discord result, also LOS of 100% for Sergio 20.07 net:

Code: Select all

Games Completed = 1000 of 1000 (Avg game length = 25.058 sec)
Settings = RR/128MB/6000ms+100ms/M 1500cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\3M_08_10.epd(395)
Time = 6436 sec elapsed, 0 sec remaining
 1.  Sergio 20.07              	534.5/1000	303-234-463  	(L: m=12 t=0 i=0 a=222)	(D: r=249 i=94 f=36 s=4 a=80)	(tpm=172.6 d=17.55 nps=1090570)
 2.  GK                       	465.5/1000	234-303-463  	(L: m=1 t=0 i=0 a=302)	(D: r=249 i=94 f=36 s=4 a=80)	(tpm=174.1 d=17.60 nps=1097271)


   # PLAYER          : RATING  ERROR    POINTS  PLAYED     (%)   CFS(next)
   1 Sergio 20.07    :  27.88   7.81     534.5    1000    53.5     100    
   2 GK              :   0.00   7.81     465.5    1000    46.5     ---    

White advantage = 136.18 +/- 8.10
Draw rate (equal opponents) = 54.76 % +/- 1.86

Now testing Sergio 21.07.

Sergio 21.07 seems even stronger than Sergio 20.07, but only 200 games. Will finish the 1000 games test.

Code: Select all

Games Completed = 200 of 1000 (Avg game length = 24.551 sec)
Settings = RR/128MB/6000ms+100ms/M 1500cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\3M_08_10.epd(395)
Time = 1264 sec elapsed, 5057 sec remaining
 1.  Sergio 21.07             	115.0/200	71-41-88  	(L: m=2 t=0 i=0 a=39)	(D: r=47 i=18 f=5 s=1 a=17)	(tpm=174.4 d=17.19 nps=1068336)
 2.  GK                       	85.0/200	41-71-88  	(L: m=1 t=0 i=0 a=70)	(D: r=47 i=18 f=5 s=1 a=17)	(tpm=176.3 d=17.14 nps=1071681)

[Laskos]

Hello.
I have done a test between SFNNUE GK 27-6 against Sergio's network. More games needed to differentiate between them.
200+1

Code: Select all

   # PLAYER              : RATING  ERROR   POINTS  PLAYED    (%)
   1 stnnuesergio2007    : 2857.8    4.7    303.5     601   50.5%
   2 stnnuegk2706        : 2854.2    4.7    297.5     601   49.5%

Games:
https://1drv.ms/u/s!Aj8wlSmr_Hq3irYEYDT ... A?e=5A1x1m

May be sergio's net is stronger due to this test:
https://discord.com/channels/4359437104 ... 6715986984

I have something similar to the discord result, also LOS of 100% for Sergio 20.07 net:

Code: Select all

Games Completed = 1000 of 1000 (Avg game length = 25.058 sec)
Settings = RR/128MB/6000ms+100ms/M 1500cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\3M_08_10.epd(395)
Time = 6436 sec elapsed, 0 sec remaining
 1.  Sergio 20.07              	534.5/1000	303-234-463  	(L: m=12 t=0 i=0 a=222)	(D: r=249 i=94 f=36 s=4 a=80)	(tpm=172.6 d=17.55 nps=1090570)
 2.  GK                       	465.5/1000	234-303-463  	(L: m=1 t=0 i=0 a=302)	(D: r=249 i=94 f=36 s=4 a=80)	(tpm=174.1 d=17.60 nps=1097271)


   # PLAYER          : RATING  ERROR    POINTS  PLAYED     (%)   CFS(next)
   1 Sergio 20.07    :  27.88   7.81     534.5    1000    53.5     100    
   2 GK              :   0.00   7.81     465.5    1000    46.5     ---    

White advantage = 136.18 +/- 8.10
Draw rate (equal opponents) = 54.76 % +/- 1.86

Now testing Sergio 21.07.

Sergio 21.07 seems even stronger than Sergio 20.07, but only 200 games. Will finish the 1000 games test.

Code: Select all

Games Completed = 200 of 1000 (Avg game length = 24.551 sec)
Settings = RR/128MB/6000ms+100ms/M 1500cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\3M_08_10.epd(395)
Time = 1264 sec elapsed, 5057 sec remaining
 1.  Sergio 21.07             	115.0/200	71-41-88  	(L: m=2 t=0 i=0 a=39)	(D: r=47 i=18 f=5 s=1 a=17)	(tpm=174.4 d=17.19 nps=1068336)
 2.  GK                       	85.0/200	41-71-88  	(L: m=1 t=0 i=0 a=70)	(D: r=47 i=18 f=5 s=1 a=17)	(tpm=176.3 d=17.14 nps=1071681)

Yup, even stronger, but within error margins of 20.07 net after 1000 games.

Code: Select all

Games Completed = 1000 of 1000 (Avg game length = 25.043 sec)
Settings = RR/128MB/6000ms+100ms/M 1500cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\3M_08_10.epd(395)
Time = 6427 sec elapsed, 0 sec remaining
 1.  Sergio 21.07               546.0/1000	322-230-448  	(L: m=8 t=0 i=0 a=222)	(D: r=234 i=90 f=32 s=6 a=86)	(tpm=172.6 d=17.77 nps=1108332)
 2.  GK                       	454.0/1000	230-322-448  	(L: m=2 t=0 i=0 a=320)	(D: r=234 i=90 f=32 s=6 a=86)	(tpm=174.1 d=17.69 nps=1114025)


   # PLAYER          : RATING  ERROR    POINTS  PLAYED     (%)   CFS(next)
   1 Sergio 21.07    :  39.74   8.01     546.0    1000    54.6     100    
   2 GK              :   0.00   8.01     454.0    1000    45.4     ---    

White advantage = 162.21 +/- 8.32
Draw rate (equal opponents) = 57.00 % +/- 2.01