Minic version 3

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Minic version 3

Post by Rebel »

xr_a_y wrote: Fri Jun 04, 2021 6:49 pm
Modern Times wrote: Thu May 20, 2021 3:26 pm I've played 500 chess960 games so far, and it is unfortunately -95 Elo to version 3.06. I'll check the engine set-up, but I don't think there is any issue there.
Minic 3.08 (https://github.com/tryingsomestuff/Mini ... s/tag/3.08) is released with a new embeded net named "negligible nystagmus". This net looks even with "naive nostalgia" (the net embeded in Minic3.07) at classic chess but is much stronger at FRC.

Binaries are here : https://github.com/tryingsomestuff/Mini ... ter/Minic3


Please note that it was reported to me that, for Minic3.07, on znver1 architecture (in this case it was a Ryzen 9) the "slylake" build (so including pext/bmi2 instructions) seems faster. This is currently under investigation.
1200 games in progress.

http://rebel13.nl/a/grl.htm
90% of coding is debugging, the other 10% is writing bugs.
Wolfgang
Posts: 894
Joined: Sat May 13, 2006 1:08 am

Re: Minic version 3

Post by Wolfgang »

xr_a_y wrote: Fri Jun 04, 2021 6:49 pm ...
Minic 3.08 (https://github.com/tryingsomestuff/Mini ... s/tag/3.08) is released with a new embeded net named "negligible nystagmus". This net looks even with "naive nostalgia" (the net embeded in Minic3.07) at classic chess but is much stronger at FRC.
Binaries are here : https://github.com/tryingsomestuff/Mini ... ter/Minic3
...
First results @ our 40/4 list here: https://cegt.forumieren.com/t1510-testi ... algia#2941
Test run with 3.07 was started before you published 3.08. Fortunately I do not play FRC... :)

Nice improvement to 3.02NaNi, +111
Best
Wolfgang
CEGT-Team
www.cegt.net
www.cegt.forumieren.com
User avatar
xr_a_y
Posts: 1871
Joined: Sat Nov 25, 2017 2:28 pm
Location: France

Re: Minic version 3

Post by xr_a_y »

Thanks for those tests !
User avatar
xr_a_y
Posts: 1871
Joined: Sat Nov 25, 2017 2:28 pm
Location: France

Re: Minic version 3

Post by xr_a_y »

Here is a test with all Minic nets again (more info about the nets here : https://github.com/tryingsomestuff/NNUE-Nets)

Code: Select all

Rank Name                                 Elo     +/-   Games   Score    Draw 
   1 minic_2.53_nn-97f742aaefcd           273      17    1488   82.8%   26.3% 
   2 minic_2.53_napping_nexus             180      15    1488   73.8%   33.5% 
   3 minic_2.53_nascent_nutrient           71      14    1488   60.1%   38.0% 
   4 minic_3.02_nettling_nemesis           26      14    1488   53.8%   36.0% 
   5 minic_3.08                            24      14    1488   53.4%   41.4% 
   6 minic_3.07                             3      13    1488   50.4%   41.6% 
   7 minic_3.06_nocturnal_nadir           -31      14    1487   45.5%   40.9% 
   8 minic_3.05_nibbled_nutshell          -34      14    1487   45.2%   36.4% 
   9 minic_3.04_noisy_notch               -65      14    1487   40.8%   40.2% 
  10 minic_3.03_niggling_nymph           -110      15    1487   34.7%   34.9% 
  11 minic_3.02_narcotized_nightshift    -120      15    1488   33.3%   34.8% 
  12 minic_3.01_nefarious_nucleus        -211      17    1488   22.9%   25.7% 
  
So clearly there is two blocks, the 3 MinicNNUE2.53 nets and a lower ratings Minic3 nets.

"nettling nemesis", is a net based on SF data, and is still the strongest (unofficial) Minic 3 net.
Good news it the very good progress between "nefarious nucleus" (that played TCECS20) and "naive nostalgia" (embeded in Minic 3.07) that is playing TCEC S21 currently. This a a more than 200Elo progress.

On Minic3 NNUE architecture, most recent net "negligible nystagmus" is playing nearly at the same level as "nettling nemesis" (based on SF data), this probably means that not much progress can now be done without changing seriously the net.

The "nibbled nutshell" net is such a net with a different topology (much smaller one) and is performing good at short TC but not a longer ones.

Finally the halfpk SF architecture and code used in Minic 2.53 is still quite far away from current Minic3 status, with some 250Elo gap between current Minic3.08 and MinicNNUE2.53 using the best SV net (back in september 2020).

So let's go back to work ...
Wolfgang
Posts: 894
Joined: Sat May 13, 2006 1:08 am

Re: Minic version 3

Post by Wolfgang »

Wolfgang wrote: Sat Jun 05, 2021 5:42 pm ...

First results @ our 40/4 list here: https://cegt.forumieren.com/t1510-testi ... algia#2941
Test run with 3.07 was started before you published 3.08. Fortunately I do not play FRC... :)

Nice improvement to 3.02NaNi, +111
1600 games played, still +108 :)
https://cegt.forumieren.com/t1510-testi ... enostalgia
Best
Wolfgang
CEGT-Team
www.cegt.net
www.cegt.forumieren.com
User avatar
xr_a_y
Posts: 1871
Joined: Sat Nov 25, 2017 2:28 pm
Location: France

Re: Minic version 3

Post by xr_a_y »

Thanks, good news.
Modern Times
Posts: 3546
Joined: Thu Jun 07, 2012 11:02 pm

Re: Minic version 3

Post by Modern Times »

Oh I forgot, the FRC result was put up yesterday or the day before.

http://computerchess.org.uk/ccrl/404FRC/index.html
User avatar
xr_a_y
Posts: 1871
Joined: Sat Nov 25, 2017 2:28 pm
Location: France

Re: Minic version 3

Post by xr_a_y »

Modern Times wrote: Tue Jun 08, 2021 8:54 pm Oh I forgot, the FRC result was put up yesterday or the day before.

http://computerchess.org.uk/ccrl/404FRC/index.html
Oh sweet, thanks a lot, so the net used with Minic3.08 (negligible nystagmus) is both good at FRC and standard chess !
User avatar
xr_a_y
Posts: 1871
Joined: Sat Nov 25, 2017 2:28 pm
Location: France

Re: Minic version 3

Post by xr_a_y »

I have made the experiment, during the last weeks, of using the now famous T74 LC0 data (https://lczero.org/blog/2021/06/the-imp ... open-data/) available in binpack format (https://drive.google.com/drive/folders/ ... U4sNhMHZU-) to generate a net for Minic.
I recently thought Minic has reached its maximum with its current NNUE topology because "Negligible Nystagmus" (last Minic net based on pure Minic data) is playing as strong as "Nettling Nemesis" (a Minic net based on SF data).
I was soooooo wrong : probably mainly because "Nettling Nemesis" was train a while ago and that my trainer is now "stronger" and also maybe because those T74 data are a little better as the SF ones I used for some reason.

Anyway, this experiment leads to an unofficial Minic net, named "Nifty Nebulosity" that plays around 90Elo strong than "Negligible Nystagmus" at standard chess (didn't try FRC yet). This net is "unofficial", meaning that is shall not be used in rating lists to replace Minic 3.08 and it won't be packed as default net in any Minic version. This net is available here (https://github.com/tryingsomestuff/NNUE-Nets) and can be tested (just for fun) by changing the default one in Minic 3.08.

Let's go back to work with some fresh Minic data now :D
User avatar
xr_a_y
Posts: 1871
Joined: Sat Nov 25, 2017 2:28 pm
Location: France

Re: Minic version 3

Post by xr_a_y »

I recently tried to better tune Minic's Elo limitation that was really off. Next release will be better on this topic, here is a comparison with Stockfish one

Code: Select all

Rank Name                          Elo     +/-   Games   Score    Draw 
   1 stockfish2800                 425      33    1177   92.0%    5.6% 
   2 minic_dev_2800                341      28    1175   87.7%    5.9% 
   3 stockfish2400                 252      24    1176   81.0%    5.9% 
   4 minic_dev_2400                217      23    1176   77.7%    7.4% 
   5 minic_dev_2100                 89      20    1177   62.5%    4.8% 
   6 stockfish2100                  52      20    1176   57.4%    3.5% 
   7 minic_dev_1800                -31      20    1177   45.6%    2.8% 
   8 stockfish1800                 -79      20    1176   38.9%    1.7% 
   9 stockfish1500                -247      25    1176   19.4%    2.0% 
  10 minic_dev_1500               -282      26    1177   16.5%    1.9% 
  11 stockfish1350                -330      29    1176   13.0%    1.7% 
  12 minic_dev_1350               -416      33    1177    8.4%    4.3%