Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Tony P.
Posts: 141
Joined: Sun Jan 22, 2017 7:30 pm
Location: Russia

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by Tony P. » Thu Jul 16, 2020 6:47 pm

I don't find Komodo an indispensable source of positional knowledge. Its big disadvantage is that it's a black box, as reverse engineering would be clearly illegal. My intuition says that an ensemble of white box engines, weaker but giving access to entire hashtables, e.g. of Arasan and Cheng, would be a satisfactory, if not superior, expert to initialize the learning with.

Chessqueen
Posts: 1063
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by Chessqueen » Thu Jul 16, 2020 8:14 pm

Milos wrote:
Thu Jul 16, 2020 3:46 pm
Raphexon wrote:
Thu Jul 16, 2020 3:29 pm
Daniel Shawul wrote:
Thu Jul 16, 2020 2:57 pm
Many AB engines use Texel tuning with data scored by Stockfish at depth=8. Remember the popular quiet.epd and the 3 million
fast SF games generated by Alvaro for the purpose of eval tuning ?
I do not consider this to be a clone nor lizard-nnue that mimics Komodo at depth=8.
Calling it a clone would be akin to calling a net trained on grandmaster games to be a human clone. :lol:
Or a painting created by an artist a clone of paint tubes :lol:.
All this could only benefit Larry Kaufman at the long run, since people will still have to purchase Komodo if they do NOT have it already before they can start using NNUE, as a matter of fact just like in the case of ChessMaster the sale of ChessMaster increased when they were creating different personalities like Queen ++ etc....since people love to be involved in projects of creativities or experiment to make anything better or different. It is NOT like he is giving LizardFish1 for Free to the Public on a certain Website, this is more like advertising for Komodo Saying hey all of you with Komodo 14 look what you can do, you can create a Hybrid cross between Komodo + Stockfish using a NNUE Net.

PS: I would like to know what the outcome of LizardFish1 is in comparison to Komodo 14 out of lets say 20 games at 10 Min + 10 Sec increment ?

dkappe
Posts: 518
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by dkappe » Thu Jul 16, 2020 11:26 pm

Oiy. What a colossal f*ck up. Before the final network save, the nnue executable loaded the nn.bin from eval and saved it in evalsave/final. The previous checkpoint was the network trained on Komodo data. While strong, it’s nowhere near Komodo or sf. The network in the eval/nn.bin? gk20200720, a very strong net trained on stockfish data. It’s byte for byte identical.

My apologies for the confusion. It was an unintentional mistake.

Tony P.
Posts: 141
Joined: Sun Jan 22, 2017 7:30 pm
Location: Russia

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by Tony P. » Thu Jul 16, 2020 11:40 pm

Phew, all is good that ends well :lol:

Chessqueen
Posts: 1063
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Lizard-NNUE Experiment NOT bad with NNUE Evaluation.........

Post by Chessqueen » Fri Jul 17, 2020 3:25 am

dkappe wrote:
Wed Jul 15, 2020 11:15 pm
Chessqueen wrote:
Wed Jul 15, 2020 11:03 pm


Ask your friend to Join this forum and provide more data about his experiment with Komodo-NNUE, he can write in Portuguese and that is fine we can translate it using https://translate.google.com/ :mrgreen:
You misunderstand. I trained the net. He ran it on his machine to make sure I wasn’t imagining things.

Here another game, this time running on one of my home machines on only 1 thread.



It’s running about even with the latest sfdev, but the score, nps and pv are very different, so it’s not accidental sf eval.
Can You list step by step how you combined the stockfish with Komodo Hybrid so other people who purchased and will purchase Komodo 14X can use the NET as well in the future, and where is the Net located available to be downloaded?

lkaufman
Posts: 4324
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Lizard-NNUE Experiment NOT bad with NNUE Evaluation.........

Post by lkaufman » Fri Jul 17, 2020 4:19 am

Chessqueen wrote:
Fri Jul 17, 2020 3:25 am
dkappe wrote:
Wed Jul 15, 2020 11:15 pm
Chessqueen wrote:
Wed Jul 15, 2020 11:03 pm


Ask your friend to Join this forum and provide more data about his experiment with Komodo-NNUE, he can write in Portuguese and that is fine we can translate it using https://translate.google.com/ :mrgreen:
You misunderstand. I trained the net. He ran it on his machine to make sure I wasn’t imagining things.

Here another game, this time running on one of my home machines on only 1 thread.



It’s running about even with the latest sfdev, but the score, nps and pv are very different, so it’s not accidental sf eval.
Can You list step by step how you combined the stockfish with Komodo Hybrid so other people who purchased and will purchase Komodo 14X can use the NET as well in the future, and where is the Net located available to be downloaded?
So far this and all other NNUE nets, whether trained on Komodo or Stockfish games, only run with the Stockfish search and software. We hope to change this in the near future, but no promise yet! I think you are confusing training on Komodo games with running inside the Komodo engine; those are two separate things, though it makes sense to combine them. If we did that, it wouldn't be Komodo + Stockfish, it would be Komodo + NNUE, which is the idea of the Japanese shogi author, not a Stockfish idea. Stockfish would have nothing to do with it in that case.
Komodo rules!

dkappe
Posts: 518
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by dkappe » Fri Jul 17, 2020 4:24 am

I was able to get the training working properly. Here is the LizardFish1 net trained on 6 million positions. Looks like it’s ballpark Texel 1.08.


dkappe
Posts: 518
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by dkappe » Fri Jul 17, 2020 5:14 am

But it’s not that far behind.


Milos
Posts: 3832
Joined: Wed Nov 25, 2009 12:47 am

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by Milos » Fri Jul 17, 2020 1:22 pm

Considering this:
dkappe wrote:
Thu Jul 16, 2020 11:26 pm
Oiy. What a colossal f*ck up. Before the final network save, the nnue executable loaded the nn.bin from eval and saved it in evalsave/final. The previous checkpoint was the network trained on Komodo data. While strong, it’s nowhere near Komodo or sf. The network in the eval/nn.bin? gk20200720, a very strong net trained on stockfish data. It’s byte for byte identical.

My apologies for the confusion. It was an unintentional mistake.
I really have to laugh at this:
Rebel wrote:
Thu Jul 16, 2020 10:57 am
dkappe wrote:
Thu Jul 16, 2020 3:58 am
If the combined search/net doesn’t play like Komodo but like something else, then it’s not a clone but some new thing. Finally some use for Rebel’s punitive clone detector! :lol: :lol:
Well, maybe it's a good idea to investigate what we are talking about, without the punitive, as Mark said NN's are a whole different world. A quick test with some adviced nets. Using a random epd of 10,000 positions at depth=1.

Code: Select all

Positions 10000 Leele NN-64 SF-NN SF11  T40-1 T60-4
Leelentein-14   ----- 73.30 43.24 38.92 77.35 73.30
NN-64194        73.30 ----- 43.17 38.83 72.31 75.97
SF-NN           43.24 43.17 ----- 45.72 42.43 42.83
SF11            38.92 38.83 45.72 ----- 38.11 38.27
T40-1541        77.35 72.31 42.43 38.11 ----- 71.45
T60-4300        73.30 75.97 42.83 38.27 71.45 -----

 1.         Engine Leelentein-14 and T40-1541 show a similarity of 77.35%
 2.         Engine NN-64194 and T60-4300 show a similarity of 75.97%
 3.         Engine Leelentein-14 and NN-64194 show a similarity of 73.30%
 4.         Engine Leelentein-14 and T60-4300 show a similarity of 73.30%
 5.         Engine NN-64194 and T40-1541 show a similarity of 72.31%
 6.         Engine T40-1541 and T60-4300 show a similarity of 71.45%
Quite some high similarities between the nets except the new approach with the 30Mb NN.BIN net from SF-NN version 0.1

Comments?
It's a wonderful demonstration of confirmation bias and also how useless and pointless are most of these "similarity tests"...

Milos
Posts: 3832
Joined: Wed Nov 25, 2009 12:47 am

Re: Lizard-NNUE Experiment NOT bad with NNUE Evaluation.........

Post by Milos » Fri Jul 17, 2020 1:28 pm

lkaufman wrote:
Fri Jul 17, 2020 4:19 am
So far this and all other NNUE nets, whether trained on Komodo or Stockfish games, only run with the Stockfish search and software. We hope to change this in the near future, but no promise yet! I think you are confusing training on Komodo games with running inside the Komodo engine; those are two separate things, though it makes sense to combine them. If we did that, it wouldn't be Komodo + Stockfish, it would be Komodo + NNUE, which is the idea of the Japanese shogi author, not a Stockfish idea. Stockfish would have nothing to do with it in that case.
Considering that all NNUE code is open source, how are you planning integrating open source code into commercial product?
Will you make Komodo search code also available as requested per GPL license?
Or you are just gonna claim you rewrote all the code and that there is no GPL code left in so called "Houdart" way? ;)

Post Reply