Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Chessqueen
Posts: 1063
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by Chessqueen » Wed Jul 15, 2020 10:35 pm

Somebody under this user name dkappe on this Forum from Rio de Janeiro decided to experiment with Komodo and NNUE and is NOT bad at all very promising with the NNUE evaluation, it just need at least 1000 more games to train it properly :roll:

Alguém com esse nome de usuário dkappe neste fórum do Rio de Janeiro decidiu experimentar o Komodo e o NNUE e NÃO é nada promissor com a avaliação do NNUE, são necessários pelo menos mais 1000 jogos para treiná-lo adequadamente :shock:

Alguien con el Nombre de usuario de dkappe decidio experimentar con Komodo 14 usanda el NET NNUE y es muy prometedor pero necesita por lo menos 1000 juegos mas de entrenamiento :roll:
dkappe wrote:
Wed Jul 15, 2020 8:08 pm
Just for fun I decided to train a NET with Komodo 14 evals at depth 8. Only 4 million positions. I was expecting something pretty weak, but it’s not half bad. Here with 30 threads vs sf10 (also with 30 threads). So far +3=9-0.


Last edited by Chessqueen on Wed Jul 15, 2020 10:55 pm, edited 2 times in total.

dkappe
Posts: 518
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Lizard-NNUE Experiment NOT bad with NNUE Evaluation.........

Post by dkappe » Wed Jul 15, 2020 10:47 pm

Chessqueen wrote:
Wed Jul 15, 2020 10:35 pm
Somebody under this user name dkappe on this Forum from Rio de Janeiro decided to experiment with Komodo and NNUE and is NOT bad at all very promising with the NNUE evaluation, it just need at least 1000 more games to train it properly :roll:
A number of people verified for me that I wasn’t imagining things. This was run by a friend of mine. I only wish I lived in Rio.

Chessqueen
Posts: 1063
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Lizard-NNUE Experiment NOT bad with NNUE Evaluation.........

Post by Chessqueen » Wed Jul 15, 2020 11:03 pm

dkappe wrote:
Wed Jul 15, 2020 10:47 pm
Chessqueen wrote:
Wed Jul 15, 2020 10:35 pm
Somebody under this user name dkappe on this Forum from Rio de Janeiro decided to experiment with Komodo and NNUE and is NOT bad at all very promising with the NNUE evaluation, it just need at least 1000 more games to train it properly :roll:
A number of people verified for me that I wasn’t imagining things. This was run by a friend of mine. I only wish I lived in Rio.
Ask your friend to Join this forum and provide more data about his experiment with Komodo-NNUE, he can write in Portuguese and that is fine we can translate it using https://translate.google.com/ :mrgreen:
dkappe wrote:
Wed Jul 15, 2020 8:08 pm
Just for fun I decided to train a NET with Komodo 14 evals at depth 8. Only 4 million positions. I was expecting something pretty weak, but it’s not half bad. Here with 30 threads vs sf10 (also with 30 threads). So far +3=9-0.



dkappe
Posts: 518
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Lizard-NNUE Experiment NOT bad with NNUE Evaluation.........

Post by dkappe » Wed Jul 15, 2020 11:15 pm

Chessqueen wrote:
Wed Jul 15, 2020 11:03 pm


Ask your friend to Join this forum and provide more data about his experiment with Komodo-NNUE, he can write in Portuguese and that is fine we can translate it using https://translate.google.com/ :mrgreen:
You misunderstand. I trained the net. He ran it on his machine to make sure I wasn’t imagining things.

Here another game, this time running on one of my home machines on only 1 thread.



It’s running about even with the latest sfdev, but the score, nps and pv are very different, so it’s not accidental sf eval.

Chessqueen
Posts: 1063
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Lizard-NNUE Experiment NOT bad with NNUE Evaluation.........

Post by Chessqueen » Wed Jul 15, 2020 11:54 pm

dkappe wrote:
Wed Jul 15, 2020 11:15 pm
Chessqueen wrote:
Wed Jul 15, 2020 11:03 pm


Ask your friend to Join this forum and provide more data about his experiment with Komodo-NNUE, he can write in Portuguese and that is fine we can translate it using https://translate.google.com/ :mrgreen:
You misunderstand. I trained the net. He ran it on his machine to make sure I wasn’t imagining things.

Here another game, this time running on one of my home machines on only 1 thread.



It’s running about even with the latest sfdev, but the score, nps and pv are very different, so it’s not accidental sf eval.
I thought that you used Komdo 14 with the NNUE Net, if that is the case and you did NOT used the Stockfish evaluation WHY do you call it LizardFish and NOT Lizard-NNUE :?:

dkappe
Posts: 518
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by dkappe » Thu Jul 16, 2020 12:10 am

I generated training data with Komodo 14 (and a modest amount of python). I used a recent nnue binary to train a net using that data and am running that net using that binary (as are my helpful friends). So this is an approximation of the Komodo 14 eval at depth 8 running on a sf-nnue binary. The name? Nothing serious. It’s a marriage of Komodo and stockfish — LizardFish.

I’ll train it up some more, but I have mixed feelings about distributing a stronger version. It seems almost like a theft of Komodo’s intellectual property. The same sort of cloning (and I think this is much more “cloning” than the usual name calling on this forum) could be done with any uci engine and a modest amount of cpu.

Albert Silver
Posts: 2917
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by Albert Silver » Thu Jul 16, 2020 12:54 am

dkappe wrote:
Thu Jul 16, 2020 12:10 am
I generated training data with Komodo 14 (and a modest amount of python). I used a recent nnue binary to train a net using that data and am running that net using that binary (as are my helpful friends). So this is an approximation of the Komodo 14 eval at depth 8 running on a sf-nnue binary. The name? Nothing serious. It’s a marriage of Komodo and stockfish — LizardFish.

I’ll train it up some more, but I have mixed feelings about distributing a stronger version. It seems almost like a theft of Komodo’s intellectual property. The same sort of cloning (and I think this is much more “cloning” than the usual name calling on this forum) could be done with any uci engine and a modest amount of cpu.
It really is not, since while the NN is trained from games played by Komodo, it is still an NN. If studying Kasparov's games and trying to emulate him makes me his clone then..... my dreams have all come true!! :D :D

I am the mysterious tester (this is all dkappe's work), and ran it for 35 games before calling it quits. It was 35 games only (not 1000, sorry), with 30 threads each, for roughly 30+ Million nps for SF10 and 14-15 Million nps for Lizard. I would have played a later version of SF but was told to not be too optimistic, so this was only chosen to try to keep it competitive. A case of underestimating itself if ever one was seen.

Here is the result:

Code: Select all

lizard1-sf10-11-254, Blitz 1.0min+1.0se  2020

                                              12345678901234567890123456789012345
1   SF NNUE halfkp-256 090720,x64 avx2  +113   ½½½½1½½½1½½1½½1½½½1½½½1111½01½1½½½1  23.0/35
2   Stockfish 10 64 POPCNT              -113  ½½½½0½½½0½½0½½0½½½0½½½0000½10½0½½½0   12.0/35

"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

mjlef
Posts: 1454
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by mjlef » Thu Jul 16, 2020 2:38 am

Albert Silver wrote:
Thu Jul 16, 2020 12:54 am
dkappe wrote:
Thu Jul 16, 2020 12:10 am
I generated training data with Komodo 14 (and a modest amount of python). I used a recent nnue binary to train a net using that data and am running that net using that binary (as are my helpful friends). So this is an approximation of the Komodo 14 eval at depth 8 running on a sf-nnue binary. The name? Nothing serious. It’s a marriage of Komodo and stockfish — LizardFish.

I’ll train it up some more, but I have mixed feelings about distributing a stronger version. It seems almost like a theft of Komodo’s intellectual property. The same sort of cloning (and I think this is much more “cloning” than the usual name calling on this forum) could be done with any uci engine and a modest amount of cpu.
It really is not, since while the NN is trained from games played by Komodo, it is still an NN. If studying Kasparov's games and trying to emulate him makes me his clone then..... my dreams have all come true!! :D :D

I am the mysterious tester (this is all dkappe's work), and ran it for 35 games before calling it quits. It was 35 games only (not 1000, sorry), with 30 threads each, for roughly 30+ Million nps for SF10 and 14-15 Million nps for Lizard. I would have played a later version of SF but was told to not be too optimistic, so this was only chosen to try to keep it competitive. A case of underestimating itself if ever one was seen.

Here is the result:

Code: Select all

lizard1-sf10-11-254, Blitz 1.0min+1.0se  2020

                                              12345678901234567890123456789012345
1   SF NNUE halfkp-256 090720,x64 avx2  +113   ½½½½1½½½1½½1½½1½½½1½½½1111½01½1½½½1  23.0/35
2   Stockfish 10 64 POPCNT              -113  ½½½½0½½½0½½0½½0½½½0½½½0000½10½0½½½0   12.0/35

I consider Albert a good friend, but I must disagree a bit. Training a NN to match the eval and search output of a single program seems to be to be a way to clone that program. We might not understand exactly how the NN works compared with say an assembly dump of a programs eval and search functions, but it is a direct attempt to duplicate the program. Training on many sources (programs, human games, self play) is not trying to specifically duplicate another programs search and eval, so I think that wold be allowed. Training for personal use is fine. I am just speaking of training against a program (especially a commercial engine) and then releasing the NN without permission is wrong. I assume testing groups and tournaments would agree, but I would like to hear more opinions.

This is a new world, but the old cloning rules would still apply.

Mark

Chessqueen
Posts: 1063
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by Chessqueen » Thu Jul 16, 2020 2:53 am

mjlef wrote:
Thu Jul 16, 2020 2:38 am
Albert Silver wrote:
Thu Jul 16, 2020 12:54 am
dkappe wrote:
Thu Jul 16, 2020 12:10 am
I generated training data with Komodo 14 (and a modest amount of python). I used a recent nnue binary to train a net using that data and am running that net using that binary (as are my helpful friends). So this is an approximation of the Komodo 14 eval at depth 8 running on a sf-nnue binary. The name? Nothing serious. It’s a marriage of Komodo and stockfish — LizardFish.

I’ll train it up some more, but I have mixed feelings about distributing a stronger version. It seems almost like a theft of Komodo’s intellectual property. The same sort of cloning (and I think this is much more “cloning” than the usual name calling on this forum) could be done with any uci engine and a modest amount of cpu.
It really is not, since while the NN is trained from games played by Komodo, it is still an NN. If studying Kasparov's games and trying to emulate him makes me his clone then..... my dreams have all come true!! :D :D

I am the mysterious tester (this is all dkappe's work), and ran it for 35 games before calling it quits. It was 35 games only (not 1000, sorry), with 30 threads each, for roughly 30+ Million nps for SF10 and 14-15 Million nps for Lizard. I would have played a later version of SF but was told to not be too optimistic, so this was only chosen to try to keep it competitive. A case of underestimating itself if ever one was seen.

Here is the result:

Code: Select all

lizard1-sf10-11-254, Blitz 1.0min+1.0se  2020

                                              12345678901234567890123456789012345
1   SF NNUE halfkp-256 090720,x64 avx2  +113   ½½½½1½½½1½½1½½1½½½1½½½1111½01½1½½½1  23.0/35
2   Stockfish 10 64 POPCNT              -113  ½½½½0½½½0½½0½½0½½½0½½½0000½10½0½½½0   12.0/35

I consider Albert a good friend, but I must disagree a bit. Training a NN to match the eval and search output of a single program seems to be to be a way to clone that program. We might not understand exactly how the NN works compared with say an assembly dump of a programs eval and search functions, but it is a direct attempt to duplicate the program. Training on many sources (programs, human games, self play) is not trying to specifically duplicate another programs search and eval, so I think that wold be allowed. Training for personal use is fine. I am just speaking of training against a program (especially a commercial engine) and then releasing the NN without permission is wrong. I assume testing groups and tournaments would agree, but I would like to hear more opinions.

This is a new world, but the old cloning rules would still apply.

Mark
Well as long as Larry Kaufman agree to take advantage of the NNUE NET, and the advantages of using NNUE NET benefit Komodo to the point that it becomes stronger than Komodo, that would NOT be any different than using StockfiNN to advance Stockfish search with a more efficient one :roll: :shock:

mjlef
Posts: 1454
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

Post by mjlef » Thu Jul 16, 2020 3:26 am

Chessqueen wrote:
Thu Jul 16, 2020 2:53 am
mjlef wrote:
Thu Jul 16, 2020 2:38 am
Albert Silver wrote:
Thu Jul 16, 2020 12:54 am
dkappe wrote:
Thu Jul 16, 2020 12:10 am
I generated training data with Komodo 14 (and a modest amount of python). I used a recent nnue binary to train a net using that data and am running that net using that binary (as are my helpful friends). So this is an approximation of the Komodo 14 eval at depth 8 running on a sf-nnue binary. The name? Nothing serious. It’s a marriage of Komodo and stockfish — LizardFish.

I’ll train it up some more, but I have mixed feelings about distributing a stronger version. It seems almost like a theft of Komodo’s intellectual property. The same sort of cloning (and I think this is much more “cloning” than the usual name calling on this forum) could be done with any uci engine and a modest amount of cpu.
It really is not, since while the NN is trained from games played by Komodo, it is still an NN. If studying Kasparov's games and trying to emulate him makes me his clone then..... my dreams have all come true!! :D :D

I am the mysterious tester (this is all dkappe's work), and ran it for 35 games before calling it quits. It was 35 games only (not 1000, sorry), with 30 threads each, for roughly 30+ Million nps for SF10 and 14-15 Million nps for Lizard. I would have played a later version of SF but was told to not be too optimistic, so this was only chosen to try to keep it competitive. A case of underestimating itself if ever one was seen.

Here is the result:

Code: Select all

lizard1-sf10-11-254, Blitz 1.0min+1.0se  2020

                                              12345678901234567890123456789012345
1   SF NNUE halfkp-256 090720,x64 avx2  +113   ½½½½1½½½1½½1½½1½½½1½½½1111½01½1½½½1  23.0/35
2   Stockfish 10 64 POPCNT              -113  ½½½½0½½½0½½0½½0½½½0½½½0000½10½0½½½0   12.0/35

I consider Albert a good friend, but I must disagree a bit. Training a NN to match the eval and search output of a single program seems to be to be a way to clone that program. We might not understand exactly how the NN works compared with say an assembly dump of a programs eval and search functions, but it is a direct attempt to duplicate the program. Training on many sources (programs, human games, self play) is not trying to specifically duplicate another programs search and eval, so I think that wold be allowed. Training for personal use is fine. I am just speaking of training against a program (especially a commercial engine) and then releasing the NN without permission is wrong. I assume testing groups and tournaments would agree, but I would like to hear more opinions.

This is a new world, but the old cloning rules would still apply.

Mark
Well as long as Larry Kaufman agree to take advantage of the NNUE NET, and the advantages of using NNUE NET benefit Komodo to the point that it becomes stronger than Komodo, that would NOT be any different than using StockfiNN to advance Stockfish search with a more efficient one :roll: :shock:
I.agree. it is fine to use your own program to make a better version of itself, just like in tuning a program.

Post Reply