Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation.........

[Tony P.]

I don't find Komodo an indispensable source of positional knowledge. Its big disadvantage is that it's a black box, as reverse engineering would be clearly illegal. My intuition says that an ensemble of white box engines, weaker but giving access to entire hashtables, e.g. of Arasan and Cheng, would be a satisfactory, if not superior, expert to initialize the learning with.

[Chessqueen]

Many AB engines use Texel tuning with data scored by Stockfish at depth=8. Remember the popular quiet.epd and the 3 million
fast SF games generated by Alvaro for the purpose of eval tuning ?
I do not consider this to be a clone nor lizard-nnue that mimics Komodo at depth=8.

Calling it a clone would be akin to calling a net trained on grandmaster games to be a human clone.

Or a painting created by an artist a clone of paint tubes .

All this could only benefit Larry Kaufman at the long run, since people will still have to purchase Komodo if they do NOT have it already before they can start using NNUE, as a matter of fact just like in the case of ChessMaster the sale of ChessMaster increased when they were creating different personalities like Queen ++ etc....since people love to be involved in projects of creativities or experiment to make anything better or different. It is NOT like he is giving LizardFish1 for Free to the Public on a certain Website, this is more like advertising for Komodo Saying hey all of you with Komodo 14 look what you can do, you can create a Hybrid cross between Komodo + Stockfish using a NNUE Net.

PS: I would like to know what the outcome of LizardFish1 is in comparison to Komodo 14 out of lets say 20 games at 10 Min + 10 Sec increment ?

[dkappe]

Oiy. What a colossal f*ck up. Before the final network save, the nnue executable loaded the nn.bin from eval and saved it in evalsave/final. The previous checkpoint was the network trained on Komodo data. While strong, it’s nowhere near Komodo or sf. The network in the eval/nn.bin? gk20200720, a very strong net trained on stockfish data. It’s byte for byte identical.

My apologies for the confusion. It was an unintentional mistake.

[Tony P.]

Phew, all is good that ends well

[Chessqueen]

Ask your friend to Join this forum and provide more data about his experiment with Komodo-NNUE, he can write in Portuguese and that is fine we can translate it using https://translate.google.com/

You misunderstand. I trained the net. He ran it on his machine to make sure I wasn’t imagining things.

Here another game, this time running on one of my home machines on only 1 thread.

[pgn]
[Event "Match"]
[Site "Chicago"]
[Date "2020.07.15"]
[Round "12"]
[White "LizardFish1"]
[Black "sfdev150720"]
[Result "1-0"]
[ECO "B28"]
[GameDuration "00:03:44"]
[GameEndTime "2020-07-15T17:27:56.345 CDT"]
[GameStartTime "2020-07-15T17:24:12.179 CDT"]
[Opening "Sicilian"]
[PlyCount "131"]
[TimeControl "60+1"]
[Variation "O'Kelly Variation"]

1. e4 {book} c5 {book} 2. Nf3 {book} a6 {book} 3. c3 {book} e6 {book}
4. d4 {+0.44/19 2.0s} d5 {-0.29/21 4.3s} 5. e5 {+0.55/20 2.1s}
Nc6 {+0.05/18 0.76s} 6. Bd3 {+0.46/19 2.4s} Nge7 {0.00/20 3.2s}
7. O-O {+0.55/17 1.2s} Nf5 {-0.01/20 0.91s} 8. dxc5 {+0.47/20 3.5s}
Bxc5 {-0.43/21 4.8s} 9. Re1 {+0.38/18 1.6s} O-O {-0.33/19 4.1s}
10. Nbd2 {+0.33/17 1.5s} h6 {-0.47/24 6.5s} 11. b4 {+0.55/21 5.8s}
Ba7 {-0.13/17 0.75s} 12. Bb2 {+0.52/18 1.1s} Bd7 {-0.29/22 5.5s}
13. a3 {+0.17/25 17s} b5 {-0.01/24 4.0s} 14. Bb1 {+0.34/19 3.2s}
Nh4 {0.00/19 1.8s} 15. Qc2 {+0.47/18 1.0s} g6 {-0.01/22 4.4s}
16. a4 {+0.47/22 5.1s} bxa4 {-0.17/24 7.3s} 17. Nxh4 {+0.35/22 0.94s}
Qxh4 {-0.66/24 1.8s} 18. Nf3 {+0.78/20 0.99s} Qh5 {-0.32/22 1.1s}
19. Rxa4 {+0.95/20 1.2s} Ne7 {-0.57/24 2.6s} 20. Ra3 {+0.70/21 1.8s}
Bb5 {-0.51/22 1.7s} 21. Bc1 {+0.54/24 9.7s} Bb6 {-0.44/24 5.2s}
22. Bf4 {+0.79/17 0.79s} Rfc8 {-0.37/22 3.0s} 23. Qd2 {+0.56/21 1.9s}
g5 {-0.31/20 1.1s} 24. Be3 {+0.63/22 1.1s} Bxe3 {-0.51/23 1.4s}
25. Rxe3 {+0.50/22 1.9s} g4 {-0.28/21 1.3s} 26. Nd4 {+0.37/22 1.4s}
Qg5 {-0.31/24 2.9s} 27. h4 {+0.55/21 0.96s} Qxh4 {0.00/22 0.74s}
28. Rg3 {+1.50/21 1.2s} Qg5 {0.00/27 1.5s} 29. f4 {+0.40/23 3.2s}
Qh4 {-0.25/21 1.4s} 30. Qf2 {+0.52/23 1.3s} Ng6 {-0.67/26 7.5s}
31. Bxg6 {+1.36/22 4.3s} fxg6 {-1.26/23 1.2s} 32. Nxe6 {+1.37/19 0.58s}
h5 {-1.22/22 2.0s} 33. Ra1 {+1.34/20 2.3s} Rc4 {-1.24/19 1.0s}
34. Nd4 {+1.35/18 0.55s} Rf8 {-1.25/19 0.65s} 35. e6 {+1.68/17 0.77s}
Rc7 {-0.38/20 2.8s} 36. Re1 {+1.90/21 1.7s} Re7 {-1.63/19 0.62s}
37. f5 {+1.98/17 0.52s} Bc4 {-1.96/20 0.74s} 38. Qf4 {+2.21/20 2.8s}
Bd3 {-0.64/20 1.4s} 39. Re5 {+2.74/18 0.47s} Be4 {-1.03/18 0.24s}
40. c4 {+4.09/18 0.63s} Rb7 {-3.66/21 2.6s} 41. cxd5 {+4.17/17 0.58s}
Bxd5 {-3.59/18 0.16s} 42. Rxd5 {+3.81/17 0.98s} Rxb4 {-2.37/16 0.26s}
43. Kf2 {+4.37/16 1.1s} Rb2+ {-5.27/17 3.1s} 44. Ne2 {+5.12/16 0.78s}
Qe7 {-4.59/17 0.48s} 45. Qh6 {+5.04/18 1.0s} Qg7 {-4.67/18 0.38s}
46. Qxg7+ {+5.76/18 0.77s} Kxg7 {-4.67/9 0s} 47. Re3 {+5.88/19 0.69s}
gxf5 {-5.84/19 1.9s} 48. e7 {+6.13/17 0.71s} Re8 {-3.97/15 0.27s}
49. Kg3 {+6.45/20 1.2s} Rb5 {-4.13/17 0.55s} 50. Rd7 {+6.66/17 0.75s}
Rc5 {-6.41/21 3.3s} 51. Nd4 {+8.06/20 0.97s} Rb5 {-8.74/18 1.5s}
52. Kf4 {+10.34/19 0.79s} Rb4 {-8.67/16 0.34s} 53. Kg5 {+11.86/20 0.75s}
Kf7 {-16.94/18 1.6s} 54. Nxf5 {+12.45/17 1.3s} Rb5 {-9.63/17 0.61s}
55. Rd6 {+12.86/16 0.72s} Rg8+ {-6.56/16 0.21s} 56. Kf4 {+12.73/16 1.7s}
Re8 {-23.46/17 1.3s} 57. Rd8 {+13.24/15 0.76s} Kf6 {-6.73/16 1.2s}
58. Rxe8 {+M17/26 0.76s} Rxf5+ {-M16/25 0.41s} 59. Kg3 {+M15/36 0.72s}
h4+ {-M14/28 0.22s} 60. Kxh4 {+M13/35 0.71s} Re5 {-M12/41 0.28s}
61. Rxe5 {+M11/51 0.74s} Kxe5 {-M10/59 0.34s} 62. Rd8 {+M9/80 0.77s}
Kf6 {-M8/201 0.39s} 63. e8=Q {+M7/245 0.69s} g3 {-M6/245 0.16s}
64. Rd6+ {+M5/245 0.13s} Kg7 {-M4/245 0.008s} 65. Qe7+ {+M3/245 0.012s}
Kh8 {-M2/245 0.003s} 66. Rd8# {+M1/245 0.008s, White mates} 1-0
[/pgn]

It’s running about even with the latest sfdev, but the score, nps and pv are very different, so it’s not accidental sf eval.

Can You list step by step how you combined the stockfish with Komodo Hybrid so other people who purchased and will purchase Komodo 14X can use the NET as well in the future, and where is the Net located available to be downloaded?

[lkaufman]

Ask your friend to Join this forum and provide more data about his experiment with Komodo-NNUE, he can write in Portuguese and that is fine we can translate it using https://translate.google.com/

You misunderstand. I trained the net. He ran it on his machine to make sure I wasn’t imagining things.

Here another game, this time running on one of my home machines on only 1 thread.

[pgn]
[Event "Match"]
[Site "Chicago"]
[Date "2020.07.15"]
[Round "12"]
[White "LizardFish1"]
[Black "sfdev150720"]
[Result "1-0"]
[ECO "B28"]
[GameDuration "00:03:44"]
[GameEndTime "2020-07-15T17:27:56.345 CDT"]
[GameStartTime "2020-07-15T17:24:12.179 CDT"]
[Opening "Sicilian"]
[PlyCount "131"]
[TimeControl "60+1"]
[Variation "O'Kelly Variation"]

1. e4 {book} c5 {book} 2. Nf3 {book} a6 {book} 3. c3 {book} e6 {book}
4. d4 {+0.44/19 2.0s} d5 {-0.29/21 4.3s} 5. e5 {+0.55/20 2.1s}
Nc6 {+0.05/18 0.76s} 6. Bd3 {+0.46/19 2.4s} Nge7 {0.00/20 3.2s}
7. O-O {+0.55/17 1.2s} Nf5 {-0.01/20 0.91s} 8. dxc5 {+0.47/20 3.5s}
Bxc5 {-0.43/21 4.8s} 9. Re1 {+0.38/18 1.6s} O-O {-0.33/19 4.1s}
10. Nbd2 {+0.33/17 1.5s} h6 {-0.47/24 6.5s} 11. b4 {+0.55/21 5.8s}
Ba7 {-0.13/17 0.75s} 12. Bb2 {+0.52/18 1.1s} Bd7 {-0.29/22 5.5s}
13. a3 {+0.17/25 17s} b5 {-0.01/24 4.0s} 14. Bb1 {+0.34/19 3.2s}
Nh4 {0.00/19 1.8s} 15. Qc2 {+0.47/18 1.0s} g6 {-0.01/22 4.4s}
16. a4 {+0.47/22 5.1s} bxa4 {-0.17/24 7.3s} 17. Nxh4 {+0.35/22 0.94s}
Qxh4 {-0.66/24 1.8s} 18. Nf3 {+0.78/20 0.99s} Qh5 {-0.32/22 1.1s}
19. Rxa4 {+0.95/20 1.2s} Ne7 {-0.57/24 2.6s} 20. Ra3 {+0.70/21 1.8s}
Bb5 {-0.51/22 1.7s} 21. Bc1 {+0.54/24 9.7s} Bb6 {-0.44/24 5.2s}
22. Bf4 {+0.79/17 0.79s} Rfc8 {-0.37/22 3.0s} 23. Qd2 {+0.56/21 1.9s}
g5 {-0.31/20 1.1s} 24. Be3 {+0.63/22 1.1s} Bxe3 {-0.51/23 1.4s}
25. Rxe3 {+0.50/22 1.9s} g4 {-0.28/21 1.3s} 26. Nd4 {+0.37/22 1.4s}
Qg5 {-0.31/24 2.9s} 27. h4 {+0.55/21 0.96s} Qxh4 {0.00/22 0.74s}
28. Rg3 {+1.50/21 1.2s} Qg5 {0.00/27 1.5s} 29. f4 {+0.40/23 3.2s}
Qh4 {-0.25/21 1.4s} 30. Qf2 {+0.52/23 1.3s} Ng6 {-0.67/26 7.5s}
31. Bxg6 {+1.36/22 4.3s} fxg6 {-1.26/23 1.2s} 32. Nxe6 {+1.37/19 0.58s}
h5 {-1.22/22 2.0s} 33. Ra1 {+1.34/20 2.3s} Rc4 {-1.24/19 1.0s}
34. Nd4 {+1.35/18 0.55s} Rf8 {-1.25/19 0.65s} 35. e6 {+1.68/17 0.77s}
Rc7 {-0.38/20 2.8s} 36. Re1 {+1.90/21 1.7s} Re7 {-1.63/19 0.62s}
37. f5 {+1.98/17 0.52s} Bc4 {-1.96/20 0.74s} 38. Qf4 {+2.21/20 2.8s}
Bd3 {-0.64/20 1.4s} 39. Re5 {+2.74/18 0.47s} Be4 {-1.03/18 0.24s}
40. c4 {+4.09/18 0.63s} Rb7 {-3.66/21 2.6s} 41. cxd5 {+4.17/17 0.58s}
Bxd5 {-3.59/18 0.16s} 42. Rxd5 {+3.81/17 0.98s} Rxb4 {-2.37/16 0.26s}
43. Kf2 {+4.37/16 1.1s} Rb2+ {-5.27/17 3.1s} 44. Ne2 {+5.12/16 0.78s}
Qe7 {-4.59/17 0.48s} 45. Qh6 {+5.04/18 1.0s} Qg7 {-4.67/18 0.38s}
46. Qxg7+ {+5.76/18 0.77s} Kxg7 {-4.67/9 0s} 47. Re3 {+5.88/19 0.69s}
gxf5 {-5.84/19 1.9s} 48. e7 {+6.13/17 0.71s} Re8 {-3.97/15 0.27s}
49. Kg3 {+6.45/20 1.2s} Rb5 {-4.13/17 0.55s} 50. Rd7 {+6.66/17 0.75s}
Rc5 {-6.41/21 3.3s} 51. Nd4 {+8.06/20 0.97s} Rb5 {-8.74/18 1.5s}
52. Kf4 {+10.34/19 0.79s} Rb4 {-8.67/16 0.34s} 53. Kg5 {+11.86/20 0.75s}
Kf7 {-16.94/18 1.6s} 54. Nxf5 {+12.45/17 1.3s} Rb5 {-9.63/17 0.61s}
55. Rd6 {+12.86/16 0.72s} Rg8+ {-6.56/16 0.21s} 56. Kf4 {+12.73/16 1.7s}
Re8 {-23.46/17 1.3s} 57. Rd8 {+13.24/15 0.76s} Kf6 {-6.73/16 1.2s}
58. Rxe8 {+M17/26 0.76s} Rxf5+ {-M16/25 0.41s} 59. Kg3 {+M15/36 0.72s}
h4+ {-M14/28 0.22s} 60. Kxh4 {+M13/35 0.71s} Re5 {-M12/41 0.28s}
61. Rxe5 {+M11/51 0.74s} Kxe5 {-M10/59 0.34s} 62. Rd8 {+M9/80 0.77s}
Kf6 {-M8/201 0.39s} 63. e8=Q {+M7/245 0.69s} g3 {-M6/245 0.16s}
64. Rd6+ {+M5/245 0.13s} Kg7 {-M4/245 0.008s} 65. Qe7+ {+M3/245 0.012s}
Kh8 {-M2/245 0.003s} 66. Rd8# {+M1/245 0.008s, White mates} 1-0
[/pgn]

It’s running about even with the latest sfdev, but the score, nps and pv are very different, so it’s not accidental sf eval.

Can You list step by step how you combined the stockfish with Komodo Hybrid so other people who purchased and will purchase Komodo 14X can use the NET as well in the future, and where is the Net located available to be downloaded?

So far this and all other NNUE nets, whether trained on Komodo or Stockfish games, only run with the Stockfish search and software. We hope to change this in the near future, but no promise yet! I think you are confusing training on Komodo games with running inside the Komodo engine; those are two separate things, though it makes sense to combine them. If we did that, it wouldn't be Komodo + Stockfish, it would be Komodo + NNUE, which is the idea of the Japanese shogi author, not a Stockfish idea. Stockfish would have nothing to do with it in that case.

[dkappe]

I was able to get the training working properly. Here is the LizardFish1 net trained on 6 million positions. Looks like it’s ballpark Texel 1.08.

[pgn]
[Event "?"]
[Site "?"]
[Date "2020.07.16"]
[Round "2"]
[White "Texel"]
[Black "LizardFish1"]
[Result "0-1"]
[ECO "B23"]
[GameDuration "00:03:57"]
[GameEndTime "2020-07-16T23:03:54.179 CDT"]
[GameStartTime "2020-07-16T22:59:57.134 CDT"]
[Opening "Sicilian"]
[PlyCount "144"]
[TimeControl "60+1"]
[Variation "Closed"]

1. e4 {book} c5 {book} 2. Nc3 {book} e6 {book} 3. Nge2 {book} Nf6 {book}
4. d4 {+0.08/14 2.9s} d5 {-0.05/20 4.4s} 5. exd5 {+0.18/16 4.4s}
exd5 {-0.07/17 0.71s} 6. Be3 {+0.13/17 10s} cxd4 {-0.03/17 1.0s}
7. Nxd4 {+0.34/16 3.4s} Bb4 {-0.05/17 2.2s} 8. Bd3 {+0.38/16 3.2s}
O-O {+0.01/16 0.95s} 9. O-O {+0.29/16 1.3s} Nc6 {0.00/19 1.6s}
10. h3 {+0.28/16 3.2s} Re8 {-0.04/19 3.6s} 11. Re1 {+0.24/15 2.1s}
Ba5 {-0.04/19 2.5s} 12. Nxc6 {+0.31/14 2.5s} bxc6 {-0.06/19 1.1s}
13. Bd4 {+0.32/16 2.8s} Be6 {+0.04/20 5.1s} 14. Qf3 {+0.14/17 3.3s}
Nd7 {+0.19/17 0.95s} 15. Qh5 {0.00/15 3.9s} h6 {+0.25/18 1.7s}
16. Re3 {0.00/17 2.9s} Bb6 {+0.16/20 3.8s} 17. Ne2 {0.00/18 2.5s}
Bc7 {+0.20/19 1.7s} 18. b4 {0.00/15 2.5s} a5 {+0.09/21 2.6s}
19. c3 {0.00/15 1.6s} Bd6 {+0.05/21 2.9s} 20. a3 {+0.09/16 2.5s}
axb4 {+0.05/20 1.9s} 21. cxb4 {+0.15/15 1.6s} Qc7 {+0.07/19 1.2s}
22. a4 {+0.32/13 2.2s} Rab8 {+0.08/23 7.7s} 23. a5 {+0.02/14 2.2s}
Rxb4 {+0.29/18 0.90s} 24. a6 {0.00/15 2.2s} Rbb8 {+0.32/20 4.1s}
25. Nc3 {-0.23/14 4.1s} Nc5 {+0.49/18 3.1s} 26. Bc2 {-0.42/14 3.2s}
Ra8 {+0.44/17 1.4s} 27. Qd1 {-0.59/14 2.6s} Re7 {+0.41/21 5.9s}
28. Qe2 {-0.47/13 3.5s} Kf8 {+0.68/21 7.4s} 29. Ra2 {-0.23/15 3.3s}
Qc8 {+0.60/19 3.5s} 30. Rg3 {-0.45/13 2.1s} f6 {+0.94/17 0.72s}
31. a7 {-1.32/13 2.0s} Bf7 {+1.10/19 3.3s} 32. Re3 {-1.68/16 1.8s}
Rexa7 {+1.11/20 1.9s} 33. Rxa7 {-1.68/15 2.0s} Rxa7 {+1.22/17 0.56s}
34. Qd1 {-1.99/14 1.0s} Ra8 {+1.26/20 3.5s} 35. Re1 {-1.70/14 1.0s}
Qa6 {+1.20/16 0.47s} 36. Be3 {-1.98/13 1.0s} Ne6 {+1.24/19 2.3s}
37. Ne2 {-2.10/14 1.0s} c5 {+1.33/18 1.5s} 38. Bd3 {-2.37/14 1.0s}
Qa5 {+1.29/18 1.7s} 39. Bd2 {-2.33/14 1.0s} Qa4 {+1.40/17 0.88s}
40. Bc2 {-2.51/15 1.0s} Qa6 {+1.45/14 0.32s} 41. Bf5 {-2.67/13 1.0s}
Re8 {+1.33/17 1.1s} 42. Qc2 {-2.64/14 1.00s} Ng5 {+1.51/16 0.51s}
43. Bc1 {-2.91/13 1.0s} d4 {+1.18/19 1.7s} 44. Qd2 {-3.85/14 1.0s}
Ne4 {+1.88/17 1.5s} 45. Qd1 {-4.19/15 1.0s} Qa2 {+2.18/17 0.69s}
46. Bxe4 {-4.73/14 1.00s} Rxe4 {+2.29/17 0.65s} 47. g3 {-5.32/14 1.0s}
Qd5 {+2.55/19 2.9s} 48. Bf4 {-4.42/15 1.0s} c4 {+2.90/17 0.51s}
49. Bxd6+ {-4.52/16 1.0s} Qxd6 {+2.92/18 0.63s} 50. Rf1 {-4.72/16 1.0s}
Qe5 {+3.10/17 0.41s} 51. Nf4 {-5.20/14 1.00s} g5 {+3.48/16 0.74s}
52. Ng2 {-5.80/13 1.0s} Re2 {+3.89/15 0.59s} 53. Qb1 {-6.26/14 1.0s}
d3 {+4.21/16 0.59s} 54. Qb7 {-6.93/13 1.0s} d2 {+5.09/17 0.90s}
55. Qd7 {-7.54/13 1.00s} Kg7 {+6.05/17 0.61s} 56. Ne3 {-8.08/14 1.0s}
Rxe3 {+6.29/19 0.71s} 57. Qxd2 {-9.16/15 1.0s} Rd3 {+6.95/20 0.67s}
58. Qe1 {-10.74/14 1.0s} Qd4 {+8.84/18 0.70s} 59. Qe7 {-12.93/14 1.00s}
Rxg3+ {+12.33/18 0.91s} 60. Kh2 {-16.72/15 1.0s} Rd3 {+13.67/20 0.74s}
61. Kg2 {-20.17/14 1.0s} c3 {+16.11/18 1.1s} 62. Qe2 {-31.66/14 1.00s}
Bd5+ {+M23/28 0.75s} 63. f3 {-970.34/13 1.0s} Rd2 {+M21/30 0.70s}
64. Rf2 {-985.42/13 0.95s} Rxe2 {+M17/32 1.0s} 65. Rxe2 {-985.80/13 0.30s}
Qd2 {+M15/35 0.74s} 66. Rf2 {-985.78/11 0.76s} Qd1 {+M13/42 0.79s}
67. Rf1 {-M12/13 0.75s} c2 {+M11/49 0.83s} 68. Rxd1 {-M10/11 0.066s}
cxd1=Q {+M9/66 0.77s} 69. Kf2 {-M8/9 0.004s} Qd2+ {+M7/140 0.78s}
70. Kg3 {-M6/7 0.002s} Qe2 {+M5/245 0.12s} 71. h4 {-M4/5 0s}
Qxf3+ {+M3/245 0.009s} 72. Kh2 {-M2/3 0s} Qg2# {+M1/245 0.008s, Black mates} 0-1
[/pgn]

[dkappe]

But it’s not that far behind.

[pgn]
[Event "?"]
[Site "?"]
[Date "2020.07.16"]
[Round "3"]
[White "Komodo13"]
[Black "LizardFish1"]
[Result "0-1"]
[ECO "D20"]
[GameDuration "00:04:12"]
[GameEndTime "2020-07-16T23:51:47.085 CDT"]
[GameStartTime "2020-07-16T23:47:34.819 CDT"]
[Opening "QGA"]
[PlyCount "156"]
[TimeControl "60+1"]
[Variation "3.e4"]

1. d4 {book} d5 {book} 2. c4 {book} dxc4 {book} 3. e4 {book} Nc6 {book}
4. Be3 {+0.71/18 2.5s} Nf6 {-0.21/21 4.9s} 5. Nc3 {+0.73/19 3.2s}
e5 {-0.26/19 2.1s} 6. d5 {+0.71/20 2.9s} Nd4 {-0.21/17 0.70s}
7. Bxc4 {+0.60/21 2.8s} Bc5 {-0.23/17 1.3s} 8. Nge2 {+0.66/21 2.6s}
Bg4 {-0.23/18 1.9s} 9. O-O {+0.48/19 3.5s} Nxe2+ {-0.12/19 2.1s}
10. Bxe2 {+0.54/21 3.5s} Bxe3 {-0.11/19 1.2s} 11. fxe3 {+0.35/20 2.3s}
Bd7 {-0.12/19 1.1s} 12. Qb3 {+0.76/18 1.7s} Rb8 {-0.13/19 3.1s}
13. Rac1 {+0.26/18 7.3s} O-O {-0.17/17 2.0s} 14. Bf3 {+0.24/19 6.1s}
Qe7 {-0.09/21 3.3s} 15. Rf2 {+0.27/17 1.4s} c6 {-0.10/20 1.9s}
16. Rd2 {+0.25/20 5.9s} Qc5 {-0.07/19 0.95s} 17. Ne2 {+0.20/20 1.7s}
Qd6 {-0.01/21 1.2s} 18. Rcd1 {+0.13/20 4.3s} cxd5 {+0.04/20 1.3s}
19. Nc3 {+0.32/19 2.0s} Bc6 {0.00/23 1.7s} 20. g3 {+0.17/20 3.1s}
Rfc8 {-0.01/22 3.3s} 21. Nxd5 {+0.08/21 2.6s} Bxd5 {-0.02/20 1.5s}
22. exd5 {-0.15/19 1.9s} Rc7 {+0.01/26 5.1s} 23. Rc2 {0.00/19 2.1s}
Rbc8 {+0.08/22 1.4s} 24. Rxc7 {0.00/22 3.1s} Rxc7 {+0.04/20 1.3s}
25. Qd3 {-0.01/21 0.98s} g6 {+0.08/21 1.8s} 26. a3 {0.00/22 2.0s}
Ne8 {+0.17/19 1.6s} 27. Kg2 {0.00/18 1.1s} Qf6 {+0.26/21 5.0s}
28. e4 {-0.14/20 2.1s} Nd6 {+0.23/19 1.1s} 29. Re1 {-0.24/20 3.1s}
Qg5 {+0.18/21 5.5s} 30. h4 {-0.12/19 0.93s} Qh6 {+0.17/21 3.7s}
31. b3 {-0.16/19 2.1s} Qf8 {+0.16/20 2.3s} 32. Qe3 {-0.28/19 2.2s}
Nb5 {+0.37/23 3.8s} 33. Rc1 {-0.72/19 1.7s} Qxa3 {+0.42/21 1.7s}
34. Rxc7 {-0.86/20 1.3s} Nxc7 {+0.45/19 0.85s} 35. Qg5 {-0.90/20 2.2s}
Qd6 {+0.47/20 1.2s} 36. Qc1 {-0.97/20 1.2s} Qe7 {+0.52/19 1.4s}
37. Be2 {-1.08/21 1.1s} Ne8 {+0.45/20 1.5s} 38. h5 {-0.94/21 0.82s}
Kg7 {+0.65/20 1.7s} 39. Bb5 {-1.07/20 0.97s} Nd6 {+0.69/20 1.7s}
40. h6+ {-1.21/20 2.1s} Kf8 {+0.59/21 1.7s} 41. Be2 {-1.26/19 1.1s}
Ne8 {+0.62/20 1.4s} 42. Qb2 {-1.34/20 3.2s} Qd6 {+0.73/22 13s}
43. b4 {-1.03/17 0.95s} Qe7 {+0.69/19 1.4s} 44. Qc3 {-1.15/20 1.9s}
Qc7 {+0.79/19 0.27s} 45. Qc5+ {-1.24/17 0.73s} Qxc5 {+1.11/21 0.47s}
46. bxc5 {-1.13/19 0.43s} b6 {+1.42/17 0.43s} 47. d6 {-2.16/23 1.6s}
Nf6 {+1.44/21 0.73s} 48. cxb6 {-2.31/25 1.5s} axb6 {+1.50/22 0.55s}
49. Kf3 {-2.66/24 1.0s} Ke8 {+1.60/23 0.55s} 50. Bb5+ {-2.66/25 0.95s}
Kd8 {+1.66/22 0.56s} 51. g4 {-2.82/26 1.1s} g5 {+1.97/23 0.66s}
52. Ba4 {-3.71/24 1.0s} Ne8 {+2.33/23 0.64s} 53. d7 {-3.95/27 1.00s}
Nf6 {+2.82/20 0.71s} 54. Bc6 {-4.18/29 1.0s} Nxd7 {+2.88/22 0.76s}
55. Ba4 {-4.58/27 1.0s} Ke7 {+3.36/21 0.74s} 56. Bd1 {-4.96/24 1.0s}
Kd6 {+3.45/21 0.93s} 57. Bb3 {-5.14/26 1.00s} Kc5 {+4.34/22 2.1s}
58. Ke3 {-5.29/22 1.0s} Nf6 {+4.41/21 0.55s} 59. Bd1 {-5.65/24 1.00s}
Kc4 {+4.79/22 0.79s} 60. Bf3 {-5.89/26 1.0s} b5 {+5.48/21 0.84s}
61. Be2+ {-5.35/22 1.0s} Kb4 {+6.01/22 0.92s} 62. Kd3 {-6.25/23 0.86s}
Kb3 {+6.39/23 1.1s} 63. Bf3 {-6.33/25 0.91s} b4 {+7.16/21 0.85s}
64. Kd2 {-8.66/23 1.2s} Ka3 {+9.07/23 1.8s} 65. Be2 {-9.77/24 1.0s}
Nxe4+ {+9.70/18 0.51s} 66. Ke3 {-13.08/23 0.80s} Nf6 {+10.63/20 0.78s}
67. Bc4 {-14.18/21 1.2s} b3 {+12.30/19 1.3s} 68. Kd3 {-12.48/20 1.0s}
e4+ {+14.86/20 2.0s} 69. Kc3 {-18.06/20 1.00s} b2 {+16.10/18 0.48s}
70. Kc2 {-19.59/19 1.0s} e3 {+17.21/21 0.86s} 71. Bd3 {-21.56/17 1.0s}
Ka2 {+154.04/25 0.86s} 72. Bc4+ {-21.58/16 1.00s} Ka1 {+M23/32 0.69s}
73. Kb3 {-22.21/18 1.0s} b1=Q+ {+M11/58 0.70s} 74. Ka4 {-M10/14 0.44s}
e2 {+M9/75 0.71s} 75. Bxf7 {-M8/25 0.074s} e1=Q {+M7/152 0.73s}
76. Bb3 {-M6/99 0.047s} Qbe4+ {+M5/245 0.13s} 77. Kb5 {-M4/99 0.004s}
Q1b4+ {+M3/245 0.010s} 78. Ka6 {-M2/5 0s} Qeb7# {+M1/245 0.006s, Black mates}
0-1
[/pgn]

[Milos]

Considering this:

Oiy. What a colossal f*ck up. Before the final network save, the nnue executable loaded the nn.bin from eval and saved it in evalsave/final. The previous checkpoint was the network trained on Komodo data. While strong, it’s nowhere near Komodo or sf. The network in the eval/nn.bin? gk20200720, a very strong net trained on stockfish data. It’s byte for byte identical.

My apologies for the confusion. It was an unintentional mistake.

I really have to laugh at this:

If the combined search/net doesn’t play like Komodo but like something else, then it’s not a clone but some new thing. Finally some use for Rebel’s punitive clone detector!

Well, maybe it's a good idea to investigate what we are talking about, without the punitive, as Mark said NN's are a whole different world. A quick test with some adviced nets. Using a random epd of 10,000 positions at depth=1.

Code: Select all

Positions 10000 Leele NN-64 SF-NN SF11  T40-1 T60-4
Leelentein-14   ----- 73.30 43.24 38.92 77.35 73.30
NN-64194        73.30 ----- 43.17 38.83 72.31 75.97
SF-NN           43.24 43.17 ----- 45.72 42.43 42.83
SF11            38.92 38.83 45.72 ----- 38.11 38.27
T40-1541        77.35 72.31 42.43 38.11 ----- 71.45
T60-4300        73.30 75.97 42.83 38.27 71.45 -----

 1.         Engine Leelentein-14 and T40-1541 show a similarity of 77.35%
 2.         Engine NN-64194 and T60-4300 show a similarity of 75.97%
 3.         Engine Leelentein-14 and NN-64194 show a similarity of 73.30%
 4.         Engine Leelentein-14 and T60-4300 show a similarity of 73.30%
 5.         Engine NN-64194 and T40-1541 show a similarity of 72.31%
 6.         Engine T40-1541 and T60-4300 show a similarity of 71.45%

Quite some high similarities between the nets except the new approach with the 30Mb NN.BIN net from SF-NN version 0.1

Comments?

It's a wonderful demonstration of confirmation bias and also how useless and pointless are most of these "similarity tests"...

[Milos]

So far this and all other NNUE nets, whether trained on Komodo or Stockfish games, only run with the Stockfish search and software. We hope to change this in the near future, but no promise yet! I think you are confusing training on Komodo games with running inside the Komodo engine; those are two separate things, though it makes sense to combine them. If we did that, it wouldn't be Komodo + Stockfish, it would be Komodo + NNUE, which is the idea of the Japanese shogi author, not a Stockfish idea. Stockfish would have nothing to do with it in that case.

Considering that all NNUE code is open source, how are you planning integrating open source code into commercial product?
Will you make Komodo search code also available as requested per GPL license?
Or you are just gonna claim you rewrote all the code and that there is no GPL code left in so called "Houdart" way?