SF-NNUE going forward...

[perejaslav]

but people are just stopping using Leela because Leela is worse than NNUE at beating Stockfish.

people are just stopping using Leela because Leela cannot be tested in very fast time control using multithreaded mode.

[marsell]

People need several expensive and power-hungry GPUs to beat Stockfish

[perejaslav]

That's true, too. I would really like to test Leela against Stockfish, but under current conditions it will take a lot of time. Besides, I don't understand how to choose the right time control so that the programs are in equal conditions.

[Ovyron]

People need several expensive and power-hungry GPUs to beat Stockfish

What is interesting is that those people got the GPUs for Leela. They still have the GPUs. The GPUs didn't just disappear.

And yet they're choosing to use their CPUs and NNUE nets because that gives them better performance (less loses, more wins). I rub shoulders with those people, and they kept complaining that Leela wasn't winning them games, and that no matter what they tried, they were still drawing their games. With NNUE they get wins and don't complain anymore...

I was really close to getting one of those GPUs, I'm just glad I never did if now I'd be dropping Leela in favor of NNUE!

[Raphexon]

People need several expensive and power-hungry GPUs to beat Stockfish

What is interesting is that those people got the GPUs for Leela. They still have the GPUs. The GPUs didn't just disappear.

And yet they're choosing to use their CPUs and NNUE nets because that gives them better performance (less loses, more wins). I rub shoulders with those people, and they kept complaining that Leela wasn't winning them games, and that no matter what they tried, they were still drawing their games. With NNUE they get wins and don't complain anymore...

I was really close to getting one of those GPUs, I'm just glad I never did if now I'd be dropping Leela in favor of NNUE!

Has NNUE refuted a few lines yet?

[Ovyron]

Has NNUE refuted a few lines yet?

Yes, there's a little opening revolution as we speak, people that haven't updated their lines have lost +70 elo points. I'm currently outperforming people in much faster hardware than mine and I'm not even using NNUE, just holding up with drawish openings that don't lose while others go down in flames.

This is very recent, since Eman Enigma 1.00 was released an started winning tourneys and people started jumping towards it, refuting opening theory still used by Stockfish dev users (mainly, out of book positions where, say, Stockfish dev depth 38 still claims 0.00, while NNUE at just depth 20 already shows 1.00 white advantage, then the white side goes on to win positions that the book claimed were fine for black, but weren't.)

NNUE is not outsearching Stockfish dev, but out-evaluating it, I used to play those 0.00 lines and get draws, now the NNUEs are killing me, I think anybody picking a great book from last week will be destroyed today when they play black (some white lines have fallen, but white still holds so NNUE isn't strong enough to break white Stockfish dev.)

Someone went and played with NNUE without book and got better performance than Stockfish dev with book, just because of those poor book likes that are being uncovered. I can't really imagine how out of opening eval will look like in 2021, but it's possible 0.00 positions of today will be 2.00 by then

[Zenmastur]

Has NNUE refuted a few lines yet?

Yes, there's a little opening revolution as we speak, people that haven't updated their lines have lost +70 elo points. I'm currently outperforming people in much faster hardware than mine and I'm not even using NNUE, just holding up with drawish openings that don't lose while others go down in flames.

This is very recent, since Eman Enigma 1.00 was released an started winning tourneys and people started jumping towards it, refuting opening theory still used by Stockfish dev users (mainly, out of book positions where, say, Stockfish dev depth 38 still claims 0.00, while NNUE at just depth 20 already shows 1.00 white advantage, then the white side goes on to win positions that the book claimed were fine for black, but weren't.)

NNUE is not outsearching Stockfish dev, but out-evaluating it, I used to play those 0.00 lines and get draws, now the NNUEs are killing me, I think anybody picking a great book from last week will be destroyed today when they play black (some white lines have fallen, but white still holds so NNUE isn't strong enough to break white Stockfish dev.)

Someone went and played with NNUE without book and got better performance than Stockfish dev with book, just because of those poor book likes that are being uncovered. I can't really imagine how out of opening eval will look like in 2021, but it's possible 0.00 positions of today will be 2.00 by then

I'm going through several opening books I have for computer play. Many of the lines give advantage to one side or another in order to get a non-book position. It seems that a lot of the lines given may be busted by SF-NNUE. I'm having to delete about 5% of the lines because they are either close to lost or ARE completely lost. Rather surprising the number of lines I'm being forced to remove loosing.

I was expecting to lose some lines, mostly the longer ones. I wasn't expecting to lose so many 4-move lines. I can do the short ones manually, but there are way too many long ones to do this way.

[Ovyron]

I think this game showcases the shift. White is regular Stockfish in much bigger hardware than mine. Black is NNUE with the slow machine with a bad out-of-book opening position:

[pgn][Event "Rated game, 5 min"]
[Site "Engine Room"]
[Date "2020.07.28"]
[White "Leluzechois, Stockfish 170720 64"]
[Black "Ulycrom, Chi-chi+NNUE"]
[Result "1/2-1/2"]
[ECO "E12"]
[WhiteElo "2629"]
[BlackElo "2539"]
[Annotator "2.04;1.69"]
[PlyCount "292"]
[EventDate "2020.07.28"]
[SourceTitle "playchess.com"]
[TimeControl "300"]

{Stockfish 170720 64 POPCNT (64 threads): 52.7 plies; 78,671kN/s AMD Ryzen
Threadripper 2990WX 32-Core Processor 3500MHz, (32 cores, 64 threads), 347 MB}
1. c4 {B 0} c5 {-0.05/1 0} 2. Nf3 {B 0} Nf6 {-0.05/1 1} 3. Nc3 {B 0} d5 {
-0.05/1 0} 4. cxd5 {B 0} Nxd5 {-0.05/1 0} 5. d4 {B 0} e6 {-0.05/1 0} 6. e3 {
B 0 (e4)} Be7 {-0.05/1 0} 7. Bd3 {B 0} Nc6 {-0.05/1 0} 8. O-O {B 0} O-O {
-0.05/1 0} 9. a3 {B 0} Nxc3 {-0.05/1 0} 10. bxc3 {B 0} b6 {-0.05/1 0} 11. Qc2 {
B 0} h6 {-0.05/1 0} 12. Bb2 {B 0} Bb7 {-0.05/1 0} 13. Qe2 {B 0} Qc7 {-0.05/1 0}
14. e4 {B 0} Rfe8 {-0.05/1 0} 15. Rad1 {B 0} Bf8 {-0.05/1 0} 16. h3 {B 0 (Rfe1)
} a6 {-0.05/1 1} 17. a4 {B 0} g6 {-0.05/1 0} 18. h4 {B 0} h5 {-0.05/1 0} 19.
Ng5 {B 0} Bg7 {-0.05/1 0} 20. Bb1 {B 0} b5 {-0.05/1 0} 21. f4 {B 0} Na5 {
-0.05/1 0} 22. e5 {B 0} cxd4 {-0.05/1 0} 23. cxd4 {B 0} Nc4 {-0.05/1 0} 24. Bc3
{B 0} Bd5 {-0.05/1 0} 25. Bb4 {B 0} bxa4 {-0.05/1 0} 26. Rc1 {B 0} Rec8 {
-0.05/1 0} 27. Bc5 {B 0} Bf8 {-0.05/1 0} 28. Bxf8 {B 0} Rxf8 {-0.05/1 0} 29.
Ne4 {B 0} Qa5 {-0.05/1 0} 30. Rxc4 {B 0} Bxc4 {-0.05/1 0} 31. Qxc4 {B 0} Rab8 {
-0.05/1 0} 32. Kh2 {B 0} Kg7 {-0.05/1 0} 33. Nd6 {B 0} Rb4 {-0.05/1 0} 34. Qc3
{B 0} Qb6 {-0.05/1 0} 35. Bc2 {B 0} Rd8 {-0.05/1 0} 36. Be4 {B 0} Rb3 {-0.05/1
0} 37. Qa1 {B 0} Rb2 {-0.05/1 0} 38. Qxa4 {B 0} Rb4 {-0.05/1 0} 39. Qc2 {B 0}
Rxd6 {-0.05/1 0} 40. exd6 {B 0} Qxd6 {-0.05/1 0} 41. g3 {B 0} Qxd4 {-0.05/1 0}
42. Bg2 {B 0} Rb2 {-0.05/1 0} 43. Rd1 {B 0} Rxc2 {-0.05/1 0} 44. Rxd4 {B 0} a5
{-0.05/1 0} 45. Kh3 {B 0} Rc7 {-0.05/1 0} 46. Be4 {2.04/41 10 (Bf3)} Kf6 {
1.69/245 17} 47. Rd6 {2.04/56 0 (Ra4)} Ra7 {1.69/38 7} 48. Rd3 {2.51/55 0} Ra6
{1.69/39 6 (Ta7-c7)} 49. Rc3 {2.51/52 29 (Ra3)} Rb6 {1.79/43 14 (e6-e5)} 50.
Rc7 {2.62/60 5} a4 {1.79/46 1 (Tb6-b3)} 51. Bc6 {2.71/67 4 (Ra7)} Rb8 {1.79/44
6 (Rf6-g7)} 52. Bxa4 {2.71/63 5 (Ra7)} Rd8 {1.79/50 12} 53. Bb3 {2.71/74 0
(Ra7)} Ra8 {1.79/50 8 (Td8-b8)} 54. Rc6 {2.71/68 5 (Bc4)} Rb8 {1.79/53 6} 55.
Bc4 {2.71/76 0} Kg7 {1.79/52 5 (Tb8-b4)} 56. Ra6 {2.71/68 4 (Rc7)} Rb2 {
1.79/49 5 (Rg7-f6)} 57. Ra7 {2.71/71 4 (Ra5)} Rb6 {1.79/51 6 (Rg7-f6)} 58. Be2
{2.71/71 8 (Ra6)} Rb2 {1.79/52 6} 59. Bc4 {2.71/74 0} Kf6 {1.79/52 4} 60. Ra6 {
2.71/75 0 (Bd3)} Kg7 {1.79/52 6 (Tb2-b8)} 61. Rc6 {2.71/66 4 (Ra5)} Rc2 {
1.79/52 5 (Rg7-f6)} 62. Bd5 {2.71/72 4 (Bb5)} Rf2 {1.79/51 6 (Tc2-b2)} 63. Bg2
{2.71/69 4} Kf6 {1.79/47 2 (Tf2-d2)} 64. Rc4 {2.71/68 8 (Rc5)} Rb2 {1.79/48 9
(Tf2-d2)} 65. Rc7 {2.71/67 3} Rb4 {1.79/47 1 (Rf6-g7)} 66. Rb7 {2.71/68 4 (Ra7)
} Rc4 {1.79/49 4} 67. Bf1 {2.71/67 1 (Ra7)} Rc2 {1.79/49 4 (Tc4-d4)} 68. Rb6 {
2.71/64 11 (Bb5)} Rc1 {1.79/49 4 (Tc2-c8)} 69. Bg2 {2.71/60 3} Kg7 {1.79/48 0
(Tc1-c8)} 70. Rc6 {2.71/59 3 (Kh2)} Rb1 {1.79/52 6 (Tc1-d1)} 71. Bf3 {2.71/56
3 (Rc7)} Rb2 {1.79/54 4} 72. Ra6 {2.71/56 0 (Rc7)} Rc2 {1.79/50 5 (Rg7-f8)} 73.
Be4 {2.71/59 8 (Rc6)} Rc4 {1.79/49 5 (Tc2-d2)} 74. Bc6 {2.71/53 4} Rb4 {
1.79/48 1 (Tc4-c3)} 75. Ba4 {2.71/51 6 (Bg2)} Rb7 {1.79/49 4} 76. Rc6 {2.71/50
0 (Ra8)} Rb2 {1.79/47 4 (Tb7-a7)} 77. Bc2 {2.71/48 3 (Bd1)} Kf8 {1.79/48 6
(Rg7-f6)} 78. Rc8+ {2.71/44 10 (Ba4)} Ke7 {1.79/45 4 (Rf8-g7)} 79. Bd3 {
2.71/45 7} Kf6 {1.79/44 13} 80. Rc6 {2.71/43 0 (Rc2)} Kg7 {1.79/42 37 (Rf6-e7)}
81. Rc2 {2.62/39 7 (Rc7)} Rb3 {1.70/37 2 (Tb2-b8)} 82. Bc4 {2.62/36 3 (Be2)}
Ra3 {1.70/37 3 (Tb3-b8)} 83. Kg2 {2.62/34 2 (Ra2)} Kf6 {1.70/34 2 (Ta3-a1)} 84.
Bf1 {2.62/33 4 (Kh3)} Kg7 {1.69/30 1 (Ta3-a7)} 85. Bc4 {2.62/31 7} Kf6 {
1.61/31 2 (Ta3-a8)} 86. Kf2 {2.58/28 8 (Kh2)} Ra7 {1.52/29 5 (Rf6-e7)} 87. Ra2
{2.23/29 14 (Bb5)} Rb7 {1.48/24 2 (Ta7-c7)} 88. Kg2 {1.66/40 5 (Ra6)} Rb4 {
1.23/24 2} 89. Rc2 {1.66/40 0 (Be2)} Rb8 {1.15/25 3 (Rf6-g7)} 90. Rc3 {1.66/40
2 (Be2)} Rc8 {0.76/23 3 (Tb8-b2+)} 91. Rc2 {1.56/40 4 (Kh3)} Rb8 {0.45/20 1
(Rf6-g7)} 92. Ra2 {1.56/44 3 (Be2)} Rb4 {0.39/20 1 (Tb8-c8)} 93. Bf1 {1.56/38 2
} Rb7 {0.36/23 0 (Tb4-b3)} 94. Ra3 {1.56/43 2 (Ra4)} Rb2+ {0.48/24 1} 95. Kf3 {
1.56/42 0 (Kh3)} Rb1 {0.49/24 1} 96. Bh3 {1.56/40 0 (Be2)} Rg1 {0.49/31 2
(Tb1-b5)} 97. Kf2 {1.56/46 1 (Re3)} Rc1 {0.49/30 1 (Tg1-d1)} 98. g4 {1.56/44 2}
hxg4 {0.53/30 0} 99. Bxg4 {1.56/51 2} Rh1 {0.53/33 0 (Rf6-e7)} 100. Bh3 {
1.56/46 2} Rh2+ {0.53/34 0 (Th1-d1)} 101. Kg3 {1.56/50 1} Rb2 {0.53/32 0
(Th2-c2)} 102. Rc3 {1.56/47 2 (Bg2)} Rb1 {0.53/31 1 (Rf6-g7)} 103. Kg4 {
1.56/49 1 (Re3)} Kg7 {0.53/33 1} 104. Bg2 {1.56/50 0} Rb4 {0.53/33 0} 105. Rc5
{1.56/50 1 (Kg3)} Rb3 {0.53/29 1 (Tb4-b2)} 106. Bf3 {1.56/50 1} Ra3 {0.52/27 0
(Tb3-b1)} 107. Rc7 {1.56/50 1} Kf6 {0.53/29 0 (Ta3-a1)} 108. Be2 {1.56/48 1}
Kg7 {0.53/30 4 (Ta3-b3)} 109. Bb5 {1.56/52 1} Ra5 {0.53/31 0 (Rg7-f8)} 110. Be8
{1.56/51 1 (Bc4)} Rf5 {0.53/32 1} 111. Ba4 {1.56/55 0 (Rb7)} e5 {0.21/21 1
(Tf5-a5)} 112. fxe5 {1.47/47 1} Rxe5 {0.18/28 0} 113. Bb3 {1.47/49 0} Rf5 {
0.18/30 0} 114. Bc4 {1.37/47 1 (Be6)} Kg8 {0.18/32 1 (Tf5-f6)} 115. Bd3 {
1.28/51 1 (Rb7)} Re5 {0.18/27 1 (Tf5-a5)} 116. Bc4 {1.19/166 7} Rf5 {0.18/42 0}
117. Kh3 {1.19/50 0 (Rb7)} Kg7 {0.18/30 1} 118. Be6 {1.19/54 0 (Kg3)} Rf3+ {
0.18/35 0 (Tf5-f4)} 119. Kg4 {1.18/52 2 (Kg2)} Rf6 {0.18/36 1 (Tf3-f1)} 120.
Bc4 {1.18/47 1} Kg8 {0.18/41 0 (Tf6-f2)} 121. Kg3 {1.09/48 1 (Rb7)} Rf5 {
0.18/33 0 (Rg8-g7)} 122. Bd3 {1.09/52 1 (Bb3)} Re5 {0.18/34 0 (Tf5-a5)} 123.
Rd7 {1.09/51 1 (Kg4)} Kf8 {0.09/26 0 (Te5-c5)} 124. Bc2 {1.09/49 1 (Kg4)} Re1 {
0.09/17 1 (Te5-e3+)} 125. Bd1 {1.09/46 2 (Rd4)} Re4 {0.17/24 0 (Te1-g1+)} 126.
Bf3 {1.00/46 1 (Rc7)} Re7 {0.17/26 1 (Te4-a4)} 127. Rd8+ {1.00/43 1 (Rd4)} Kg7
{0.16/30 1} 128. Rd4 {1.00/49 0 (Rc8)} Rc7 {0.16/29 1 (Te7-a7)} 129. Rd3 {
1.00/48 1 (Bd1)} f5 {0.06/25 1 (Tc7-a7)} 130. Rd4 {1.00/46 1} Kh6 {0.00/26 0
(Tc7-c1)} 131. Rd6 {0.91/45 1} Rc1 {0.00/27 0} 132. Rb6 {0.91/49 0 (Ra6)} Kg7 {
0.00/28 1} 133. Kf4 {0.91/51 0 (Ra6)} Rf1 {0.00/28 1 (Rg7-h6)} 134. Rc6 {
0.82/51 1 (Ra6)} Rb1 {0.00/31 0 (Tf1-g1)} 135. Rc4 {0.73/48 1 Ulycrom,Chi-chi
offers a draw} Kh6 {0.00/36 0} 136. Rd4 {0.73/47 0} Rb6 {0.00/37 0 (Tb1-c1)}
137. Ra4 {0.64/49 1 (Bd1)} Rb3 {0.00/33 0} 138. Rd4 {0.54/47 0} Rb6 {0.00/35 0
(Tb3-c3)} 139. Ra4 {0.45/44 1 (Bd1)} Rb3 {0.00/28 0} 140. Rc4 {0.45/34 0 (Ra6)}
Rb6 {0.00/29 0 (Tb3-d3)} 141. Rd4 {0.09/37 1} Rb1 {0.00/35 0 (Tb6-b3)} 142. Bd1
{0.00/42 1} Rb6 {0.00/37 0} 143. Bf3 {0.00/56 1 (Rd7)} Rb3 {0.00/30 0} 144. Rd8
{0.00/37 0 (Bd1)} Rb6 {0.00/28 0 (Tb3-b4+)} 145. Rh8+ {0.00/37 1 (Rc8)} Kg7 {
0.00/1 0} 146. Ra8 {0.00/43 0 (Rd8)} Rb3 {0.00/25 0 (Rg7-h6)} 1/2-1/2[/pgn]

Stockfish is reaching much higher depths with outstanding eval and can't win. I think I'd have lost this game if I had hardware as fast as my opponent's but Stockfish dev. Using NNUE is like playing with software from the future.

[Dann Corbit]

People need several expensive and power-hungry GPUs to beat Stockfish

What is interesting is that those people got the GPUs for Leela. They still have the GPUs. The GPUs didn't just disappear.

GPUs are marvelous, incredible things. I bought GPUs long before I found out they can be useful for chess. I have GPUs in all of my machines and boxes of GPUs besides.

You can test Mersenne Prime candidates.
You can fold proteins and cure diseases.
You can look for gravitational waves
It's not a case of what can you do with a GPU, it's what can't you do. I guess you can even stop a herd of stampeding Yak with GPUs.

The uses are endless. I think GPUs are the most fabulous thing (hardware-wise) on the surface of the earth. I can't wait to buy some more and some really smoking hot ones are just around the corner

[Ovyron]

It's not a case of what can you do with a GPU, it's what can't you do.

Well, I can't run Leela on my GPU. It's NVIDIA, but from before they implemented CUDA. And I'm on Windows 8.1 so their DirectX 12 solution doesn't work, and the CPU Leela I can run is laughably bad.

So on all accounts NNUE is a winner for me.