Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
corres
Posts: 3482
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Post by corres » Sun Sep 27, 2020 7:15 am

"As the basement of NNUE-net is the Stockfish evaluation, the chess power of NNUE-net is restricted. During the process of reinforcement learning the NNUE-net does not get new information so the enhancement of chess power of NNUE-net is also restricted."

Who think the evaluation power of Stockfish is endless? I hope nobody. As in principle the self-play of Stockfish give the base of making nn-xyz.nnue, obviously the evaluation power of the nn-xyz.nnue using Stockfish is also no endless. During reinforcement learning for self playing engines use the earlier made net to gain a more stronger net. But if somebody think, during the process we can get a chess engine what can 'solve the chess' that people think badly.
The picture above obviously proves the course is not a self-enhancing course.

"I think the power of SF+NNUE may reach near its height and now the development of Classical Stockfish can enhance the chess power of SF+NNUE. But it is a much more slower course than you can expect from the earlier results."

The slower enhancement of Stockfish Elo is saw also on the above picture.
Now this enhancement basically arise from the work of the Stockfish`sdevelopers. In the better case we hope the enhancement give new information to the maker of the nets getting even stronger nets. So the course maybe endless, but the reached chess power of SF+NNUE is restricted.
Any other question?

syzygy
Posts: 4807
Joined: Tue Feb 28, 2012 10:56 pm

Re: Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Post by syzygy » Sun Sep 27, 2020 10:10 am

corres wrote:
Sun Sep 27, 2020 7:15 am
"As the basement of NNUE-net is the Stockfish evaluation, the chess power of NNUE-net is restricted. During the process of reinforcement learning the NNUE-net does not get new information so the enhancement of chess power of NNUE-net is also restricted."

Who think the evaluation power of Stockfish is endless? I hope nobody. As in principle the self-play of Stockfish give the base of making nn-xyz.nnue, obviously the evaluation power of the nn-xyz.nnue using Stockfish is also no endless.
You seem to be responding only to yourself instead of to my reply, so I will just repeat:

Do you realise that Alpha Zero and LC0 both have ZERO "basement"?
Their first-iteration nets learned from moving around pieces blindly and looking at the results.
Their second-iteration nets learned from the slightly less blind first iteration nets.
And so on.
Each iteration made it see clearer.

The first-iteration SF NNUE net was trained on SF-classical's evaluation.
The second-iteration leans from the first-iteration net, and so on.
Each iteration makes it see clearer.

One could have trained SF NNUE nets starting from a ZERO evaluation, but it would have taken a lot longer. (It would be interesting to see the result, though.)

corres
Posts: 3482
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Post by corres » Sun Sep 27, 2020 12:22 pm

syzygy wrote:
Sun Sep 27, 2020 10:10 am
corres wrote:
Sun Sep 27, 2020 7:15 am
"As the basement of NNUE-net is the Stockfish evaluation, the chess power of NNUE-net is restricted. During the process of reinforcement learning the NNUE-net does not get new information so the enhancement of chess power of NNUE-net is also restricted."

Who think the evaluation power of Stockfish is endless? I hope nobody. As in principle the self-play of Stockfish give the base of making nn-xyz.nnue, obviously the evaluation power of the nn-xyz.nnue using Stockfish is also no endless.
You seem to be responding only to yourself instead of to my reply, so I will just repeat:

Do you realise that Alpha Zero and LC0 both have ZERO "basement"?
Their first-iteration nets learned from moving around pieces blindly and looking at the results.
Their second-iteration nets learned from the slightly less blind first iteration nets.
And so on.
Each iteration made it see clearer.
The first-iteration SF NNUE net was trained on SF-classical's evaluation.
The second-iteration leans from the first-iteration net, and so on.
Each iteration makes it see clearer.
One could have trained SF NNUE nets starting from a ZERO evaluation, but it would have taken a lot longer. (It would be interesting to see the result, though.)
I hope you can answer your own question too.
What about you write is the reinforcement learning.
I think starting from zero evaluation the net making is possible in that case if the net contain policy head also - opposite to the nnue-nets.

User avatar
MikeB
Posts: 4412
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Post by MikeB » Tue Sep 29, 2020 3:24 am

Chessqueen wrote:
Wed Sep 02, 2020 3:16 am
https://tcec-chess.com/live.html
To answer your question , yes it is better and we are sure. ;>)
Image

RogerC
Posts: 32
Joined: Tue Oct 29, 2019 7:33 pm
Location: French Polynesia
Full name: Roger C.

Re: Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Post by RogerC » Tue Sep 29, 2020 10:20 am

SF dev : +23.2 ELO vs SF12 in just 26 days on 1 CPU thread !

During this time 4 new NNUE nets and lots of AB search improvements, scaling, tweaking, vectorization, hybrid rules evaluations and compilation speedups.

SF13 development is the fastest of all versions of SF :

https://github.com/glinscott/fishtest/w ... sion-Tests
Image

JohnWoe
Posts: 228
Joined: Sat Mar 02, 2013 10:31 pm

Re: Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Post by JohnWoe » Tue Sep 29, 2020 1:28 pm

After 2 games 50% score. Should've stopped right there :D

Code: Select all

Finished game 2 (Stockfish 12 vs Sapeli 1.91): 0-1 {Black mates}
Score of Sapeli 1.91 vs Stockfish 12: 1 - 1 - 0  [0.500] 2
In reality the amount of games needed in chess is massive. As chess is drawish by nature.

Jouni
Posts: 2227
Joined: Wed Mar 08, 2006 7:15 pm

Re: Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Post by Jouni » Tue Sep 29, 2020 6:55 pm

Also current NCM testings shows sudden +10 or more gain. I am not anymore surprised of anything! BTW TCEC chat said, that Vieri has stopped NN training, true?
Jouni

Cornfed
Posts: 99
Joined: Sun Apr 26, 2020 9:40 pm
Full name: Brian D. Smith

Re: Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Post by Cornfed » Tue Sep 29, 2020 11:57 pm

JohnWoe wrote:
Tue Sep 29, 2020 1:28 pm
After 2 games 50% score. Should've stopped right there :D

Code: Select all

Finished game 2 (Stockfish 12 vs Sapeli 1.91): 0-1 {Black mates}
Score of Sapeli 1.91 vs Stockfish 12: 1 - 1 - 0  [0.500] 2
In reality the amount of games needed in chess is massive. As chess is drawish by nature.
I know it is my contrarian nature kicking in...but if it seems like a well chosen set of positions to play out mitigates that considerably. And given the strength of engines these days and chess being a game of 'mistakes' great and small...just seems like one should be able to spot (in engine vs engine play) which is better when 'non-draws' start showing up.

User avatar
yurikvelo
Posts: 583
Joined: Sat Dec 06, 2014 12:53 pm

Re: Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Post by yurikvelo » Wed Sep 30, 2020 7:25 am

Is it possible to have couple of specialized NNUE networks for different game phases?

Vinvin
Posts: 4766
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Are we sure that Stockfish NNUE is better than the Normal Stockfish ?

Post by Vinvin » Wed Sep 30, 2020 9:51 am

JohnWoe wrote:
Tue Sep 29, 2020 1:28 pm
After 2 games 50% score. Should've stopped right there :D

Code: Select all

Finished game 2 (Stockfish 12 vs Sapeli 1.91): 0-1 {Black mates}
Score of Sapeli 1.91 vs Stockfish 12: 1 - 1 - 0  [0.500] 2
In reality the amount of games needed in chess is massive. As chess is drawish by nature.
"As chess is drawish by nature."
Note 1 : this is a belief not a proven fact. We saw this kind of allegation in different times :
- In the years '70s, when Russian GMs made more and more draws.
- When Rybka (around year 2010) was dominating the chess scene, the number of draws was rising so much that many people said that we reached a limit where only draws was possible.
- and so on ...
But now top engines are 500 Elo above Rybka and 1000 Elo above the level of play of '70s

Note 2 : it's amazing to point that "chess is drawish by nature" after 2 non draw games :lol: :lol: :lol:

Post Reply