As we all know, the classic evaluation functions are a tradeoff between speed and accuracy but given that most NNUE nets are trained using fixed depth search results, speed would no longer be a factor.
I was wondering if much more complex/accurate evaluation functions would result in stronger nets.
What's your take on this?
Would More Complex Classic Evals Improve NNUE Nets?
Moderators: hgm, Rebel, chrisw
-
- Posts: 25
- Joined: Mon Nov 02, 2020 5:14 pm
- Full name: Thomas J. Miller
-
- Posts: 27811
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Would More Complex Classic Evals Improve NNUE Nets?
It depends on how you train the nets, right? If you train them on game results, using high-level games will only be counterproductive. Because you will hardly see things that are very wrong in such games, so the net can never learn to avoid those. And when engines with poor evaluations will get into positions with bad features that they do not recognize, they would still lose the game. So that is good enough to train the net, even if the engine itself never understood why it lost.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Would More Complex Classic Evals Improve NNUE Nets?
Accurate evaluation is always a plus, unfortunately in most contest there is a time factor, so you have to balance accuracy and speed. If you already have a strong net you can use it to generate a another net then another net until it would no longer result to a stronger net.ThomasMiller wrote: ↑Wed Apr 07, 2021 7:48 pm As we all know, the classic evaluation functions are a tradeoff between speed and accuracy but given that most NNUE nets are trained using fixed depth search results, speed would no longer be a factor.
I was wondering if much more complex/accurate evaluation functions would result in stronger nets.
What's your take on this?
There can be a limit of how accurate an evaluation function is, example given a drawish rook and pawn ending, a better net would evaluate it to +10 and a regular net would evaluate it to +15. +10 is close to zero so it is more accurate, but the result of the game is only 1/2-1/2.
-
- Posts: 1564
- Joined: Thu Jul 16, 2009 10:47 am
- Location: Almere, The Netherlands
Re: Would More Complex Classic Evals Improve NNUE Nets?
It's my feeling that training a net with better examples in theory would result in a stronger net, but it solely depends upon how much data the network is able to absorb. When you train a network with a classic eval it usually gets somewhat better because the net generalizes and the classic eval does not.
Using the results of a fixed depth search also helps to make the network stronger, but this has some drawbacks as well. In my experience a FCNN like we use for NNUE has very big difficulties understanding search scores that are partially or fully determined by tactics. If you want to use search scores to train a network better make sure that the variation you train on doesn't contain any tactics.
I've not tried yet, but it could be that a NNUE network trained on LC0 search scores will produce a better net than when it is trained on e.g. Stockfish search scores with classic evaluation.
Using the results of a fixed depth search also helps to make the network stronger, but this has some drawbacks as well. In my experience a FCNN like we use for NNUE has very big difficulties understanding search scores that are partially or fully determined by tactics. If you want to use search scores to train a network better make sure that the variation you train on doesn't contain any tactics.
I've not tried yet, but it could be that a NNUE network trained on LC0 search scores will produce a better net than when it is trained on e.g. Stockfish search scores with classic evaluation.
-
- Posts: 12542
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Would More Complex Classic Evals Improve NNUE Nets?
Yassar Sierawan said something interesting about tactics.
When you have not developed yet, do not look for tactics.
First you must develop, and then you must look for tactics.
In fact, tactics without development are usually a mistake.
When you have not developed yet, do not look for tactics.
First you must develop, and then you must look for tactics.
In fact, tactics without development are usually a mistake.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.