this will be the merge of a lifetime : SF 80 Elo+

[jhellis3]

I doubt it would make that big of a difference really. There is one very small change I would make to nodchip's repo for the training data gen. Maybe I will submit a PR.....

[Jhoravi]

What if LC0 is forced to use NNUE eval? Do you think it's just as strong as LC0 CPU?

[M ANSARI]

I think the main reason for switching to Classical search and creating a hybrid has nothing to do with Classical evaluation being better ... just that it seems that with less pieces on the board depth of calculation makes a difference. If the classical evaluation runs 50% faster it can perform better than the NN evaluation even if the NN evaluation is theoretically better simply because with less pieces on the board there is less of a safety net in case a deep tactic is missed. So for the time being this seems like a reasonable band aid fix ... that is until a smaller super fast NN network which is created from training on positions with less pieces is made. It doesn't take much imagination to recognize that such a network will be much more superior than any classic evaluation created manually by humans. But really ... until that happens the hybrid approach makes a lot of sense and I don't get all the hate about this temporary solution.

These are exciting times as I can see the future top engine as being an engine that starts of with a big network and then switches to smaller networks depending on some criteria ... what that criteria is I am still not sure of. Should the criteria be the number of possible moves without a large drop of evaluation? Should it be dependent on how many pieces are in play? Time will tell and for sure it will be interesting to follow and I suspect this will be a time of huge experimentation as the possible permutations in testing are mind boggling.

[zullil]

I think the main reason for switching to Classical search and creating a hybrid has nothing to do with Classical evaluation being better ... just that it seems that with less pieces on the board depth of calculation makes a difference. If the classical evaluation runs 50% faster it can perform better than the NN evaluation even if the NN evaluation is theoretically better simply because with less pieces on the board there is less of a safety net in case a deep tactic is missed. So for the time being this seems like a reasonable band aid fix ... that is until a smaller super fast NN network which is created from training on positions with less pieces is made. It doesn't take much imagination to recognize that such a network will be much more superior than any classic evaluation created manually by humans. But really ... until that happens the hybrid approach makes a lot of sense and I don't get all the hate about this temporary solution.

I don't sense a lot of "hate". That said, it would be nice if Stockfish offered users a choice between NNUE only, hybrid, and classical evaluations. Many users cannot edit and recompile code. And not everyone is interested in maximizing Elo at Fishtest (very short) time controls.

[Dann Corbit]

What if LC0 is forced to use NNUE eval? Do you think it's just as strong as LC0 CPU?

I think not, because MCTS search is not quite as good as alpha-beta.
For instance, Komodo MCTS is a bit weaker than the alpha-beta version.
LC0 was written with MCTS because (I think) it is hard to write a good recursive search for GPUs that fully utilize the hardware
Srdja could probably correct me if I am exaggerating unintentionally

[smatovic]

What if LC0 is forced to use NNUE eval? Do you think it's just as strong as LC0 CPU?

I think not, because MCTS search is not quite as good as alpha-beta.
For instance, Komodo MCTS is a bit weaker than the alpha-beta version.
LC0 was written with MCTS because (I think) it is hard to write a good recursive search for GPUs that fully utilize the hardware
Srdja could probably correct me if I am exaggerating unintentionally

Hehe, I think you got to the point well. LC0 runs batches of NN eval via MCTS to utilize the GPU, and even then it may be recommendend to run two threads per GPU.

--
Srdja

[schack]

@deberger - what value should I use for NNUEThreshold2 in the updated code after today's "fallback to NNUE" push? 0? Infinite?

[quote=Deberger post_id=855403 time=1596939643 user_id=12155]
[quote=schack post_id=855399 time=1596937188 user_id=5142]
Ok. What's the trivial change to the source code?

[quote=Deberger post_id=855397 time=1596931848 user_id=12155]
[quote=carldaman post_id=855385 time=1596920236 user_id=6402]
It defeats the purpose if the NNUE cannot always be set to active [as an option], because then you're at the mercy of the engine only using NNUE when it chooses. Hopefully the SF team will wake up to the fact that people also use the engine for analysis.
[/quote]

With a trivial change to the source code, you can build your own executable which always uses NNUE.

However this change can never yield stronger analysis.

Over time, using "always NNUE" will be gradually weaker and weaker as the nets are optimized for their intended use in balanced positions.
[/quote]
[/quote]

src/evaluate.cpp:
[code]- constexpr Value NNUEThreshold = Value(520);
+ constexpr Value NNUEThreshold = VALUE_INFINITE;
[/code]
[/quote]

[zullil]

@deberger - what value should I use for NNUEThreshold2 in the updated code after today's "fallback to NNUE" push? 0? Infinite?

If you want to have NNUE used for all evaluations, I suggest you get your source code here:

https://github.com/joergoster/Stockfish ... nue_always

Simply click on the green Code button and then Download ZIP.

[Deberger]

@deberger - what value should I use for NNUEThreshold2 in the updated code after today's "fallback to NNUE" push? 0? Infinite?

src/evaluate.cpp:

Code: Select all

-  constexpr Value NNUEThreshold1  =   Value(550);
+  constexpr Value NNUEThreshold1  =   VALUE_INFINITE;

There's no need to touch NNUEThreshold2 because the logical subexpression (classical && Eval::useNNUE) will always be false

Code: Select all

if (classical && Eval::useNNUE && abs(v) * 16 < NNUEThreshold2 * (16 + pos.rule50_count()))

[schack]

Thanks! Much obliged.