this will be the merge of a lifetime : SF 80 Elo+

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
jhellis3
Posts: 488
Joined: Fri Aug 16, 2013 10:36 pm

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by jhellis3 » Tue Aug 11, 2020 6:08 pm

I doubt it would make that big of a difference really. There is one very small change I would make to nodchip's repo for the training data gen. Maybe I will submit a PR.....

Jhoravi
Posts: 285
Joined: Wed May 08, 2013 4:49 am

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Jhoravi » Wed Aug 12, 2020 5:10 am

What if LC0 is forced to use NNUE eval? Do you think it's just as strong as LC0 CPU?

User avatar
M ANSARI
Posts: 3502
Joined: Thu Mar 16, 2006 6:10 pm

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by M ANSARI » Wed Aug 12, 2020 9:49 am

I think the main reason for switching to Classical search and creating a hybrid has nothing to do with Classical evaluation being better ... just that it seems that with less pieces on the board depth of calculation makes a difference. If the classical evaluation runs 50% faster it can perform better than the NN evaluation even if the NN evaluation is theoretically better simply because with less pieces on the board there is less of a safety net in case a deep tactic is missed. So for the time being this seems like a reasonable band aid fix ... that is until a smaller super fast NN network which is created from training on positions with less pieces is made. It doesn't take much imagination to recognize that such a network will be much more superior than any classic evaluation created manually by humans. But really ... until that happens the hybrid approach makes a lot of sense and I don't get all the hate about this temporary solution.

These are exciting times as I can see the future top engine as being an engine that starts of with a big network and then switches to smaller networks depending on some criteria ... what that criteria is I am still not sure of. Should the criteria be the number of possible moves without a large drop of evaluation? Should it be dependent on how many pieces are in play? Time will tell and for sure it will be interesting to follow and I suspect this will be a time of huge experimentation as the possible permutations in testing are mind boggling.

zullil
Posts: 6442
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by zullil » Wed Aug 12, 2020 10:01 am

M ANSARI wrote:
Wed Aug 12, 2020 9:49 am
I think the main reason for switching to Classical search and creating a hybrid has nothing to do with Classical evaluation being better ... just that it seems that with less pieces on the board depth of calculation makes a difference. If the classical evaluation runs 50% faster it can perform better than the NN evaluation even if the NN evaluation is theoretically better simply because with less pieces on the board there is less of a safety net in case a deep tactic is missed. So for the time being this seems like a reasonable band aid fix ... that is until a smaller super fast NN network which is created from training on positions with less pieces is made. It doesn't take much imagination to recognize that such a network will be much more superior than any classic evaluation created manually by humans. But really ... until that happens the hybrid approach makes a lot of sense and I don't get all the hate about this temporary solution.
I don't sense a lot of "hate". That said, it would be nice if Stockfish offered users a choice between NNUE only, hybrid, and classical evaluations. Many users cannot edit and recompile code. And not everyone is interested in maximizing Elo at Fishtest (very short) time controls. :wink:

Dann Corbit
Posts: 12168
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Dann Corbit » Wed Aug 12, 2020 3:43 pm

Jhoravi wrote:
Wed Aug 12, 2020 5:10 am
What if LC0 is forced to use NNUE eval? Do you think it's just as strong as LC0 CPU?
I think not, because MCTS search is not quite as good as alpha-beta.
For instance, Komodo MCTS is a bit weaker than the alpha-beta version.
LC0 was written with MCTS because (I think) it is hard to write a good recursive search for GPUs that fully utilize the hardware
Srdja could probably correct me if I am exaggerating unintentionally
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

smatovic
Posts: 1681
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by smatovic » Wed Aug 12, 2020 5:37 pm

Dann Corbit wrote:
Wed Aug 12, 2020 3:43 pm
Jhoravi wrote:
Wed Aug 12, 2020 5:10 am
What if LC0 is forced to use NNUE eval? Do you think it's just as strong as LC0 CPU?
I think not, because MCTS search is not quite as good as alpha-beta.
For instance, Komodo MCTS is a bit weaker than the alpha-beta version.
LC0 was written with MCTS because (I think) it is hard to write a good recursive search for GPUs that fully utilize the hardware
Srdja could probably correct me if I am exaggerating unintentionally
Hehe, I think you got to the point well. LC0 runs batches of NN eval via MCTS to utilize the GPU, and even then it may be recommendend to run two threads per GPU.

--
Srdja

schack
Posts: 156
Joined: Thu May 27, 2010 1:32 am
Contact:

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by schack » Tue Aug 18, 2020 7:47 pm

@deberger - what value should I use for NNUEThreshold2 in the updated code after today's "fallback to NNUE" push? 0? Infinite?

[quote=Deberger post_id=855403 time=1596939643 user_id=12155]
[quote=schack post_id=855399 time=1596937188 user_id=5142]
Ok. What's the trivial change to the source code?

[quote=Deberger post_id=855397 time=1596931848 user_id=12155]
[quote=carldaman post_id=855385 time=1596920236 user_id=6402]
It defeats the purpose if the NNUE cannot always be set to active [as an option], because then you're at the mercy of the engine only using NNUE when it chooses. Hopefully the SF team will wake up to the fact that people also use the engine for analysis.
[/quote]

With a trivial change to the source code, you can build your own executable which always uses NNUE.

However this change can never yield stronger analysis.

Over time, using "always NNUE" will be gradually weaker and weaker as the nets are optimized for their intended use in balanced positions.
[/quote]
[/quote]

src/evaluate.cpp:
[code]- constexpr Value NNUEThreshold = Value(520);
+ constexpr Value NNUEThreshold = VALUE_INFINITE;
[/code]
[/quote]

zullil
Posts: 6442
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by zullil » Tue Aug 18, 2020 7:56 pm

schack wrote:
Tue Aug 18, 2020 7:47 pm
@deberger - what value should I use for NNUEThreshold2 in the updated code after today's "fallback to NNUE" push? 0? Infinite?
If you want to have NNUE used for all evaluations, I suggest you get your source code here:

https://github.com/joergoster/Stockfish ... nue_always

Simply click on the green Code button and then Download ZIP.

User avatar
Deberger
Posts: 71
Joined: Sat Nov 02, 2019 5:42 pm
Full name: ɹǝƃɹǝqǝᗡ ǝɔnɹꓭ

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Deberger » Wed Aug 19, 2020 12:15 am

schack wrote:
Tue Aug 18, 2020 7:47 pm
@deberger - what value should I use for NNUEThreshold2 in the updated code after today's "fallback to NNUE" push? 0? Infinite?
src/evaluate.cpp:

Code: Select all

-  constexpr Value NNUEThreshold1  =   Value(550);
+  constexpr Value NNUEThreshold1  =   VALUE_INFINITE;
There's no need to touch NNUEThreshold2 because the logical subexpression (classical && Eval::useNNUE) will always be false

Code: Select all

if (classical && Eval::useNNUE && abs(v) * 16 < NNUEThreshold2 * (16 + pos.rule50_count()))

schack
Posts: 156
Joined: Thu May 27, 2010 1:32 am
Contact:

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by schack » Wed Aug 19, 2020 1:34 am

Thanks! Much obliged.

Post Reply