Scorpio 2.8.6 MCTS neural network evaluation

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.

Albert Silver
Posts: 2759
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: Scorpio 2.8.6 MCTS neural network evaluation

Post by Albert Silver » Mon Jul 23, 2018 3:39 pm

I should add that one can use Leela weights too. I have not tested beyond seeing they work.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

User avatar
CMCanavessi
Posts: 691
Joined: Thu Dec 28, 2017 3:06 pm
Location: Argentina

Re: Scorpio 2.8.6 MCTS neural network evaluation

Post by CMCanavessi » Mon Jul 23, 2018 9:02 pm

Do those provided nets (or any other net) work though? How to check if they are working or not?
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls

Robert Pope
Posts: 424
Joined: Sat Mar 25, 2006 7:27 pm

Re: Scorpio 2.8.6 MCTS neural network evaluation

Post by Robert Pope » Tue Jul 24, 2018 2:02 am

Albert Silver wrote:
Mon Jul 23, 2018 3:39 pm
I should add that one can use Leela weights too. I have not tested beyond seeing they work.
That is very surprising.

Milos
Posts: 3174
Joined: Wed Nov 25, 2009 12:47 am

Re: Scorpio 2.8.6 MCTS neural network evaluation

Post by Milos » Tue Jul 24, 2018 2:59 am

Robert Pope wrote:
Tue Jul 24, 2018 2:02 am
Albert Silver wrote:
Mon Jul 23, 2018 3:39 pm
I should add that one can use Leela weights too. I have not tested beyond seeing they work.
That is very surprising.
Why would it be surprising? LC0 search is anyway total crap as it A0 search, and ppl currently involved in LC0 development clearly have not much clue how to improve or even only evaluate search performance.

peter
Posts: 1520
Joined: Sat Feb 16, 2008 6:38 am

Re: Scorpio 2.8.6 MCTS neural network evaluation

Post by peter » Tue Jul 24, 2018 5:44 am

Albert Silver wrote:
Mon Jul 23, 2018 3:39 pm
I should add that one can use Leela weights too. I have not tested beyond seeing they work.
Could you give it a try, Albert?
That sounds very interesting to me!
Peter Martan

Robert Pope
Posts: 424
Joined: Sat Mar 25, 2006 7:27 pm

Re: Scorpio 2.8.6 MCTS neural network evaluation

Post by Robert Pope » Tue Jul 24, 2018 3:36 pm

Milos wrote:
Tue Jul 24, 2018 2:59 am
Robert Pope wrote:
Tue Jul 24, 2018 2:02 am
Albert Silver wrote:
Mon Jul 23, 2018 3:39 pm
I should add that one can use Leela weights too. I have not tested beyond seeing they work.
That is very surprising.
Why would it be surprising? LC0 search is anyway total crap as it A0 search, and ppl currently involved in LC0 development clearly have not much clue how to improve or even only evaluate search performance.
Because the weights are tied to the specific neural network implementation format of LC0. In order for them to function in another engine, that engine has to have the exact same input planes, with the same orientation, in the exact same order, in the exact same format. Otherwise, it would be like porting Stockfish evaluation weights into another engine, except you are assigning them at random. Not even all LC0 neural nets are compatible across versions of LC Zero.

So for them to actually work in another engine is either a mind-boggling coincidence, or intentional copying of the format (not that there's anything wrong with that [/seinfeld]). IIRC, ScorpioMCTS came first, so it would be surprising to find that its neural net layout is exactly the same.

User avatar
Guenther
Posts: 2412
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Scorpio 2.8.6 MCTS neural network evaluation

Post by Guenther » Tue Jul 24, 2018 4:14 pm

Robert Pope wrote:
Tue Jul 24, 2018 2:02 am
Albert Silver wrote:
Mon Jul 23, 2018 3:39 pm
I should add that one can use Leela weights too. I have not tested beyond seeing they work.
That is very surprising.
I highly doubt this too. For me the Scorpio NN files are completely different.

Milos
Posts: 3174
Joined: Wed Nov 25, 2009 12:47 am

Re: Scorpio 2.8.6 MCTS neural network evaluation

Post by Milos » Tue Jul 24, 2018 5:12 pm

Robert Pope wrote:
Tue Jul 24, 2018 3:36 pm
Milos wrote:
Tue Jul 24, 2018 2:59 am
Robert Pope wrote:
Tue Jul 24, 2018 2:02 am
Albert Silver wrote:
Mon Jul 23, 2018 3:39 pm
I should add that one can use Leela weights too. I have not tested beyond seeing they work.
That is very surprising.
Why would it be surprising? LC0 search is anyway total crap as it A0 search, and ppl currently involved in LC0 development clearly have not much clue how to improve or even only evaluate search performance.
Because the weights are tied to the specific neural network implementation format of LC0. In order for them to function in another engine, that engine has to have the exact same input planes, with the same orientation, in the exact same order, in the exact same format. Otherwise, it would be like porting Stockfish evaluation weights into another engine, except you are assigning them at random. Not even all LC0 neural nets are compatible across versions of LC Zero.

So for them to actually work in another engine is either a mind-boggling coincidence, or intentional copying of the format (not that there's anything wrong with that [/seinfeld]). IIRC, ScorpioMCTS came first, so it would be surprising to find that its neural net layout is exactly the same.
I don't think you understand how MCTS works. In Scorpio it is using just value network as direct rollout replacement (i.e. as eval). Policy network is not used to guide the search. MCTS is completely agnostic to the way evaluation only is performed (A/B rollout, handcrafted NN eval, LC0/A0 NN eval, etc.). Format, number of layers, input planes of NN and all the internals are irrelevant for search.
How efficient it is, is another question. But knowing that UCT in LC0/A0 is not very efficient it should not be too difficult to make search that would work better using the same NN from LC0/A0 for eval only.
And I am not buying this crap that policy network is enough to make efficient search with UCT from A0. Sorry, but it doesn't work that way, just because some Google guys published some crappy shady preprint.

Robert Pope
Posts: 424
Joined: Sat Mar 25, 2006 7:27 pm

Re: Scorpio 2.8.6 MCTS neural network evaluation

Post by Robert Pope » Tue Jul 24, 2018 5:52 pm

Milos wrote:
Tue Jul 24, 2018 5:12 pm
I don't think you understand how MCTS works. In Scorpio it is using just value network as direct rollout replacement (i.e. as eval). Policy network is not used to guide the search. MCTS is completely agnostic to the way evaluation only is performed (A/B rollout, handcrafted NN eval, LC0/A0 NN eval, etc.). Format, number of layers, input planes of NN and all the internals are irrelevant for search.
Except....the value network is an output of the neural network, and is simply a series of transformations of the inputs and the weights. So of course the format of the neural network matters! If you don't have the same format, then the value network that gets output is junk. How MCTS uses that to guide the search is what is irrelevant, since it's garbage information.

Post Reply