Scorpio 2.8.6 MCTS neural network evaluation

[Norbert Raimund Leisner]

Hello,

cf. https://sites.google.com/site/dshawul/home - http://www.sdchess.ru/engines/scorpio_2_8_6_mac_osx.zip

Regards,
Norbert

[Albert Silver]

I should add that one can use Leela weights too. I have not tested beyond seeing they work.

[CMCanavessi]

Do those provided nets (or any other net) work though? How to check if they are working or not?

[Robert Pope]

I should add that one can use Leela weights too. I have not tested beyond seeing they work.

That is very surprising.

[Milos]

I should add that one can use Leela weights too. I have not tested beyond seeing they work.

That is very surprising.

Why would it be surprising? LC0 search is anyway total crap as it A0 search, and ppl currently involved in LC0 development clearly have not much clue how to improve or even only evaluate search performance.

[peter]

I should add that one can use Leela weights too. I have not tested beyond seeing they work.

Could you give it a try, Albert?
That sounds very interesting to me!

[Robert Pope]

I should add that one can use Leela weights too. I have not tested beyond seeing they work.

That is very surprising.

Why would it be surprising? LC0 search is anyway total crap as it A0 search, and ppl currently involved in LC0 development clearly have not much clue how to improve or even only evaluate search performance.

Because the weights are tied to the specific neural network implementation format of LC0. In order for them to function in another engine, that engine has to have the exact same input planes, with the same orientation, in the exact same order, in the exact same format. Otherwise, it would be like porting Stockfish evaluation weights into another engine, except you are assigning them at random. Not even all LC0 neural nets are compatible across versions of LC Zero.

So for them to actually work in another engine is either a mind-boggling coincidence, or intentional copying of the format (not that there's anything wrong with that [/seinfeld]). IIRC, ScorpioMCTS came first, so it would be surprising to find that its neural net layout is exactly the same.

[Guenther]

I should add that one can use Leela weights too. I have not tested beyond seeing they work.

That is very surprising.

I highly doubt this too. For me the Scorpio NN files are completely different.

[Milos]

I should add that one can use Leela weights too. I have not tested beyond seeing they work.

That is very surprising.

Why would it be surprising? LC0 search is anyway total crap as it A0 search, and ppl currently involved in LC0 development clearly have not much clue how to improve or even only evaluate search performance.

Because the weights are tied to the specific neural network implementation format of LC0. In order for them to function in another engine, that engine has to have the exact same input planes, with the same orientation, in the exact same order, in the exact same format. Otherwise, it would be like porting Stockfish evaluation weights into another engine, except you are assigning them at random. Not even all LC0 neural nets are compatible across versions of LC Zero.

So for them to actually work in another engine is either a mind-boggling coincidence, or intentional copying of the format (not that there's anything wrong with that [/seinfeld]). IIRC, ScorpioMCTS came first, so it would be surprising to find that its neural net layout is exactly the same.

I don't think you understand how MCTS works. In Scorpio it is using just value network as direct rollout replacement (i.e. as eval). Policy network is not used to guide the search. MCTS is completely agnostic to the way evaluation only is performed (A/B rollout, handcrafted NN eval, LC0/A0 NN eval, etc.). Format, number of layers, input planes of NN and all the internals are irrelevant for search.
How efficient it is, is another question. But knowing that UCT in LC0/A0 is not very efficient it should not be too difficult to make search that would work better using the same NN from LC0/A0 for eval only.
And I am not buying this crap that policy network is enough to make efficient search with UCT from A0. Sorry, but it doesn't work that way, just because some Google guys published some crappy shady preprint.

[Robert Pope]

I don't think you understand how MCTS works. In Scorpio it is using just value network as direct rollout replacement (i.e. as eval). Policy network is not used to guide the search. MCTS is completely agnostic to the way evaluation only is performed (A/B rollout, handcrafted NN eval, LC0/A0 NN eval, etc.). Format, number of layers, input planes of NN and all the internals are irrelevant for search.

Except....the value network is an output of the neural network, and is simply a series of transformations of the inputs and the weights. So of course the format of the neural network matters! If you don't have the same format, then the value network that gets output is junk. How MCTS uses that to guide the search is what is irrelevant, since it's garbage information.