LCzero sacs a knight for nothing

[Milos]

Seriously, equivalent power use is a fair metric. Otherwise what would be the point of improving hardware at all? If Alpha-Beta crunchers find a good way to use modern GPUs they should implement these by all means...

Equivalent power is totally BS metric. Two architectures that have almost nothing in common are compared based on power consumption.
in OPS/W even Arduino's are killing Intel CPUs.
The only even remotely fair metric is performance per buck.
And there CPU still dominates both GPUs and TPUs.

[Milos]

Not sure what you're talking about here. Leela (as in Leela 0.11) certainly has tactical weaknesses, but that's an MCTS engine with a neural net, not a pure NN engine like Leela Zero.

And while Leela Zero may still have some tactical vulnerabilities, they're getting really hard to exploit, certainly for humans.

Agreed that policy guided search has some similarity to Alpha-Beta on a mature, larger neural net.

Cut the crap. Return here when LC0 network alone (single playout) is able to beat SF depth 1 search.
I'm pretty confident that will not happen any time soon, especially if you don't increase the size of NN.

[Michel]

I am not so sure actually. If you run SF purely on evaluation (i.e. no quiescence search) then I suspect LC0 will do better. It would be an interesting experiment.

[Milos]

I am not so sure actually. If you run SF purely on evaluation (i.e. no quiescence search) then I suspect LC0 will do better. It would be an interesting experiment.

Lol, why would you run SF without QS???
Just to make LC0 eval look good?
QS takes less than 100 nodes of average and is executed 100x faster than single NN eval on 1080Ti.
You are just burring your head in the sand. Nothing else.

[jkiliani]

Not sure what you're talking about here. Leela (as in Leela 0.11) certainly has tactical weaknesses, but that's an MCTS engine with a neural net, not a pure NN engine like Leela Zero.

And while Leela Zero may still have some tactical vulnerabilities, they're getting really hard to exploit, certainly for humans.

Agreed that policy guided search has some similarity to Alpha-Beta on a mature, larger neural net.

Cut the crap. Return here when LC0 network alone (single playout) is able to beat SF depth 1 search.
I'm pretty confident that will not happen any time soon, especially if you don't increase the size of NN.

Just tested exactly that, with Id 150, against Stockfish with fixed depth 1:

./cutechess-cli -rounds 400 -tournament round-robin -concurrency 2 -pgnout results_tuning.pgn \
-engine name=Id_152 cmd=lczero_tunenew2 arg="--threads=1" arg="--weights=$WDR/weights_152.txt" arg="--noponder" nodes=1 tc=inf\
-engine name=sf_d1 cmd=stockfish_x86-64 option.Threads=1 depth=1 tc=inf \
-each proto=uci

Result: 1-1-0. Obviously I ran more games than two, but it turns out that both Stockfish and Lc0 are deterministic at these settings, so the end result was 200-200-0.

Unless you can come up with a way to make Stockfish non-deterministic at fixed depth 1, I consider this point now proven.

Edit: Id 150 actually wins with 1-0-1, Id 129 also scores equal with 1-1-0, only Id 125 loses with 0-1-1.

So in total, we have Lc0 performing comparably to Stockfish Depth 1. Any further questions?

[Michel]

If you allow quiescence search then it is not purely a depth one search.

[Michel]

Unless you can come up with a way to make Stockfish non-deterministic at fixed depth 1, I consider this point now proven.

Use a book.

EDIT: BTW SF was not really designed run at depth one (even with QS). It does extremely heavy pruning at low depths so it will probably drop pieces (it is known that it does so up to depth 5).

[Milos]

If you allow quiescence search then it is not purely a depth one search.

Lol, so now is the matter of definition .
Just keep burring your head. Or another challenge for you.
Please come back when LC0 with 100 playouts is able to beat SFdev depth 8 search.
I can tell you the score with ID150. SFdev depth 8 search gets around 90% against 100 playouts LC0 .

[Milos]

Unless you can come up with a way to make Stockfish non-deterministic at fixed depth 1, I consider this point now proven.

Use a book.

Or your precious dirichlet noise parameter . Do you know it even exists?

[jkiliani]

Unless you can come up with a way to make Stockfish non-deterministic at fixed depth 1, I consider this point now proven.

Use a book.

Or your precious dirichlet noise parameter . Do you know it even exists?

I know more than you think, and fyi --noise doesn't add enough variability to alter the PV in almost all cases by now. I tried the match with noise, result was exactly the same.

Your turn to produce data now.