Official Lc0 0.29.0 released yesterday

Jouni · Post by **Jouni** » Wed Dec 14, 2022 2:33 pm

https://github.com/LeelaChessZero/lc0/r ... ag/v0.29.0

But I haven't found any measurable progress after 0.26.0! It always solves same test positions no improvement with any net. I gave soon up

.

Stephen Ham · Post by **Stephen Ham** » Wed Dec 14, 2022 11:27 pm

Hi Jouni,

Did you just test with the default net or select a different one?

I downloaded iteration 29.0 a couple months ago, but selected a different net for CUDNN. Their NN selection list displays estimated elo increases. However, my gripe is we're not informed whether the elo increase was based upon super-fast speed games or normal TC games. I'm told some NNs are better for speed chess while others offer a qualitative improvement for analysis, but may be slower. As an ICCF GM, I want a Leela that's best for analysis over long periods of time rather than blitz chess. I'm thus willing to wait for a more accurate evaluation and move-selection output.

That said, my human perception is that I'm seeing a small qualitative improvement over Leela 28.2 with this latest iteration and the NN I selected.

All the best,
-Steve-

Stephen Ham · Post by **Stephen Ham** » Thu Dec 15, 2022 7:34 am

Oops! Typo correction: I selected a CUDA NN and not a CUDNN net.

-Steve-

Jouni · Post by **Jouni** » Thu Dec 15, 2022 9:48 am

Default net is small and weak. I tried with 78XXXX and 80XXXX nets of course.

Branko Radovanovic · Post by **Branko Radovanovic** » Thu Dec 15, 2022 9:55 am

The release notes say:

Full attention policy support in cuda, cudnn, metal, onnx, blas, dnnl, and eigen backends.

What is "attention policy"? I've been puzzled by this term since it first appeared in connection with LC0 and I couldn't find even a one-sentence explanation. To me it's just fantastic that anyone would simply say "hey, we have attention policy now, you know", without even attempting to explain what this means.

AAce3 · Post by **AAce3** » Fri Dec 16, 2022 3:22 am

Branko Radovanovic wrote: ↑Thu Dec 15, 2022 9:55 am The release notes say:
Full attention policy support in cuda, cudnn, metal, onnx, blas, dnnl, and eigen backends.
What is "attention policy"? I've been puzzled by this term since it first appeared in connection with LC0 and I couldn't find even a one-sentence explanation. To me it's just fantastic that anyone would simply say "hey, we have attention policy now, you know", without even attempting to explain what this means.

All the stuff about attention policy can be found in the Lc0 discord, if you do some digging.

Attention policy is based on self-attention, a concept that is used in transformers for natural language processing. It's a pretty complicated topic, one that I won't really go into here.

The general idea, however, is to compute policy based on move-from and move-to squares, rather than having the network learn each move individually. The results from the vector produced from the first convolution in the policy head is split into two vectors, the "from" vector and the "to" vector, which are then dotted together. On a high level, the "query" part of the self attention net represents the "from" square, and the "key" represents the "to" square. (I could be completely wrong about this, so anyone who's actually worked with the attention head feel free to correct me.)

This results in faster learning as well as better generalization, as well as slightly faster training time. Hope this helps!

Dariusz · Post by **Dariusz** » Fri Dec 16, 2022 4:43 pm

Lc0 0.29.0 for Mac is stronger by several hundred Elo than its predecessor

https://www.chessengeria.com/post/lc0-0-29-0-released

Congratulations to the Creators !!

Jouni · Post by **Jouni** » Sat Dec 17, 2022 1:19 pm

Pohl test -27 to 0.30.dev

.

Steppenwolf · Post by **Steppenwolf** » Sat Dec 17, 2022 4:58 pm

Hallo Dariusz,

thank you for providing the macOS Community with the binaries of Lc0!

I just downloaded your Intel Version from your Homepage and tested it.
Unfortunately, it worked not for my system (macbook pro with big sur, 11.7.2), especially the metal backend i could not find in the options.

Maybe you have some ideas...I appreciate your help

Dariusz · Post by **Dariusz** » Sat Dec 17, 2022 6:17 pm

Steppenwolf wrote: ↑Sat Dec 17, 2022 4:58 pm Hallo Dariusz,

thank you for providing the macOS Community with the binaries of Lc0!

I just downloaded your Intel Version from your Homepage and tested it.
Unfortunately, it worked not for my system (macbook pro with big sur, 11.7.2), especially the metal backend i could not find in the options.

Maybe you have some ideas...I appreciate your help

Steppenwolf, I will write you some suggestions in a private message. Check back in some time.