Eduard wrote: ↑Sat Jan 14, 2023 6:51 pmAs far as I know, the new nets are trained with more and more Lc0 games. As a result, Stockfish's playing style adapts more and more to Lc0. Too bad that the original (namely Lc0) can play this style better. The new nets are making the big cat SF more and more a tame cat. I noticed this and threw the new nets out of my engine some time ago.
Hasn't Lc0 recently improved, or have I misremembered? Didn't people say that version 0.29, the 13-Dec-22 release (32 days ago), was an impressive jump?
The leela team has been getting a lot of value out of the attention mechanism. There have been several extremely impressive jumps in the neural network architecture, with more coming on the horizon.
Full attention policy support in cuda, cudnn, metal, onnx, blas, dnnl, and eigen backends.
Partial attention policy support in onednn backend (good enough for T79).
The default net is now 791556 for most backends except opencl and dx12 that get 753723 (as they lack attention policy support).
One of the first fruits of the ongoing experimentation with new neural network architectures is the already mentioned attention policy. The policy output of the neural net, for a given position, gives the probability each legal move is the best one. Attention policy is a new way to calculate this, giving more accurate results, but requires modifications of the code that does the neural net computation. This code is broken down in several backends, each one targeting a specific method of computation, either specific gpus or external libraries. Unfortunately, only a few members of our team are able to do these performance critical bits of code and had limited time available, so it took a while to get everything working. We have, finally, solutions that cover most of our users and an approach that can hopefully allow us to have support for new architectures a lot quicker - more on that later.
Currently all recent network series (T78, T79 and T80) utilize attention policy, meaning that the backends that haven’t been updated (dx12 and opencl) do not work with those nets. Further, the T78 nets require some of the optional attention policy features and as such are usually the last to be supported: the onednn backend is not yet at this point, but can be used with all other network architectures.
Could also be that the GPU's or the hardware that runs Lc0 has reached a point where it can play much stronger. I remember watching live some of the older SF vs Lc0 games and Lc0 strength would get dramatically poorer when time became a factor. SF would always seem to play without much drop in strength in time trouble. So maybe the improvement in hardware has helped Lc0 more than it has helped SF.
GPU hardware is improving at a much faster rate than CPU it seems. So the future of Lc0 is looking good, apart from the huge power consumption of the high-end cards.
Modern Times wrote: ↑Sun Jan 15, 2023 10:40 pm
GPU hardware is improving at a much faster rate than CPU it seems. So the future of Lc0 is looking good, apart from the huge power consumption of the high-end cards.
Use MacBook Pro M1 MAX 32 GPU cores. Or soon 64 GPU cores.
