Re: Lc0 51010
Posted: Sun Mar 31, 2019 6:53 pm
NN 51010 has had 559441 games. The NN size is smaller (10x128) than the A0 one and 41xxx NNs (20x256), but the training param is visits=10000 (visits=800 for 41xxx).lkaufman wrote: ↑Sun Mar 31, 2019 6:07 amOne question: does anyone know roughly how many 2080s would be needed to duplicate the training that this 51xxx series has averaged so far? It doesn't mean much to say that it has trained for two days without stating what the average resources used for the training were. I imagine that they were just a tiny percentage of the resources used to train AlphaZero in 9 hours or so.
I'll let others say what 2080 time that works out to.
I will try to estimate (writing on the phone, I am on a vacation ). About 0.2s/move on 2080, meaning games in about 20s. About 200 games per hour, so half a million games need 2500 hours, or about 100 days needed for training the NN 51010 on a single 2080. Or 100x 2080 GPUs needed to train it in one day.
The main unknown is the time to get 10000 visits, I took it as 0.2s on 2080. So this calculation is just for the order of magnitude.
Thanks. So the training resources were a lot more than I thought they were. It makes the amazing results at least easier to accept.
I will repeat: the average nodes per visit in this training run is around 800 nodes, not 10 thousand. It is a complete misunderstanding of what is being done. The average nodes per move per game is around 800 still.
Ok, then I would estimate a game on 2080 to take some 6 seconds. Meaning 30 days of 2080 training for half a million games, or 30x 2080 GPUs training for one day.
Much closer to what I imagined.
This is the problem with the "NN Concept" It is always "Best" in chess engine tourneys to try and Limit the GUI opening book that way you are testing engine overall strength and not just playing "opening book moves" Found this a problem with Lokasoft, Chess genius and the Arena GUi's.. found a workaround by using the engine opening books (as long as they were not too large) Cerebellum books caused quite a stir for this very reason.(too large) most engine games go "out of book" quite early of left to their own devices, so if a few engines(2-3) had 'different' Cerebellum books, not much of a issue..big problem if they all used the same book. Huge Neural Nets using Milions & Millions of Games as a "basis for play" creates a Big Big Problem. CMC chess engine I remember used the entire "Week In Chess Database" as it's opening book..To me that was a problem...(best to limit books 7-8-9 moves)..AR