This sounds well but I would be curious to know the opinion of the authors of Leela.chrisw wrote: ↑Wed Apr 24, 2019 3:32 pmI would assume what is happening (I didn’t implement hash yet), is that if a new node matches with a hash table entry, the new node gets given the win and visit count from the hash, and backs that (win,visits) entry back to the root. That root move gets a higher visit count as a result even though it didn’t actually make the visits.corres wrote: ↑Tue Apr 23, 2019 7:05 amDuring the tests there were some cases when Hasfull reached 1000. In these cases the value of nps stopped to grow but the nps was not reduced.Albert Silver wrote: ↑Mon Apr 22, 2019 7:36 pmJust a sidenote: the NNcache can lead to somewhat misleading nodes per second counts. If you benchmark with a large value, and it does not fill it before the end, you will get significant NPS results, but they will not represent your actual speed throughout the game. Once filled, the NPS will drop a lot, and that is actually the speed you will get for the rest of a game.
I tried NNCacheSize=20000000 but lc0 got fried.
I do not know the developers of LC0 why set such a low value of nncachesize for default.
Have you any information about the dimension of nncachesize and minibatchsize?
And where they are: in the RAM or in the VRAM?
Developers of LC0 back some information from us, what is a pity thing.
But, the line that put the original data (wins, visits) into the hash was also backed up at the time, and its (wins,visits) will also be represented at the root by its root move.
Then LC0 computes node-count by adding up the root move visits. Hence the double counting. Anyway, if this theory is correct, a non-doubles node-count could be done at the point of summing the root visits, by subtracting the given hash visits. nodecount=oldnodecount minus hashhits.
So, why in this scenario does the double counting stop when hash is full? Because the hash entries are still there and you’ld assume other hits from other nodes will still take place? Mystery.
Another thought, when backpropagating (wins,visits), if a node on the backprop path is in the hash table, presumably the hash entry should get updated at the same time.
The are very taciturn peoples.