TalkChess.com

Progress is still being made! It's been much tougher than anticipated, but we are only 5 elo from the best SV net now. That SV net was then optimized using SPSA to produce the best master net, so still about 15 elo from master. Best results from @vondele (who has been training up a storm): # PLAYER ...

xr_a_y wrote: ↑

Thu Nov 26, 2020 7:55 pm

But current official or master nets has been built with data of what depth ?

I believe Sergio used depth 12-14 data, with about 10 billion positions.

Another quick update on some experiments: - Tried training from scratch using 1B d20 games, and it was disastrous. ELO was far worse than the d5 positions. So, interestingly, it appears that depth is just way too deep for the network to learn on. - vondele validated that you can train a net equal to...

So I am curious how did you resolve the "qsearch inference" issue above? Did you re-implement the Stockfish qsearch in python? Let's say suppose we did not use the qsearch to do inference but directly call NNUE evaluation over the sfen position, do you think that will make a big difference on learn...

So a question, which is not really per this threads name, but was touched on here. What efforts can be taken, during training when using floats, to try to map the weights and biases into a smaller range. IE, get most of the weights within [-2.0, 2.0], such that quantization can be done with minimal...

We've been trying a bunch of experiments, but not a huge amount of forward progress. We are at about -80 elo from master, but still a long ways away. This is all still training on d5 data though, so I'm hopeful that higher depth data will be a big win (unfortunately, it takes a really long time to g...

I started an implementation of the SF NNUE training in Pytorch: https://github.com/glinscott/nnue-pytorch . Perhaps a silly question, but I'm having a hard time understanding this line in the step_ function https://github.com/glinscott/nnue-pytorch/blob/9e4b6b2edd4155b1ed87c7a03d7d1bbf08583df8/mode...

thank you for your explanation and code let me recap what I have understood on factorizer so far: 1) you have 2 different network topologies, 1 one for training (TNNUE) and 1 for the engine (ENNUE) 2) you have a trick when saving the weight in the trainer so TNNUE and ENNUE have the same output for...

So I wrote an NNTraining tool in C, made it very well threaded, cranked up optimizations, wrote AVX2 code for portions to perform weight updates faster, came up with a novel addition to Adam to limit throughput when dealing with sparse inputs by batching in fixed sizes, and refusing to shuffle intr...

It's a method to augment/generalize the data the net is training on. Since halfkp splits out piece positions by king position, the net loses some generalization. The factorizer adds those "simple" features (eg. pure piece position, not dependent on king) during training, and then sums up all the va...