Search found 562 matches

by gladius
Sat Jan 09, 2021 5:17 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Pytorch NNUE training
Replies: 61
Views: 13002

Re: Pytorch NNUE training

Progress is still being made! It's been much tougher than anticipated, but we are only 5 elo from the best SV net now. That SV net was then optimized using SPSA to produce the best master net, so still about 15 elo from master. Best results from @vondele (who has been training up a storm): # PLAYER ...
by gladius
Fri Nov 27, 2020 4:20 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Pytorch NNUE training
Replies: 61
Views: 13002

Re: Pytorch NNUE training

xr_a_y wrote:
Thu Nov 26, 2020 7:55 pm
But current official or master nets has been built with data of what depth ?
I believe Sergio used depth 12-14 data, with about 10 billion positions.
by gladius
Thu Nov 26, 2020 6:02 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Pytorch NNUE training
Replies: 61
Views: 13002

Re: Pytorch NNUE training

Another quick update on some experiments: - Tried training from scratch using 1B d20 games, and it was disastrous. ELO was far worse than the d5 positions. So, interestingly, it appears that depth is just way too deep for the network to learn on. - vondele validated that you can train a net equal to...
by gladius
Wed Nov 25, 2020 5:00 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Pytorch NNUE training
Replies: 61
Views: 13002

Re: Pytorch NNUE training

So I am curious how did you resolve the "qsearch inference" issue above? Did you re-implement the Stockfish qsearch in python? Let's say suppose we did not use the qsearch to do inference but directly call NNUE evaluation over the sfen position, do you think that will make a big difference on learn...
by gladius
Wed Nov 25, 2020 4:58 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Pytorch NNUE training
Replies: 61
Views: 13002

Re: Pytorch NNUE training

So a question, which is not really per this threads name, but was touched on here. What efforts can be taken, during training when using floats, to try to map the weights and biases into a smaller range. IE, get most of the weights within [-2.0, 2.0], such that quantization can be done with minimal...
by gladius
Sun Nov 22, 2020 3:17 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Pytorch NNUE training
Replies: 61
Views: 13002

Re: Pytorch NNUE training

We've been trying a bunch of experiments, but not a huge amount of forward progress. We are at about -80 elo from master, but still a long ways away. This is all still training on d5 data though, so I'm hopeful that higher depth data will be a big win (unfortunately, it takes a really long time to g...
by gladius
Fri Nov 20, 2020 2:26 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Pytorch NNUE training
Replies: 61
Views: 13002

Re: Pytorch NNUE training

I started an implementation of the SF NNUE training in Pytorch: https://github.com/glinscott/nnue-pytorch . Perhaps a silly question, but I'm having a hard time understanding this line in the step_ function https://github.com/glinscott/nnue-pytorch/blob/9e4b6b2edd4155b1ed87c7a03d7d1bbf08583df8/mode...
by gladius
Thu Nov 19, 2020 3:19 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Pytorch NNUE training
Replies: 61
Views: 13002

Re: Pytorch NNUE training

thank you for your explanation and code let me recap what I have understood on factorizer so far: 1) you have 2 different network topologies, 1 one for training (TNNUE) and 1 for the engine (ENNUE) 2) you have a trick when saving the weight in the trainer so TNNUE and ENNUE have the same output for...
by gladius
Thu Nov 19, 2020 9:39 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: You've trained a brilliant NN(UE) King-Piece Network. Now what?
Replies: 22
Views: 4299

Re: You've trained a brilliant NN(UE) King-Piece Network. Now what?

So I wrote an NNTraining tool in C, made it very well threaded, cranked up optimizations, wrote AVX2 code for portions to perform weight updates faster, came up with a novel addition to Adam to limit throughput when dealing with sparse inputs by batching in fixed sizes, and refusing to shuffle intr...
by gladius
Wed Nov 18, 2020 4:38 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Pytorch NNUE training
Replies: 61
Views: 13002

Re: Pytorch NNUE training

It's a method to augment/generalize the data the net is training on. Since halfkp splits out piece positions by king position, the net loses some generalization. The factorizer adds those "simple" features (eg. pure piece position, not dependent on king) during training, and then sums up all the va...