Search found 562 matches
- Sat Jan 09, 2021 5:17 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: Pytorch NNUE training
- Replies: 61
- Views: 13002
Re: Pytorch NNUE training
Progress is still being made! It's been much tougher than anticipated, but we are only 5 elo from the best SV net now. That SV net was then optimized using SPSA to produce the best master net, so still about 15 elo from master. Best results from @vondele (who has been training up a storm): # PLAYER ...
- Fri Nov 27, 2020 4:20 am
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: Pytorch NNUE training
- Replies: 61
- Views: 13002
- Thu Nov 26, 2020 6:02 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: Pytorch NNUE training
- Replies: 61
- Views: 13002
Re: Pytorch NNUE training
Another quick update on some experiments: - Tried training from scratch using 1B d20 games, and it was disastrous. ELO was far worse than the d5 positions. So, interestingly, it appears that depth is just way too deep for the network to learn on. - vondele validated that you can train a net equal to...
- Wed Nov 25, 2020 5:00 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: Pytorch NNUE training
- Replies: 61
- Views: 13002
Re: Pytorch NNUE training
So I am curious how did you resolve the "qsearch inference" issue above? Did you re-implement the Stockfish qsearch in python? Let's say suppose we did not use the qsearch to do inference but directly call NNUE evaluation over the sfen position, do you think that will make a big difference on learn...
- Wed Nov 25, 2020 4:58 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: Pytorch NNUE training
- Replies: 61
- Views: 13002
Re: Pytorch NNUE training
So a question, which is not really per this threads name, but was touched on here. What efforts can be taken, during training when using floats, to try to map the weights and biases into a smaller range. IE, get most of the weights within [-2.0, 2.0], such that quantization can be done with minimal...
- Sun Nov 22, 2020 3:17 am
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: Pytorch NNUE training
- Replies: 61
- Views: 13002
Re: Pytorch NNUE training
We've been trying a bunch of experiments, but not a huge amount of forward progress. We are at about -80 elo from master, but still a long ways away. This is all still training on d5 data though, so I'm hopeful that higher depth data will be a big win (unfortunately, it takes a really long time to g...
- Fri Nov 20, 2020 2:26 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: Pytorch NNUE training
- Replies: 61
- Views: 13002
Re: Pytorch NNUE training
I started an implementation of the SF NNUE training in Pytorch: https://github.com/glinscott/nnue-pytorch . Perhaps a silly question, but I'm having a hard time understanding this line in the step_ function https://github.com/glinscott/nnue-pytorch/blob/9e4b6b2edd4155b1ed87c7a03d7d1bbf08583df8/mode...
- Thu Nov 19, 2020 3:19 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: Pytorch NNUE training
- Replies: 61
- Views: 13002
Re: Pytorch NNUE training
thank you for your explanation and code let me recap what I have understood on factorizer so far: 1) you have 2 different network topologies, 1 one for training (TNNUE) and 1 for the engine (ENNUE) 2) you have a trick when saving the weight in the trainer so TNNUE and ENNUE have the same output for...
- Thu Nov 19, 2020 9:39 am
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: You've trained a brilliant NN(UE) King-Piece Network. Now what?
- Replies: 22
- Views: 4299
Re: You've trained a brilliant NN(UE) King-Piece Network. Now what?
So I wrote an NNTraining tool in C, made it very well threaded, cranked up optimizations, wrote AVX2 code for portions to perform weight updates faster, came up with a novel addition to Adam to limit throughput when dealing with sparse inputs by batching in fixed sizes, and refusing to shuffle intr...
- Wed Nov 18, 2020 4:38 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: Pytorch NNUE training
- Replies: 61
- Views: 13002
Re: Pytorch NNUE training
It's a method to augment/generalize the data the net is training on. Since halfkp splits out piece positions by king position, the net loses some generalization. The factorizer adds those "simple" features (eg. pure piece position, not dependent on king) during training, and then sums up all the va...