Search found 157 matches

by Fabio Gobbato
Fri Oct 23, 2020 3:38 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Train a neural network evaluation
Replies: 15
Views: 3556

Re: Train a neural network evaluation

How do you choose the positions for the training? Is it better to use only quiet positions?
by Fabio Gobbato
Thu Sep 10, 2020 11:22 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Neural network quantization
Replies: 11
Views: 1847

Re: Neural network quantization

I have tried to use quantized weights in the training, only when I calculate the error of the network and seems to work. I have to try with int8 but with int32 works well.
Thank you!
by Fabio Gobbato
Tue Sep 08, 2020 7:24 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Neural network quantization
Replies: 11
Views: 1847

Re: Neural network quantization

But the net calculations would be all on floating point values while stockfish use only integer arithmetic. It's an advantage to use only ints because the net calculation is faster. But I miss something because while training the net the weights differs a lot and int8_t gives low accuracy. But maybe...
by Fabio Gobbato
Tue Sep 08, 2020 1:29 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Neural network quantization
Replies: 11
Views: 1847

Re: Neural network quantization

But if I have understood correctly in this way you get an index for an array but the weights of the neural net are multiplied with the input and added together. If I have understood stockfish use a fixed point arithmetic because the output is shifted by 6 places so is like a division by 64. If it's ...
by Fabio Gobbato
Tue Sep 08, 2020 8:31 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Neural network quantization
Replies: 11
Views: 1847

Neural network quantization

I have a neural network with floating point weights and I want them to fit in int8_t . The weights goes from +64 and -57 and some are very low like 1e-5. How can I translate all these values in int8_t without losing accuracy? Is there a way to train the net to easily translate the weights to int8_t?...
by Fabio Gobbato
Wed Sep 02, 2020 8:56 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Train a neural network evaluation
Replies: 15
Views: 3556

Re: Train a neural network evaluation

I am very familiar with Leela type networks and far less so with the NNUE nets. That said, your LR seems extremely low for initial training (although no batch size was mentioned). I suggest asking in the SF-NNUE Discord but I'm not sure how joining that works. I update the weights after every sampl...
by Fabio Gobbato
Tue Sep 01, 2020 6:29 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Train a neural network evaluation
Replies: 15
Views: 3556

Re: Train a neural network evaluation

Is unclear the range of centipawn values your training and test sets have, but, if you were training on win probability (0.0 to 1.0 range) then a completely untrained (random) net is going to come back with an average “error” of about 0.3 which could well translate to your 200 cp. So it’s entirely ...
by Fabio Gobbato
Tue Sep 01, 2020 4:00 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Train a neural network evaluation
Replies: 15
Views: 3556

Re: Train a neural network evaluation

More information about the network architecture and hyper-parameters (learning rate, etc) might provide some clues. Just a quick stab, but depth 4 does not seem very deep relative to what the SF-NNUE Discord posts talk about. It might not be enough improvement from your current eval. I'm trying a s...
by Fabio Gobbato
Tue Sep 01, 2020 12:25 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Train a neural network evaluation
Replies: 15
Views: 3556

Train a neural network evaluation

I'm implementing a neural network that should replace the evaluation of my engine. I've made a dataset of 200M fens with associated the score of a 4 depth search of my engine. For the optimization algorithm I have used Adagrad and Gradient descent, the loss function is the sum squared error. I have ...
by Fabio Gobbato
Fri Aug 21, 2020 9:56 am
Forum: Computer Chess Club: General Topics
Topic: New release Pedone 2.1
Replies: 6
Views: 1915

Re: New release Pedone 2.1

Neural nets give a better evaluation than an handcrafted one. I'm going to try to implement my own neural network with my own training method so the evaluations will be different from other implementations. Improve the evaluation is very difficult and if I can replace it with a better one I will be ...