Search found 156 matches

by Fabio Gobbato
Thu Sep 10, 2020 11:22 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Neural network quantization
Replies: 11
Views: 563

Re: Neural network quantization

I have tried to use quantized weights in the training, only when I calculate the error of the network and seems to work. I have to try with int8 but with int32 works well.
Thank you!
by Fabio Gobbato
Tue Sep 08, 2020 7:24 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Neural network quantization
Replies: 11
Views: 563

Re: Neural network quantization

But the net calculations would be all on floating point values while stockfish use only integer arithmetic. It's an advantage to use only ints because the net calculation is faster. But I miss something because while training the net the weights differs a lot and int8_t gives low accuracy. But maybe...
by Fabio Gobbato
Tue Sep 08, 2020 1:29 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Neural network quantization
Replies: 11
Views: 563

Re: Neural network quantization

But if I have understood correctly in this way you get an index for an array but the weights of the neural net are multiplied with the input and added together. If I have understood stockfish use a fixed point arithmetic because the output is shifted by 6 places so is like a division by 64. If it's ...
by Fabio Gobbato
Tue Sep 08, 2020 8:31 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Neural network quantization
Replies: 11
Views: 563

Neural network quantization

I have a neural network with floating point weights and I want them to fit in int8_t . The weights goes from +64 and -57 and some are very low like 1e-5. How can I translate all these values in int8_t without losing accuracy? Is there a way to train the net to easily translate the weights to int8_t?...
by Fabio Gobbato
Wed Sep 02, 2020 8:56 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Train a neural network evaluation
Replies: 13
Views: 769

Re: Train a neural network evaluation

I am very familiar with Leela type networks and far less so with the NNUE nets. That said, your LR seems extremely low for initial training (although no batch size was mentioned). I suggest asking in the SF-NNUE Discord but I'm not sure how joining that works. I update the weights after every sampl...
by Fabio Gobbato
Tue Sep 01, 2020 6:29 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Train a neural network evaluation
Replies: 13
Views: 769

Re: Train a neural network evaluation

Is unclear the range of centipawn values your training and test sets have, but, if you were training on win probability (0.0 to 1.0 range) then a completely untrained (random) net is going to come back with an average “error” of about 0.3 which could well translate to your 200 cp. So it’s entirely ...
by Fabio Gobbato
Tue Sep 01, 2020 4:00 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Train a neural network evaluation
Replies: 13
Views: 769

Re: Train a neural network evaluation

More information about the network architecture and hyper-parameters (learning rate, etc) might provide some clues. Just a quick stab, but depth 4 does not seem very deep relative to what the SF-NNUE Discord posts talk about. It might not be enough improvement from your current eval. I'm trying a s...
by Fabio Gobbato
Tue Sep 01, 2020 12:25 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Train a neural network evaluation
Replies: 13
Views: 769

Train a neural network evaluation

I'm implementing a neural network that should replace the evaluation of my engine. I've made a dataset of 200M fens with associated the score of a 4 depth search of my engine. For the optimization algorithm I have used Adagrad and Gradient descent, the loss function is the sum squared error. I have ...
by Fabio Gobbato
Fri Aug 21, 2020 9:56 am
Forum: Computer Chess Club: General Topics
Topic: New release Pedone 2.1
Replies: 6
Views: 1401

Re: New release Pedone 2.1

Neural nets give a better evaluation than an handcrafted one. I'm going to try to implement my own neural network with my own training method so the evaluations will be different from other implementations. Improve the evaluation is very difficult and if I can replace it with a better one I will be ...
by Fabio Gobbato
Thu Aug 20, 2020 6:34 pm
Forum: Computer Chess Club: General Topics
Topic: New release Pedone 2.1
Replies: 6
Views: 1401

New release Pedone 2.1

I've just released a new version of my engine. These are the changes: Speed optimizations Tuning Bug fixing Don't try null move at low depths Improved history Prune moves with bad history Default DrawScore set to 0 Improved time management Improved imbalances evaluation it's 30 elo stronger than 2.0...