Cwtch - an experimental NNUE Javascript engine

[op12no2]

I didn't think (hand coded) Javascript would be fast enough for NNUE, but a simple 768->128->1 net trained quickly on relatively few positions (~8M) seems about on par with Lozza's eval (~2700 on CCRL). Lozza is my HCE Javascript engine.

I'm looking forward to playing around with network shapes after I've generated some DIY data. Currently the trainer is just using quiet-labeled.epd and lichess-big3-resolved.epd (thanks to the authors).

The trainer is also Javascript using TensorFlow.js.

I currently scale the network output by 143 - calculated by tuning it over the EPDs using the original Texel sigmoid with K=1. Not sure what other people do..?

For anybody interested, here is the repo. Currently Cwtch is a simple full width PVS searcher without beta pruning, null move and piece lists etc.

https://github.com/op12no2/cwtch

Run from the command line and enter UCI commands. See also uciExec() for custom commands.

Code: Select all

node lozza.js

PS: The trainer generates weights in a .bin file and a .js file for inclusion in the source. I currently use the latter so I can see the weights easily; with half an eye on quantisation.

[op12no2]

Oops, I mean...

Code: Select all

node cwtch.js

[op12no2]

Since everybody is obviously on the edge of their seats, and I've had a few beers, I thought I'd give an update

Dumped tensorflow.js in preference for a DIY trainer. This also helped me realise that using a 'default' sigmoid is wrong (too steep) - it needs to be 'stretched' and am currently using -x/100.

Decided to use my own data and chose to bootstrap from a 768x1 white-relative linear net with weights that mimic a material only eval (using the Larry Kaufman's 'imbalance article' values). Using that net with random move ordering to play 40k games with max 12k nodes/move generated about 7M FENs. Using them (shuffled but no filtering) to train a 768 x 8 network blew the 768 x 1 network out of the water, phew! Rinse, repeat, kindasortaish...

https://github.com/op12no2/cwtch

I was confused about perspective nets and the good people on Discord were very polite in explaining them to me. I was also able to quickly borrow half a billion FENs for some quick experiments (thanks Jo and No4b) Discord is amazing, TalkChess is amazing; they fill different niches. Use them both!