chrisw wrote: ↑Fri Jul 24, 2020 7:52 pmYes, else ply 0 Leela would beat GMs at non-blitz and GMs/IMs can mostly express in language why so and so move and so on.
I would agree on this point: I want chess solved before I die. If nobody else will do it then I will.
The results are very impressive, but it comes at a price: the NN, and hence the amount of knowledge it can store, has been made smaller. From the POV of solving chess, this is a slight negative.
The main problem for getting in the non-linearities is already solved by tuning.
No. You can't solve chess by taking a numerical expression and solving its weightings. You need an expression that captures everything that needs to be captured, and THEN you need to tune it.
The difference between tuning a polynomial of known themes and a layered spreadsheet of one hots, is that human can understand the polynomial and devise more terms and combine terms, all on basis of human ingenuity...
Not going to happen: the chess polynomial will not be a common 2 variable polynomial (y = 4x^2 - 2x - 3).
Chess has:
* 64 squares
* 7 pieces (rook, knight, bishop (ws), bishop(bs), queen, king, pawn)
* 2 colours
So the base number of variables in the polynomial will be: 64*7*2 = 896
Each of the bishops can only go on half the squares, so reduce that by 4 * 32 = 128
There are 16 squares that the 16 pawns can't go on, so reduce the number again by 16 * 16 = 256
So the number of polynomial variables will be 896 - 256 - 128 = 512
That's too many dimensions for the human brain to work in, unless somebody can come up with a way of visualising a polynomial in that many dimensions.
Could I write a program to fit curves in 512 dimensions?
HELL YES!!! 
Worse still, I can't think of anything else I'd rather be doing!
I'm not going into detail in this thread (this is NNUE's place), but I'll quickly say two things:
1. NN's don't require accurate arithmetic. This is why vector processing chips for NN training (like Google's TPU -
link) tend to use half-precision arithmetic. In respect of this, to make my curve fitting easier, I'm planning to have the input data in ranges rather than exact points. This will make the curve fitting MASSIVELY easier, and keep the polynomial level as low as possible (high level polynomials are lumpy, low level polynomials are smooth. Smoother is better!)
2. In contrast to NN learning, this curve-fitting does have one cost: in the input data, you can never have the same same chess position twice with different "scores" (or incompatible score ranges), so the data will have to be checked for this before the curve fitting begins.
Thank you for the spreadsheet mention. I looked it up, and while it might be difficult to train an NN in a spreadsheet, a trained NN can certainly be written as a spreadsheet. There's an article
here - I've already copied the sheet into my own documents!
, whereas the layered spreadsheet technician has absolutely no idea at all what is going on (well maybe some heat maps, but those are in effect no more than pretty pictures). Spreadsheet technician is at the mercy of multi-dimensional connections that make no readable sense.
Writing is the antidote to confusion.
It's not "how smart you are", it's "how are you smart".
Your brain doesn't work the way you want, so train it!
