Can somebody explain what makes Leela as strong as Komodo?

[hgm]

I did that for pawn structure patterns in earlier versions of Schooner. However, lack of sufficient training data gave unreliable results. It seemed to help a bit - but not enough to make a 100 elo of difference. Mostly I set intuitive values for "patterns" from experience. Are not stored position values more reliable than pattern values?

Perhaps your patterns were not sophisticated (complex) enough? Or you had too few of them?

As Dann explained stored position values are more reliable only in about 0% of the cases. In the remaining 100% they are completely useless, because they don't include the sought position.

[D Sceviour]

I did that for pawn structure patterns in earlier versions of Schooner. However, lack of sufficient training data gave unreliable results. It seemed to help a bit - but not enough to make a 100 elo of difference. Mostly I set intuitive values for "patterns" from experience. Are not stored position values more reliable than pattern values?

Perhaps your patterns were not sophisticated (complex) enough? Or you had too few of them?

I did share code on pawns patterns with Andrew Grant some time ago, but I never posted anything on it. Yes, maybe a larger data base might improve matters. Something to experiment with again!

[Guenther]

How does Leela work - it plays scores of millions of games with itself and analyze the moves , the position, and the result of those games to come up with a predictive analytic outcome for each position. It takes the the position that has the highest chance of winning the game based on its history and analysis of its passed games and plays that move.

How does this differ from polyglot or other opening books?

Because above explanation is wrong. In reality it extracts bazillions of patterns (not positions) and tries to use them in real games.

You can be one the most miserable people on the face of the earth as you seeming have an over whelming desire to tell someone is wrong when they are not really wrong. Semantically there is no almost no difference between a chess position and a chess pattern except for what's in your head. A chess position is a chess pattern by definition. Every chess position is a chess pattern,

Criticism is vain, because in judging others, we regard ourselves as more righteous than they. For some people, their self-esteem and identity is tied to “being right” and “being knowledgeable. They feel that they are a worthwhile person if they are correct, and more importantly, if other people know it. If you derive your self-esteem from being right… why? Why is that your source of self-esteem? Do you feel embarrassed being wrong or making a mistake? by Tim Marks

You seem to have a bad day?

[Dann Corbit]

As Dann explained stored position values are more reliable only in about 0% of the cases. In the remaining 100% they are completely useless, because they don't include the sought position.

I think this should be qualified as a function of time.
Of course, mathematically you are fully correct. The number of chess positions is large enough that a 64 bit number can't hold it, and the number we can store to disk is a teeny-tiny fraction of that (essentially zero percent).

However, at the start of the game, there are only 20 choices. We could (for instance) analyze all of these 20 choices to 60 plies and store to disk.
So, for our first white move (if we are playing white) we will have 100% chance to find our data.

On the first response for black, there are 400 possible choices (twenty squared). We could also analyze and store these to tremendous depth.
So, for our first black move, we could have every possible choice figured out to our satisfaction.

But as you can see, this very rapidly becomes unfeasible:

Code: Select all

perft 1           20
perft 2          400
perft 3         8902
perft 4       197281
perft 5      4865609
perft 6    119060324
perft 7   3195901860

Applying even a 36 ply search to 3,195,901,860 positions is not going to happen, so a complete 7 ply (3 and a half full move) book is not going to happen. Besides which, most of your energy would be spent analyzing positions that will never be played, and I think that would be most unsatisfying.

Hence, the standard approach of forming a book from analysis and pre-existing data on positions has limited usefulness.

A further problem is that the stored book will have wrong answers in it, no matter how carefully you build it. If you analyze to a depth of 60 plies (super-high-end machine will take a day or more per position) there will still be reversals (though infrequent) at 61 plies.
And if you base the nodes on statistical outcomes, there are statistical outcomes that are flawed because humans and machines did not think enough and made the wrong move or they made a move by rote because an opening was memorized or they were given a pre-assigned book move to make without even thinking on it.

Even so, a good opening book can add quite a few Elo. For instance, Attacker-Pro-3 book is worth 113 Elo points over having no book in Sedat's contest:
https://sites.google.com/site/computers ... book-cs-23

In short, HGM is right, but for the first few plies it is possible to get highly probable book hits.

[chrisw]

Is it the Hardware speed or some breakthrough programming innovation?
So far what is the score between Komodo and Leela ?
https://www.youtube.com/watch?v=ymDA7-hdkKA

Nope, nobody can explain, there is no explanation, it just is. I can explain why that. First it understands nothing, no patterns, nothing. It knows nothing either, these are all terms desperate humans try to project onto it, because desperate humans just gotta have an explanation, innit? If you had a 32 man egtb, would you be trying to tell us it understands tactics, or the value of the Bishop pair, or the queen? I think not, you’ld just says it’s a bleeding great lookup table. Well a chess evaluating neural net is just a squashed and imperfect 32man egtb. That’s it. A compressed lookup table that get away with it by being good enough. Same as you really, haha!

[Ras]

I think what makes Leela as strong as Komodo is that Leela plays as good moves as Komodo.

[Dann Corbit]

Well a chess evaluating neural net is just a squashed and imperfect 32man egtb.

So is a chess player's brain, for that matter.

[chrisw]

Well a chess evaluating neural net is just a squashed and imperfect 32man egtb.

So is a chess player's brain, for that matter.

Yes, exactly. Knowledge is statistical. We know things because the thing worked for us in the past more often than not. Or somebody told us and we believed.

[corres]

...
How does Leela work - it plays scores of millions of games with itself and analyze the moves , the position, and the result of those games to come up with a predictive analytic outcome for each position. It takes the the position that has the highest chance of winning the game based on its history and analysis of its passed games and plays that move. In fact, Leela has an output option to display White or Black's "chances of winning percentage" rather than the customary centipawn evaluation. This is a gross simplification of the process - to gain a better and more compete understanding - read through this link below which has links to 20 or so other websites with more information:

https://deepmind.com/blog/alphazero-she ... gi-and-go/

At last there is somebody who understand how Leela (and AlphaZero) works.
It would be good if everybody would know at least this concise description.

[corres]

There are two parts to leela or other neural network based engines: search and evaluation.

Leela (and many go programs) use a form of search called mcts, or Monte Carlo tree search (See UCT). It’s fundamentally different from the minimax search in an engine like stockfish. It uses a strategy (in it’s most primitive form, random) to pick a leaf node to expand. Then plays out a bunch of games (again, in its most primitive form, random) from that leaf node to determine how likely a win/draw/loss is. (This random playouts is where the “Monte-Carlo” comes from.) These results get backed up to the root of the tree. With almost no game strategic knowledge, you could achieve surprisingly good results.
People discovered that if you added some heuristics to guide the selection of leaf nodes and the playouts, you could significantly improve the results (see the pre-nn go program pachi for an example). Then along came DeepMind and trained a neural net to rank the most promising moves (policy) for selection of a leaf node to expand and predict outcome (value), eliminating the need for playouts (there is no “Monte-Carlo” anymore in this nn driven search).
The neural network started out as two nets — a policy net and a value net — in DeepMind’s early design, but got combined into one that takes game state and produces both outputs. The net is trained through reinforcement learning — it plays itself with 800 node searches and trains the net on the improved vale (game outcome) and policy (from the search). It learns over millions of games.
If you look at the games of an early net, it plays randomly, eventually like a child, then progressively stronger. It discovers opening by itself, often preferring openings like the French and Caro-Kann early before preferring d4 openings.
If it has one weakness, it might be the endgame, though there are differing opinions as to why that might be.

This is a more detailed write down of Leela/AlphaZero working.
But I am afraid it is too hard to understand for some chess expert...