Awful paper)

[Sergei S. Markoff]

Is it only me who's being driven mad by reading this paper?

https://www.technologyreview.com/s/5412 ... al-master/

Most enraging quotations:

"...while computers have become faster, the way chess engines work has not changed. Their power relies on brute force, the process of searching through all possible future moves to find the best next one"

"In a world first, a machine plays chess by evaluating the board rather than using brute force to work out every possible move"

And so on.

Human brain contains about 86 000 000 000 neurons and several hundreds of trillions of synapses. Comparing to our best elecronic machines it is much bigger and stronger in a terms of logical elements number. That's why computer algorithms for chess looks to be much more sophisticated than human ones because they works on descent hardware but provides a better play)

When someone says: "Let we limit computer speed to only 5 nps and will see...", I want to reply: "Let we limit your brain size to only 1000 neurons and then will see!"

Any strong modern chess program is a masterpiece of machine learning. We have at avarage 35 possible moves in average chess position. So if chess programs are so dumb and using exhaustive search, Deep Blue with it's 300 mln nps will search about 11 days to solve average "mate in 6", but at practice most programs can solve 99% of these puzzles in a few seconds using conventional desktop.

Best chess program in 2016 has more then 300 Elo advantage over the best 2006 program *at the same hardware*.

But sometimes some people are going to jump out like the Jack in the box to say: "You're all idiots! I'm just invented the selective search! And I want to publish the epic folio about it in MIT journal!"

[Daniel Shawul]

Is it only me who's being driven mad by reading this paper?

Sergei, if we think with a cool head, it is not all that awful i think

Any strong modern chess program is a masterpiece of machine learning

The Giraffee chess, that used deep learning of which the article is based on, is here so he can comment better than me.

However, I highly doubt chess engines are good examples of machine learning. It is mostly heavy selective search, where the knowledge (eval) and selectivity decisions all pre-programmed. Alphago uses neural nets for both decisions invalidating the need, for instance, a Larry Kaufmann in your group to tune your parameters with human chess knowledge. Machine learning is extracting that knowledge from a database with tools. There is not so much supervised+reinforcement learning going on in most chess programs; but ofcourse there are always exceptions. We had chess engine using different machine learning techinques temporal diffrenece, neural nets, book learning etc. with different success. The most successful are deep selective searchers like Stockfish.

Cheers
Daniel

[Sergei S. Markoff]

However, I highly doubt chess engines are good examples of machine learning. It is mostly heavy selective search, where the knowledge (eval) and selectivity decisions all pre-programmed.

I'm disagree. Most strong programs are tuned using huge position/games datasets, eval function is complicated and has a lot of params, at least several thousands.
In other hand a neural network is nothing more then hierarchy of logistic (or other activation) functions. I don't see any significant difference, wouldn't you?..

[Daniel Shawul]

However, I highly doubt chess engines are good examples of machine learning. It is mostly heavy selective search, where the knowledge (eval) and selectivity decisions all pre-programmed.

I'm disagree. Most strong programs are tuned using huge position/games datasets, eval function is complicated and has a lot of params, at least several thousands.
In other hand a neural network is nothing more then hierarchy of logistic (or other activation) functions. I don't see any significant difference, wouldn't you?..

Well things like doubled pawns are bad, passed pawns are good, rook on open file is good etc are all hand selected features that the neural network could actually find out with out human interference. Actually, once the neural network is built, alphago people do sort of 'tuning' (reinforcement learning) by palying self games. But clearly the supervised learning part where the neural nets finds features by itself from a dataset is quite different from how it is done often in chess - the programer/grandmaster selects the features (most important i think) for the program.

[Sergei S. Markoff]

Well things like doubled pawns are bad, passed pawns are good, rook on open file is good etc are all hand selected features

It is very depends of structure of the evaluation function. Depending on other factors sometimes doubled pawns are good and unprotected passers are bad. Features extraction is good, but overfitting and slow eval are bad. Everyone tries to find balance — and this is a part of a big data analysis and model mastering. Anyone who works with big data and machine learning knows, that direct application of crude neural network is much less effective then creating a custom model that applies most sutable machine learning methods. Neural network is needed when you want to get model for almost no cost. In my company I have a huge intelligent framework for auto-building models which using regressional trees, different forms of regression, neural networks, SVM, nearest neighbours, covariants, heuristic search for factors etc. But anyway it's almost always possible to improve this model with some hand-tuned tricks.

Please understand me correctly: I'm not NN-hater, but when someone says that he is a pioneer of selective approach and other people are doing brute force — that's completely and clearly incorrect. Looks like just a self-promotion, not science. Well, if they will show us the program stronger than Stockfish & Komodo — then it will be enogh to trust in their revolutionary approach, but the program playing at the same level as IM?.. Are they joking? 2500 elo in year 2016?..

alphago people do sort of 'tuning' (reinforcement learning) by palying self games.

I'm doing the same, and I think most of the contemporary programmers are doing the same. Moreover, Deep Blue did the same and very old dedicated systems did the same. Where are the novelty?

I think that the real novelty is a good verbal brands: "deep learning", "reinforcement learning". Really looks like "intelligent design" or "buy two our models by the price of one"

But clearly the supervised learning part where the neural nets finds features by itself from a dataset is quite different from how it is done often in chess - the programer/grandmaster selects the features (most important i think) for the program.

It will be nice to see these features

[Dann Corbit]

I think that giraffe is doing something fundamentally different.

It is the first really successful learning experiment where the learning is not directed with human intervention (though Andrew Tridgell's program KnightCap had some limited success). For the most part, automated learning leads to bad evaluation functions unless humans intervene and carefully guide the learning in some way.

For instance, did you tell your program to try parameter x, iterating from y to z and then choose the best? For this case, it is partly the human injecting knowledge and partly the engine learning it. You told it what to do and how to do it and it cycled over the input to make an output.

If you look at what is learned and how it is learned, to me it looks quite different than what everyone else is doing.

It is possible you are doing the same thing as giraffe, because I do not know how you are performing the learning. But from the learning approaches I have read, they are not doing the same thing as giraffe.

I do not know that the approach of giraffe is better than other approaches. But it looks quite different and very interesting to me.

His engine is not the strongest engine, so one might even argue that the approach is bad. But maybe refinement of this method can make the world's best evaluation function. I do not think that the answer is known one way or the other.

I think it would be a very interesting experiment to have the entire experiment performed on a GPU, since it is heavily floating point oriented.

[Daniel Shawul]

Please understand me correctly: I'm not NN-hater, but when someone says that he is a pioneer of selective approach and other people are doing brute force — that's completely and clearly incorrect. Looks like just a self-promotion, not science. Well, if they will show us the program stronger than Stockfish & Komodo — then it will be enogh to t

I don't think that is what is claimed. My wood counter chess engine is 2300 elo due to searching deep with lmr+nullmove etc. They call this brute force approach because it doesn't think like a human does. That is why they call Deepblue brute force even though had null move, singular search etc.
For GO this was significant because you can not use the "brute force" approach due to the huge branching factor. Now, listen to this ... a deep learning neural network that makes moves immediately beat the state-of-the-art MCTS engine 85% of the time. If that doesn't bamboozle you, I don't know what will

brands: "deep learning"

This has proven its power in several other fields recently vision/sound recognition, and user preferences in google/facebook. The convolutional neural network idea is truely impressive to me since the idea came looking at the arrangement of receptors in animal eye. We had multilayer perceptrons before but didn't perform good enough. Add this new ingredient from biological fact, it suddenly moves forward many fields. Who knows what other things could be learned from the way human brain works. Demis Hassabiss is the neuroscientist in the alphago groupd. So I don't think it is just branding.

I don't want to be too optimistic, but I think a real progress has been made in the AI, which the brute force chess approach of deepblue didn't do, or the recent event of solving checkers.

[Sergei S. Markoff]

I don't think that is what is claimed. My wood counter chess engine is 2300 elo due to searching deep with lmr+nullmove etc. They call this brute force approach because it doesn't think like a human does. That is why they call Deepblue brute force even though had null move, singular search etc.

Human brain and von-Neumann-style computer are devices with very different architecture. It's expected and logical that they had different optimal approches to solve the same problem. As I mentioned before human brain is bigger in the terms of imaginary transistors, every brain cell is processing device while our best electronic devices has several thousands of cores, not more. Why do not call a neural network a brute force approach? NN program code is simplier and smaller then a code of modern chess engine.
There are two different approaches in technology. One is bionic: when be're going to replicate some natural techniques in our machinery. The other one is completely different. For example, first muscular fibres was grown in vitro only at the end of XX century, but we already have had lifting cranes and other tools to lift heavy weights *without* replicating any muscular fibres. But lifting cranes and chess programs for relatively weak von-Neumann-style machines are stronger than humans. There are no practical needs to replicate natural approach to solve these particular problems like weights lifting and chess playing. It will be great experiment anyway to create some bionic-style crane or NN playing chess (especially using some appropriate devices like TrueNorth for example: https://en.wikipedia.org/wiki/TrueNorth).

Of course, sometimes we can benefit from some synthetic approach. Scruffy school approaches are mighty [https://en.wikipedia.org/wiki/Neats_vs._scruffies], and moreover most part of chess engine is completely scruffy, only with exception of endgame tablebases) But don't forget that scruffy approach is for the case when you don't understand something and simply cannot be neat enough) The most interesting part of NN chess engine would be it's parts *except* NN

For GO this was significant because you can not use the "brute force" approach due to the huge branching factor. Now, listen to this ... a deep learning neural network that makes moves immediately beat the state-of-the-art MCTS engine 85% of the time. If that doesn't bamboozle you, I don't know what will

But the checkers was solved without any dirty things Chess is in between of this two worlds so I think if will forever be a place of merely synthetic approaches.

[tpetzke]

Hi Dann,

I tune iCE with a genetic algorithm. I start all values from scratch and tune all at once. Sure I have a possible value range for each parameter which is naturally given by the bits I dedicate to each feature.

In my last tuning run it took the algorithm 36 hours to go from random values to values that were only 50 ELO weaker than those of my fully tuned base version. So this means 36 hours to produce the eval weights of an 3000 ELO engine (CCRL).

http://macechess.blogspot.de/2016/02/ch ... uning.html

With more running time the algorithm improved the set further but very fast the set was already pretty good. So 72 hours for 2500 ELO is not so hard to beat.

[Sergei S. Markoff]

BTW, one deep learning evangelist in Russia (Anatoly Levenchuk) was so excited after reading news about Giraffe that said in one of his public lectures (literally):

...all teams that for 5-6 years with the help of grandmasters worked on game tricks and positions analysis... that teams are now fired, they are resting now bacause the new type of computer programming is now appeared!

http://22century.ru/video/levenchuk-deep-learning [from 16:56] (sorry, only Russian)
A lot of news sites in Russia wrote a lot of sensational and merely idiotic relations about this project.

http://medialeaks.ru/news/1509dalex_chess
https://geektimes.ru/post/262612/
http://www.interfax.ru/world/466755

etc.