Human Beats Top Go Computer

[towforce]

Kellin Pelrine, an amateur Go player, and his team uncovered weaknesses in the play of the top Go computers - a mistake a human simply wouldn't make (misunderstanding the nature of a group of stones). They built a program that exploited this weakness, and it consistently beat the top Go program.

Kellin then went one step further: he mastered the exploitation of this weakness himself, and then was able to consistently beat this superhuman machine (the top ranking Go computer) himself - without assistance.

AI is not "there" yet.

This happened in February - so why haven't you heard about it? Because all types of media have been swamped with positive AI news stories (in the sense of its capability, so ignoring the "it's going to destroy the world" stories).

https://arstechnica.com/information-tec ... y-over-ai/

IMO this news supports my assertion, that I've made many times at this forum, that NNs encode a large number of surface (shallow) patterns but miss important underlying deep patterns.

Despite this, IMO it is still true that intelligent people's advantages over the herd are rapidly eroding right now.

[Alexander Schmidt]

"by taking advantage of a previously unknown flaw that had been identified by another computer."

A computer beats another computer, and a human uses the same strategy to beat the computer in the same way. To question the superiority of Go programs over humans because of this is quite far-fetched...

Of course, AI make mistakes, and one should not blindly rely on it. But still in Go and in Chess the AI do have a much deeper positional understanding than humans.

[towforce]

"by taking advantage of a previously unknown flaw that had been identified by another computer."

A computer beats another computer, and a human uses the same strategy to beat the computer in the same way. To question the superiority of Go programs over humans because of this is quite far-fetched...

Of course, AI make mistakes, and one should not blindly rely on it. But still in Go and in Chess the AI do have a much deeper positional understanding than humans.

I didn't intend to give the impression that human players are superior to computers (though in Go, right now, we are until the NN trainers can find a remedy for the identified weakness).

You said "in Go and in Chess the AI do have a much deeper positional understanding than humans": why do you think that the NN's understanding is deep, rather than them having shallow, but a large number of instances of, positional knowledge?

[M ANSARI]

It is interesting to see that it took another AI to find the flaws of the first AI. If anything, that shows that AI will totally revolutionize the way we live in the near future. And while this can have many bad implications, there hopefully is more good than bad. AI's will continuously find flaws of the previous generation of AI's and this will only lead to stronger AI's.

[towforce]

It is interesting to see that it took another AI to find the flaws of the first AI. If anything, that shows that AI will totally revolutionize the way we live in the near future. And while this can have many bad implications, there hopefully is more good than bad. AI's will continuously find flaws of the previous generation of AI's and this will only lead to stronger AI's.

AIs will get stronger - but, for me, this incident provides valuable insight:

* why is it that small alterations to images, which are barely visible to humans, causes AIs to fail to recognise what's in the image?

* why is it such a struggle to get AIs to do the simple task of driving a car - a task that even low IQ people can do without difficulty?

The answer is that this encoding of a large number of surface (simple) patterns can create the illusion of understanding - but in reality, these huge NN's are lacking in the understanding of what, to us, seems "obvious". The top Go programs demonstrating that, in reality, they don't understand the simple concept of a group of stones, is actually a REALLY good demonstration of this. I've heard it said that AIs for driving cars struggle with this concept: when a cyclist/pedestrian disappears behind another vehicle, it doesn't cease to exist - it's going to reappear a few seconds later.

As a human, I am grateful that, in childhood, we are somehow able to encode deep (complex) patterns of information about the way the world works: this is a MASSIVE part of human intelligence. We still make silly mistakes - but an impressive range of difficult behaviours, thinking tasks, and sophisticated understanding come easily to us as adults. Per the other thread, this is impressive from a device encoded with 23,000 genes with a massive amount of other complex functionality!

I remember the long wait for chess computers to reach GM level: in 1996, after the first Kasparov v Deep Blue, I was still regularly seeing the expression "not in my lifetime" - then it happened the very next year (then after the 1997 rematch, it took a long time for ordinary computers to reach GM level). However, AI experts had been saying that computer chess would reach GM level within 10 years since the 1950s. History telescopes events: when you're living through them, it seems to take a lot longer than it does looking back (then, suddenly, you're there).

[chrisw]

It is interesting to see that it took another AI to find the flaws of the first AI.

Wasn't it humans who found the flaw?

[mig2004]

A very important hallmark of human intelligence is flexibility. Humans can reprogram their mental algorithms on the fly. So we interact with the real world. AI algorithms tend to crash when finding events outside the scope of the data they ´ve been trained on. Self-programming is still a chimera in the world of AI.

[towforce]

It is interesting to see that it took another AI to find the flaws of the first AI.

Wasn't it humans who found the flaw?

Humans found the flaw with the aid of computers. So... are the computers used to help find the flaw a tool (and tools are legitimate because the use of tools is a hallmark of humanity), or is it cheating because they can be classified as an AI - so the humans weren't really using their own intelligence?

Or are we just tying ourselves up in philosophical knots?

[towforce]

A very important hallmark of human intelligence is flexibility. Humans can reprogram their mental algorithms on the fly. So we interact with the real world. AI algorithms tend to crash when finding events outside the scope of the data they've been trained on. Self-programming is still a chimera in the world of AI.

This is a very important point: GMs spend time doing analysis of positions from their games. I think that doing this gives them more than knowledge about that position: it also gives them deep knowledge about the nature of chess in general.

As we learned from watching pre NN chess computers steadily improving, there's more than one way to play chess: building a huge game tree can substitute for a lack of knowledge about the game. This doesn't work in the game of Go, because it has a much higher branching factor than chess does (which is why humans held out in Go so much longer than they did in chess). Alpha Go was intended to show that NNs can use knowledge in Go to make up for the reduced search depth: the project was obviously successful, but, for me, discovering that this knowledge doesn't include something as basic as understanding a group of stones is very revealing.

The results are very impressive, and humanity will be very well served in all sorts of areas - but it seems to me that, while humans have a deep understanding of things, the current NN based AI is somewhat analogous to a large cage of parrots.

[smatovic]

[...] the current NN based AI is somewhat analogous to a large cage of parrots.

Haha, a large cage of parrots vs. a bunch of monkeys in front of typewriters!

--
Srdja