Chess, Chatbots And Intelligence: I Make A Big Claim!

[towforce]

This is based on the assumption of supervised learning, meanwhile big tech is running out of human generated data, and meanwhile human data is generative AI polluted, so the next step in line is some kind of reinforcement learning for generative AI (we already had such a transition in Go and Chess). IMO this is the part where funny things can start to happen....when the machine starts to teach itself.

I agree. A similar question is whether LLMs have "emergent properties" (link) that act as a force multiplier to their intelligence. For me, the answer is "yes": I believe LLMs to be more intelligent than they would be if they did not have them. They sometimes hallucinate, but they also, IMO, give good answers to questions which are not in their training data more often than they would do if no emergence had happened.

Quickly looking for evidence, I found this section of a wiki article - link. In that section, I found:

* on the positive side, good patterns containing generalised models of the world do appear in LLMs

* on the negative side, it is shown that sometimes, what looks like highly intelligent work, turns out to have been caused by reasoning shortcuts

Regarding "reasoning shortcuts": in the early days of chatbots (starting with Eliza in 1966), people were more impressed than they ought to have been because "...human judges are so ready to give the benefit of the doubt when conversational responses are capable of being interpreted as intelligent" (from link).

A quick chess thought (my opinion): the metrics measured by clauses of code in an HCE (hand coded evaluation) can fairly be described as "reasoning shortcuts", because they run very quickly, and they give some guidance as to the quality of white's position, but they don't genuinely measure that quality in a way that a sophisticated understanding of chess would, and they are known to not be useful in many positions.

[chrisw]

Moving this thread to forum 14 was a negative indicator on the quality of the moderation team.

If you see chess as just a fun game (though obviously a popular game), and computer chess as just another hobby, like collecting stamps, then the moderation decision would have been correct.

If, as many members do, you see computer chess as having value in terms of learning about the software, technology and patterns of progress involved in generating intelligent behaviour in machines, then the decision was not correct.

A total of nine "I, me, myself and I" in the starting post plus the title, takes it right through the limit for narcissism index and indicate it's not really about chess or chatbots but the OP. KIndergarden for flagrant insertion of self, since you ask.

[Ras]

Is there any intelligence in chatbots at all? I thought these are just search machines that efficiently find existing solutions or answers on the internet

Not even that. What they actually are is stochastic parrots. They just predict, depending on the context as captured in large NNs, what next word would probably follow after this word without understanding anything or verifying it. That's also why they "hallucinate". They mimic the form while not having any understanding of the content. There is no "intelligence" in this so-called "AI".

[towforce]

A total of nine "I, me, myself and I" in the starting post plus the title, takes it right through the limit for narcissism index and indicate it's not really about chess or chatbots but the OP. KIndergarden for flagrant insertion of self, since you ask.

I am not a narcissist (note: in order to say that, I used the word "I": I could, of course, have used the expression "the author of this post" in order to avoid totting up points on your N word index).

The model presented for the ongoing progress of chatbots based on what happened in computer chess is worthy of discussion, but it is obviously not hard science: they are the thoughts of the author, so it shouldn't be surprising that it contains references to that person.

[towforce]

Is there any intelligence in chatbots at all? I thought these are just search machines that efficiently find existing solutions or answers on the internet

Not even that. What they actually are is stochastic parrots. They just predict, depending on the context as captured in large NNs, what next word would probably follow after this word without understanding anything or verifying it. That's also why they "hallucinate". They mimic the form while not having any understanding of the content. There is no "intelligence" in this so-called "AI".

1. Roughly speaking (there may be the odd exception in the animal kingdom), the only known entity that builds and uses sophisticated mental models for understanding is the human brain: in general, non-human animals act without thinking things through using language. By your definition, anything that doesn't behave like a human brain is not intelligent: it's just a Chinese Room: in a Chinese Room, you don't speak any Chinese - but questions in Chinese are shoved under the door: you go through your reference materials, find the correct answer, and pass that back under the door. You've correctly answered the question, but you don't have the slightest idea what was asked, and what you answered - link.

2. LLMs actually do generate models of the world - link. If you visit social media discussions about chatbots, you'll see that older people tend to be strongly vocal in their dislike of them for what are very clearly emotional reasons: are you in this category?

3. Using the same link as used in point (2) above, you'll also notice that in general, when some behaviour that looks intelligent was actually generated by a reasoning shortcut, humans are very willing to give the process behind the behaviour the benefit of the doubt and hence to label that behaviour as intelligent (I can give you personal examples if you wish). It's close to 100% certain that at times in your life you have marvelled at some "intelligent" behaviour that was actually generated by a reasoning shortcut.

[towforce]

While there are obviously big differences between chatbots and chess engines, going forward from where we are now, the similarities might increase: it turns out that building a data tree greatly enhances a chatbot's intelligence - and, at heart, a chess engine's game tree is a data tree. This makes it even more appropriate for people who have experienced the progress of chess engines to use this experience to intelligently speculate how chatbots will progress.

Here's where these similarities were discussed in the Gemini thread - link.

[towforce]

Another similarity between chess engines and chatbots: there are now league tables of how intelligent chatbots are (though your probably shouldn't choose which chatbot to use just on this basis right now: some people who use them a lot advocate using several of them, since different chatbots are strong in different areas in which you might want help):

https://lmarena.ai/ (accept conditions, then select the "Leaderboard" tab, then scroll down).

Edit: the above league table seems to be based on user preference rather than intelligence - so not quite SSDF.

[towforce]

...That's also why they "hallucinate".

1. Hallucination is still there - but it is improving as the chatbots become more intelligent.

2. Humans hallucinate as well: they often misunderstand things.

3. Hallucination can be seen as a metric by which chatbots can be measured - in the same way that blunders can in chess engines.