How to measure human-like intelligence...modern Turing test proposed

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

smatovic
Posts: 3351
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

How to measure human-like intelligence...modern Turing test proposed

Post by smatovic »

DeepMind Co-Founder Proposes a New Kind of Turing Test For Chatbots
https://slashdot.org/story/23/06/20/212 ... r-chatbots
Mustafa Suleyman, co-founder of DeepMind, suggests chatbots like ChatGPT and Google Bard should be put through a "modern Turing test" where their ability to turn $100,000 into $1 million is evaluated to measure human-like intelligence. He discusses the idea in his new book called "The Coming Wave: Technology, Power, and the Twenty-first Century's Greatest Dilemma." Insider reports: In the book, Suleyman dismissed the traditional Turing test because it's "unclear whether this is a meaningful milestone or not," Bloomberg reported Tuesday. "It doesn't tell us anything about what the system can do or understand, anything about whether it has established complex inner monologues or can engage in planning over abstract time horizons, which is key to human intelligence," he added. The Turing test was introduced by Alan Turing in tnewhe 1950s to examine whether a machine has human-level intelligence. During the test, human evaluators determine whether they're speaking to a human or a machine. If the machine can pass for a human, then it passes the test. Instead of comparing AI's intelligence to humans, Suleyman proposes tasking a bot with short-term goals and tasks that it can complete with little human input in a process known as "artificial capable intelligence," or ACI. To achieve ACI, Suleyman says AI bots should pass a new Turing test in which it receives a $100,000 seed investment and has to turn it into $1 million. As part of the test, the bot must research an e-commerce business idea, develop a plan for the product, find a manufacturer, and then sell the item. He expects AI to achieve this milestone in the next two years. "We don't just care about what a machine can say; we also care about what it can do," he wrote, per Bloomberg.

:roll:

--
Srdja
User avatar
towforce
Posts: 12536
Joined: Thu Mar 09, 2006 12:57 am
Location: Birmingham UK
Full name: Graham Laight

Re: How to measure human-like intelligence...modern Turing test proposed

Post by towforce »

Bad idea IMO: most humans don't turn $100,000 into a million dollars.

The basic idea is good: a goal with multiple stages and multiple different types of intelligence required. In the end, though, you've got the "Jeopardy" problem: whatever the particular game is, the system can be developed to play it - as IBM's Watson did with that game.

I can tell you what intelligence is in two words: for each individual person, it's an entity that... "understands me". :)
Human chess is partly about tactics and strategy, but mostly about memory
smatovic
Posts: 3351
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: How to measure human-like intelligence...modern Turing test proposed

Post by smatovic »

towforce wrote: Wed Jun 21, 2023 4:04 pm Bad idea IMO: most humans don't turn $100,000 into a million dollars.
[...]
It is a bad idea, what is the shortest, quickest way to make a 1 million bucks? It tells more about how humans view/value/measure/use intelligence than about intelligence itself...that being said, I forgot who it was, but somebody said intelligence is about how you are able to perform in a given environment/system...so it does make sense these days, or alike.

--
Srdja
User avatar
Bo Persson
Posts: 261
Joined: Sat Mar 11, 2006 8:31 am
Location: Malmö, Sweden
Full name: Bo Persson

Re: How to measure human-like intelligence...modern Turing test proposed

Post by Bo Persson »

smatovic wrote: Wed Jun 21, 2023 1:47 pm DeepMind Co-Founder Proposes a New Kind of Turing Test For Chatbots
https://slashdot.org/story/23/06/20/212 ... r-chatbots
To achieve ACI, Suleyman says AI bots should pass a new Turing test in which it receives a $100,000 seed investment and has to turn it into $1 million. As part of the test, the bot must research an e-commerce business idea, develop a plan for the product, find a manufacturer, and then sell the item. He expects AI to achieve this milestone in the next two years. "We don't just care about what a machine can say; we also care about what it can do," he wrote, per Bloomberg.
So the bot test is that the AI actually can achieve this?!

Unlike most humans...
User avatar
towforce
Posts: 12536
Joined: Thu Mar 09, 2006 12:57 am
Location: Birmingham UK
Full name: Graham Laight

Re: How to measure human-like intelligence...modern Turing test proposed

Post by towforce »

Bo Persson wrote: Wed Jun 21, 2023 8:44 pm
smatovic wrote: Wed Jun 21, 2023 1:47 pm DeepMind Co-Founder Proposes a New Kind of Turing Test For Chatbots
https://slashdot.org/story/23/06/20/212 ... r-chatbots
To achieve ACI, Suleyman says AI bots should pass a new Turing test in which it receives a $100,000 seed investment and has to turn it into $1 million. As part of the test, the bot must research an e-commerce business idea, develop a plan for the product, find a manufacturer, and then sell the item. He expects AI to achieve this milestone in the next two years. "We don't just care about what a machine can say; we also care about what it can do," he wrote, per Bloomberg.
So the bot test is that the AI actually can achieve this?!

Unlike most humans...

It all depends on the environment and the competition. Same applies to the bots: for them, making money in the human world, competing against humans, might be doable. If bots turn out to be better at making money than humans, then soon our world will become bot v bot. Then it will be difficult for bots to make money.
Human chess is partly about tactics and strategy, but mostly about memory
smatovic
Posts: 3351
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: How to measure human-like intelligence...modern Turing test proposed

Post by smatovic »

Bo Persson wrote: Wed Jun 21, 2023 8:44 pm [...]
So the bot test is that the AI actually can achieve this?!

Unlike most humans...
Idk if it is intended as simulation or real game thing, did not read the book:
He discusses the idea in his new book called "The Coming Wave: Technology, Power, and the Twenty-first Century's Greatest Dilemma."
Would like to add:

Turing Test, Metzinger Test, Lemoine Test
https://epsilon.app26.de/post/turing-te ... oine-test/

Google LaMDA - Is LaMDA Sentient?
https://epsilon.app26.de/post/google-la ... -sentient/

GPT-3 - Scratching at the Edge of the Metzinger Test?
https://epsilon.app26.de/post/gpt-3-scr ... nger-test/

I myself prefer the Metzinger Test.

--
Srdja
smatovic
Posts: 3351
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: How to measure human-like intelligence...modern Turing test proposed

Post by smatovic »

towforce wrote: Wed Jun 21, 2023 8:56 pm ...
High-frequency trading
https://en.wikipedia.org/wiki/High-frequency_trading

Knightmare on Wallstreet
https://en.wikipedia.org/wiki/Knight_Ca ... disruption

--
Srdja
User avatar
towforce
Posts: 12536
Joined: Thu Mar 09, 2006 12:57 am
Location: Birmingham UK
Full name: Graham Laight

Re: How to measure human-like intelligence...modern Turing test proposed

Post by towforce »

I think we (the humans) are steadily changing our thinking about intelligence, and AI will speed this up.

Basically, there's more than one way to be intelligent: this applies to both humans and machines. A sentence that encapsulates this nicely:

"It's not how smart are you - it's how are you smart."

For me, the dedicated chess computers powered by 8-bit microprocessors were smart: they could beat around 99% of human chess players!
Human chess is partly about tactics and strategy, but mostly about memory
Uri Blass
Posts: 10903
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: How to measure human-like intelligence...modern Turing test proposed

Post by Uri Blass »

smatovic wrote: Wed Jun 21, 2023 1:47 pm DeepMind Co-Founder Proposes a New Kind of Turing Test For Chatbots
https://slashdot.org/story/23/06/20/212 ... r-chatbots
Mustafa Suleyman, co-founder of DeepMind, suggests chatbots like ChatGPT and Google Bard should be put through a "modern Turing test" where their ability to turn $100,000 into $1 million is evaluated to measure human-like intelligence. He discusses the idea in his new book called "The Coming Wave: Technology, Power, and the Twenty-first Century's Greatest Dilemma." Insider reports: In the book, Suleyman dismissed the traditional Turing test because it's "unclear whether this is a meaningful milestone or not," Bloomberg reported Tuesday. "It doesn't tell us anything about what the system can do or understand, anything about whether it has established complex inner monologues or can engage in planning over abstract time horizons, which is key to human intelligence," he added. The Turing test was introduced by Alan Turing in tnewhe 1950s to examine whether a machine has human-level intelligence. During the test, human evaluators determine whether they're speaking to a human or a machine. If the machine can pass for a human, then it passes the test. Instead of comparing AI's intelligence to humans, Suleyman proposes tasking a bot with short-term goals and tasks that it can complete with little human input in a process known as "artificial capable intelligence," or ACI. To achieve ACI, Suleyman says AI bots should pass a new Turing test in which it receives a $100,000 seed investment and has to turn it into $1 million. As part of the test, the bot must research an e-commerce business idea, develop a plan for the product, find a manufacturer, and then sell the item. He expects AI to achieve this milestone in the next two years. "We don't just care about what a machine can say; we also care about what it can do," he wrote, per Bloomberg.

:roll:

--
Srdja
If somebody is going to develop this type of bot then I fully expect him to earn money again and again and not to tell the public that it is thanks to a bot.

Maybe there is already some bot that does it for some rich person.