Gemini

[towforce]

One more thing about this: these trees would have similarities to game trees in computer chess...
[...]

You are not the only one to draw these similarities:

Re: Insight About Genetic (Evolutionary) Algorithms
viewtopic.php?p=949466#p949466

--
Srdja

I'd forgotten that gem of a thread - thank you! Exalted company indeed - Demis Hassabis.

[Werewolf]

I have now spent many hours writing a chess engine with 4 AIs.

There is no doubt about it the order of coding usefulness goes like this:

1) ChatGPT-01 Mini
2) ChatGPT-01 Preview

Then miles behind
3) Gemini Advanced
4) Claude Sonnet 3.5

What ChatGPT-01 seems to be doing to achieve more steps of logic is, roughly:

1. Solving a bit of what has been asked
2. Taking this bit of the solution and automatically re-prompting itself to get the next bit

So basically - using itself iteratively. Would you agree?

In case anyone is unaware, the price of using ChatGPT-01 is a lot higher than other LLMs (in the sense that you are allowed far fewer prompts for your subscription fee), but it will still be the best option for some use cases.

I’m not sure what it’s doing, they seem to have disguised the workings to make it hard to copy. If anyone asks it for “trace reasoning” they get a warning.

Both GPT-01 and 01 mini can do things previous versions can’t, so the progress seems genuine.

[towforce]

I’m not sure what [GPT-o1 is] doing, they seem to have disguised the workings to make it hard to copy. If anyone asks it for “trace reasoning” they get a warning.

Both GPT-01 and 01 mini can do things previous versions can’t, so the progress seems genuine.

I completely agree: I haven't used it, but I accept that it's very likely to be the best chatbot available right now.

Per Srdja's post above, there's a good chance that it generates a data tree for its reasoning choices, which would be a similarity with chess engines (which generate data trees called "game trees", as you probably know!). I started a new thread making the case that chatbots might follow a similar path to chess engines, that there's an upper limit to how good they can get, and that if their journey is similar to chess engines, then they could get near that level in 20-30 years (link). As I pointed out, you would need limits on what you'd expect of a chatbot: chess engines have the advantage here, having a more limited domain.

[jefk]

the AI's are evolving, and the latest Qstar/Strawberry/openai40
are becoming better, but for chess moves you anyway need search
(preferably looking up a neural net but then also still some search in the tree,
simply because of game theory and eg. the minimax theorem by von Neumann).

Here's a guy going to do a little project/test like towforce also seems to
want to do:
https://saychess.substack.com/p/can-i-b ... ed-opening
After a while it might work, and then hopefully better than eg. as the chinese database
(with all the drawing moves in the opening stage) because as i said a few times already
this nowadays isn't so good for human chess (nor for correspondence chess imo unless
you only want to play draws, and/or waiting for your opponent to make a severe input mistake
(or passing away). For a human repertoire you need to specify how sharp you want to play,
etc. etc. (eg. adapting to your opponent but also taking into consideration your own style/
preference of playing.

[Werewolf]

the AI's are evolving, and the latest Qstar/Strawberry/openai40
are becoming better, but for chess moves you anyway need search
(preferably looking up a neural net but then also still some search in the tree,
simply because of game theory and eg. the minimax theorem by von Neumann).

Here's a guy going to do a little project/test like towforce also seems to
want to do:
https://saychess.substack.com/p/can-i-b ... ed-opening
After a while it might work, and then hopefully better than eg. as the chinese database
(with all the drawing moves in the opening stage) because as i said a few times already
this nowadays isn't so good for human chess (nor for correspondence chess imo unless
you only want to play draws, and/or waiting for your opponent to make a severe input mistake
(or passing away). For a human repertoire you need to specify how sharp you want to play,
etc. etc. (eg. adapting to your opponent but also taking into consideration your own style/
preference of playing.

Well it depends on what level you want to play at. For 1400 Elo you don't need a search, and in some positions it's already beyond that.

[towforce]

...but for chess moves you anyway need search.. ..simply because of game theory and eg. the minimax theorem by von Neumann).

Many high quality people have been saying this for a long time (even Donald Knuth said it in The Art Of Computer Programming: it's been over 40 years since I read that, but I think his wording was something like "such [problems] defy analytical solution". I accept that it has been proven that the number of moves rises at a polynomial rate with the size of the board (so a game of chess on, say, a 16x16 board would take a HUGE number of moves (though it would solve the draw problem for today's computers)), but I don't accept that it's impossible to create heuristics that could play the game extremely well without search.

[jefk]

I don't accept that it's impossible to create heuristics that could play the game extremely well without search.

well i'm not saying that;
i have the Nibbler GUI, and despite a very slow GPU (but at least a
GPU as it wasn't on my older comp), I find the Lc0 eval often to be quite
accurate even at very shallow search (if you wouldn't want search at all (and still
decent play in tactical positions) , you imio would need much bigger and
sophisticated neural nets (*); then it would be more like a lookup like e.g. a lookup
in an opening book as the Chinese database, or and endgame base like the syzygy 7.

(*) and preferably some additional code not with alfabeta search but
simply known tactical search patterns as forks, skewers etc etc,
possibly something to code with Python (but then why would you
do this considering the current SF17 system already is effective enough

[towforce]

A basic question: what are the "requirements" for chatbots?

Originally, they were supposed to mimic human conversation - link. The famous "Turing Test" was originally titled, "The Imitation Game" (link), and the test was to be as human-like as possible, not to be as intelligent as possible.

However, this is no longer the case: if you go for a drink with a friend, you don't usually ask him to write a block of code for you, to write a legal document for you, or to explain how to do your school homework in detail.

Thinking about this, the "use case" is clear: there are (or soon will be) a billion people using chatbots, and their job is to answer each of those different people's prompts as best they can. Astonishingly, they're already "reasonably good" at this, and they're improving rapidly.

Earlier in this thread, I was a bit harsh, and said that a chatbot shouldn't be expected to play chess: however - if that's what the user wants, then that's what the chatbot should do: it should make a chess board appear on the screen, and start playing.

If you want it to play at GM level, it should do this is as well: however - it's needn't have the skill to do that itself: it should, however, know where there's an API available from which it can get good moves. Likewise, if the user wants a numerical solution to a maths problem that requires a huge amount of calculation, it should be able to call up that resource as well.

To some extent, this is already happening: ChatGPT can use other systems to draw pictures for example.

So the long term future of chatbots is: ask for absolutely any kind of response, and get a good quality response that answers that request.

Those of us using them know that they're advancing VERY quickly right now. The prediction that their progress will follow an "S curve", and get close to "as good as they're ever going to be" in about 20-30 years, seems reasonable.

[smatovic]

....

At first, these are not simple chatbots anymore, meanwhile they are multi-modal, can handle multiple in/output, and are able to reason.

Second, the Turing Test was IMO not meant to mimic human conservation:

The Turing Test
https://luddite.app26.de/post/the-turing-test/

Third, you can classify (historically) chatbots (I call them meme machines) into three strands:

GOFAI vs. Pattern Matching vs. Neural Networks
https://epsilon.app26.de/post/gofai-vs- ... -networks/

Meme Machines
https://epsilon.app26.de/post/meme-machines/

In 2018 Transformers for LLMs took off, and reached top scores in language tasks, outperforming "pattern matching" approaches prev. used.

Fourth, the goal is now clearly the AGI/ASI, artificial general intelligence/artificial super intelligence. Different people have different time estimates for AGI/ASI, I say if current development continues at such a pace, until 2030 we will have reached AGI/ASI.

Fifth, considering that people view LLMs as stochastic parrots, it is remarkable that they are able to play chess at all.

--
Srdja

[towforce]

...Fourth, the goal is now clearly the AGI/ASI, artificial general intelligence/artificial super intelligence. Different people have different time estimates for AGI/ASI, I say if current development continues at such a pace, until 2030 we will have reached AGI/ASI.

Fifth, considering that people view LLMs as stochastic parrots, it is remarkable that they are able to play chess at all.

Regarding your point (4): chatbots are the nearest thing to AGI that most people have access to right now. I don't think that most people would want ASI (artificial super-intelligence), except in the sense that they could have stupid entertainment that's personally suited to their tastes: I am confident that most people won't want great literary works, or science papers which are at the level of the top peer-reviewed publications. Hence chatbots might be the wrong medium for ASI. Having said that, everyone IS going to want something that can diagnose disorders than their doctor cannot, explain difficult concepts in a way they can understand, create plans for them, write legal documents for them, and coach them well for as rapid progress as they are willing to make.

Given a choice between personalised dumb-shit and rapid personal development, which will people choose? It's not obvious: self-improvement is increasing in a big way: the middle-aged are fitter than ever, and old people are no longer taking to the armchair.

Regarding your point (5): a lot of what LLMs do is stochastic parroting, but it's also easy to show that they build, via language, sophisticated models of the world. Were this not so, they wouldn't be able to intelligently answer a wide range of prompts that aren't in their training material, and they wouldn't be able to play chess. IMO, it won't be long before many prompts, probably including chess, receive an applet in response: when I asked Gemini Advanced about tomorrow's weather, the response was an applet from the weather.com channel.

A bit more speculation: in the medium term, chatbots will evolve into personal assistants that are with us all the time. When ASI arrives, I'm not sure that chatbots will be the way it's used. Maybe there are benefits to having ASI in a chatbot that I haven't seen - just as 1960s supercomputer engineers would not have foreseen the benefits of having small smartphones with many orders of magnitude more computational power than their machines. Given the speculation that chatbots will get close to the limit of their usefulness in 20-30 years, we can also speculate that there's a limit to the usefulness of having a friend who is both really clever and really helpful. Thus, just as there is an upper limit to the usefulness of increased chess skill, there is also a point at which giving a chatbot more skill brings very little benefit.

I'm sure there will be uses for AI that is a lot more intelligent than that in some way, but this will probably be for very special purposes.