GPT-4o made a chess engine

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

jefk
Posts: 693
Joined: Sun Jul 25, 2010 10:07 pm
Location: the Netherlands
Full name: Jef Kaan

Re: GPT-4o made a chess engine

Post by jefk »

as for LLM's i don't believe in (much) 'emergent' capabilities, this imo is mostly sales talk (while
transformer backpropagation trick as invented by the Swiss dr Schmidthuber was a big improvement
in AI, it still is far of any real rational intelligent capability). It may become a bit better
after implementation of Q* reasoning algorithms, but this will take a while; then we are
still far of from AGI, which is the latest hype (at least in the Usa).

As for cgpt4's chess capabilities, it generates language on the basis of statistics, trained from
data looked up from the web, in this situation chess opening moves, so after 1.e4 it finds 1...e5
eg. in a Ruy lopez, so it then responds with e5. It won't go into chess databases, pgn or not.
So after some moves it will start 'hallucinating' (bs-t-ting) as it still usually does; in the
latter respect one could conclude it already is exhibiting some typical human qualities.
Last edited by jefk on Sat Jun 15, 2024 2:38 pm, edited 2 times in total.
smatovic
Posts: 2797
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: GPT-4o made a chess engine

Post by smatovic »

Well, it programs better than it plays chess.

Alone to implement AlphaBeta as Negamax with double negation is IMHO remarkable for LLMs.

Sure, it does fantasize and lacks a decent reasoner, but we just started to open this can.

--
Srdja
jefk
Posts: 693
Joined: Sun Jul 25, 2010 10:07 pm
Location: the Netherlands
Full name: Jef Kaan

Re: GPT-4o made a chess engine

Post by jefk »

when i played around with C++, i remember that programming is only 1 pct of the job;
the rest is debugging. Generating some code may look (and sometimes be impressive, but i
suggest to wait until you've compiled the stuff. In other words you need a human programmer
to integrate all code and get a working programmer. Meanwhile they are hyping about
artificial programmers etc like eg. on a website Devin or so

(not to mention hype about the AI's later going to reprogram themselves, and then becoming
'superintelligence' and converting earth into paperclips; or chess pieces; well that would be fun).

NB i don't deny it can be of some help, but with the thousands of AI tool websites being
generated, usually with AI ofcourse, working with IT doesn't become easier nowadays,
on the contrary (imo a big chaos which only seems to be increasing, with only the big
players making big bucks; reminds me of the internet bubble 1999-2000).
smatovic
Posts: 2797
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: GPT-4o made a chess engine

Post by smatovic »

Sure, hype, bubbles come to ones mind, but this time the generative AI generates so called surplus value, you invest resources in creating AI based on neural networks, and it returns surplus value, an economical feedback loop is established, the AI "delivers", what ever this is in the eye of the observer, it is measurable in dollars, first economical feedback loop, then maybe technological feedback loop, the takeoff of the technological singularity. We will see soon how this will play out, if the hype dooms or blooms, or alike.

--
Srdja
User avatar
towforce
Posts: 11751
Joined: Thu Mar 09, 2006 12:57 am
Location: Birmingham UK

Re: GPT-4o made a chess engine

Post by towforce »

jefk wrote: Sat Jun 15, 2024 2:26 pm as for LLM's i don't believe in (much) 'emergent' capabilities, this imo is mostly sales talk...
I am not sure whether emergent capabilities are real of not, but I can understand why people think they are real.

I use Gemini Pro (the paid version) most days, and while this is not hard science, I am finding two things:

1. I am personally getting great value from it by using it as a life coach, and in situations in which it's large input window (up to a million tokens) enable it to, for example, answer questions about large bodies of input text

2. Sometimes, what it is able to do is genuinely jaw-dropping. I am absolutely 100% STONE COLD CERTAIN that nobody has specifically programmed it to do some of the astonishing things that I have seen it do, so while I admit that I am not using the term in its technically correct way here, it seems to me that some of this capability must be in some way "emergent"

As for cgpt4's chess capabilities, it generates language on the basis of statistics, trained from
data looked up from the web, in this situation chess opening moves, so after 1.e4 it finds 1...e5
eg. in a Ruy lopez, so it then responds with e5. It won't go into chess databases, pgn or not.
So after some moves it will start 'hallucinating' (bs-t-ting) as it still usually does; in the
latter respect one could conclude it already is exhibiting some typical human qualities.
This looks broadly correct here - but it is very wrong to say that it is usually bullshitting: I've personally seen some very impressive hallucination, but most GPT and Gemini output is actually good - and it's getting better with each release (better quality, less hallucination).
The simple reveals itself after the complex has been exhausted.
User avatar
towforce
Posts: 11751
Joined: Thu Mar 09, 2006 12:57 am
Location: Birmingham UK

Re: GPT-4o made a chess engine

Post by towforce »

smatovic wrote: Sat Jun 15, 2024 3:26 pm Sure, hype, bubbles come to ones mind, but this time the generative AI generates so called surplus value, you invest resources in creating AI based on neural networks, and it returns surplus value, an economical feedback loop is established, the AI "delivers", what ever this is in the eye of the observer, it is measurable in dollars, first economical feedback loop, then maybe technological feedback loop, the takeoff of the technological singularity...

+1

Very good point. Early computers were very expensive, had a poor specification, had to be reprogrammed by rewiring circuits, and were horribly unreliable: but they were better than anything else, they made money, and they attracted investment.

Technologies that persist in failing to make a return on investment would be expected to eventually die out. I am at a loss to understand why people keep building and buying quantum computers: at this time, there's nothing they can do that cheap standard computers cannot do better.

Per my previous post, LLMs do genuinely useful work that other technologies cannot do today, and they are improving rapidly. I think they're here to stay.
The simple reveals itself after the complex has been exhausted.
Werewolf
Posts: 1847
Joined: Thu Sep 18, 2008 10:24 pm

Re: GPT-4o made a chess engine

Post by Werewolf »

Worth watching even if you disagree:

https://x.com/elonmusk/status/180197648 ... iIC4EvAGIA
User avatar
towforce
Posts: 11751
Joined: Thu Mar 09, 2006 12:57 am
Location: Birmingham UK

Re: GPT-4o made a chess engine

Post by towforce »

Werewolf wrote: Sat Jun 15, 2024 10:43 pm Worth watching even if you disagree:

https://x.com/elonmusk/status/180197648 ... iIC4EvAGIA

A very smart and articulate 76 year old whom I'd never heard of!

https://en.wikipedia.org/wiki/Geoffrey_Hinton
The simple reveals itself after the complex has been exhausted.
smatovic
Posts: 2797
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: GPT-4o made a chess engine

Post by smatovic »

towforce wrote: Sun Jun 16, 2024 12:26 am [...]
A very smart and articulate 76 year old whom I'd never heard of!

https://en.wikipedia.org/wiki/Geoffrey_Hinton
Turing Award 2018, Yoshua Bengio, Geoffrey Hinton, Yann LeCun.

--
Srdja
JohnWoe
Posts: 511
Joined: Sat Mar 02, 2013 11:31 pm

Re: GPT-4o made a chess engine

Post by JohnWoe »

The whole AGI talk is just snake oil. We don't even have fully auto driving AIs on cars w/o killing everybody on road.

Are you telling me human brain is just a loss function w/ a sum of neurons connected to it? Come on.
When I learned to drive I didn't watch 1000000TiB+ videos of training data.
LLM it's just auto complete on steroids.

It's good for creating templates. But writing good code you need a good human programmer to sense the quality.