Gemini

[Werewolf]

There is a new AI out called Gemini. I believe it is by Google Deep Mind and designed to take on ChatGPT.

The video of what it can do is incredible:
https://deepmind.google/technologies/gemini/#hands-on

I asked it if it could play chess and it boasted it was around 3000 Elo!

I then setup the Saitek Simultano (1800 Elo, late 1980s chess computer) and took on the current version of ChatGPT 4:

White: Simultano (15 seconds / move)
Black: ChatGPT 4

1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Ba4 Nf6 5. O-O Be7 6. d3 b5 7. Bb3 d6 8. Be3 Na5 9. Nc3 Nxb3 10. axb3 O-O 11. Nxb5 Bg4 12. Na7 Qd7 13. Rxa6 c5 14. Qa1 Rxa7 15. Rxa7 Qxa7 16. Qxa7 Re8 17. Bg5 Bxf3 18. Bxf6 Bxf6 19. gxf3 Ra8 20. Qxa8+ Bd8 21. Qxd8#


Then I tried again with Genini:

White: Simultano (15 seconds / move)
Black: Gemini

1. e4 e5 2. Nf3 Nc6 3. Bb5 Bc5 4. c3 Nge7 5. d4 exd4 6. cxd4 Bb4+ 7. Bd2 Nxd4 8. Nxd4

Here black was unable to make a legal move despite many tries.

Your robot overlords are coming...but your're still safe in 2023 I think

[Werewolf]

I was right, Gemini was too good to be true.

No only does it fail at chess it also sucks at other things:

https://www.bbc.co.uk/news/technology-67650807

[AdminX]

I was right, Gemini was too good to be true.

No only does it fail at chess it also sucks at other things:

https://www.bbc.co.uk/news/technology-67650807

Haha, when you are caught with your hand in the cookie jar.

[Werewolf]

ChatGPT has been upgraded for paying users to version 4.5 Turbo.

It denied this, but when I asked it for its specific model number it told me. So what did I do with this new powerful AI? Play chess of course!

White: Carl Bicknell
Black: ChatGPT 4.5 Turbo

1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Bxc6 dxc6 5. O-O Bd6 6. d4 exd4 7. Qxd4 f6 8. Nbd2 Ne7 9. Nc4 Be6 10. b3 O-O 11. Rd1 Nc8 12. Ba3 Qe7 13. Nxd6 Rd8 14. Nxc8 Raxc8 15. Bxe7 Re8 16. Bxf6 gxf6 17. Qxf6 Rf8 18. Qxe6+ Kh8 19. Rd7 Rg8 20. Qf6+ Rg7 21. Qxg7#

It still sucks, but it's getting a little better. I'd estimate it's at 1200 Elo.

[Werewolf]

Gemini has been upgraded to Gemini Pro.
Google's ultimate goal is to launch Gemini Ultra in a few weeks, which they claim is smarter than ChatGPT.

In the meantime I played Gemini Pro to see if it was any better than Gemini at chess:

White: Carl Bicknell
Black: Gemini Pro

1. e4 c5 2. c3 d6 3. d4 Nf6 4. Bd3 Nc6 5. Nf3 e6 6. O-O Be7 7. Qe2 O-O 8. e5 d5 9. exf6 gxf6 10. Bh6

Here the machine tried 5 times in a row to play the illegal 10...Qh4. Several times it asked me for advice about what to play (!!).
After 3 attempts to play 10...Qh4 I warned it I would not tolerate this any longer and would claim a win.

After saying I was not playing in the spirit of the game the machine eventually conceded defeat after 5 tries.

Your robot Overlords are coming...but not yet.

[Werewolf]

Gemini Pro has given way to Gemini Ultra! This is an even bigger LLM, designed to slay ChatGPT.

I have to say its no better than before and was playing illegal moves at around move 7. ChatGPT is (a little) better. I wonder how long it'll be before it reaches 1400 elo?

[Vinvin]

Nice video testing Gemini Ultra : https://www.youtube.com/watch?v=gexI6Ai3X0U

"Today I own 3 cars but last year I sold 2 cars. How many cars do I own today ?"
- Gemini Ultra : 1 car (???)

"Here is a bag filled with popcorn. There is no chocolate in the bag. The bag is made of transparent plastic, so you can see what's inside. Yet, the label on the bag says 'chocolate' and not 'popcorn'. Sam finds the bag. She had never seen the bag before. She cannot see what is inside the bag. She reads the label. She believes that the bag is full of ..."
- Gemini Ultra : Chocolate (???)

[Werewolf]

Nice video testing Gemini Ultra : https://www.youtube.com/watch?v=gexI6Ai3X0U

"Today I own 3 cars but last year I sold 2 cars. How many cars do I own today ?"
- Gemini Ultra : 1 car (???)

"Here is a bag filled with popcorn. There is no chocolate in the bag. The bag is made of transparent plastic, so you can see what's inside. Yet, the label on the bag says 'chocolate' and not 'popcorn'. Sam finds the bag. She had never seen the bag before. She cannot see what is inside the bag. She reads the label. She believes that the bag is full of ..."
- Gemini Ultra : Chocolate (???)

Yes, it's shockingly beta. I note they've called it Gemini Ultra 1.0 to draw attention that the 1.0 will rise, but it's not ready for release IMO.

I asked it "In the Bible, who was stronger Samson or Goliath?" I was ready for any sensible answer, but not this one:
Gemini Ultra 1.0 " Samson was stronger because he killed Goliath"

However, one tester reported Google Deep Mind have trained a chess engine without any search at all to hit 2900 Elo and this may be incorporated into Gemini soon. I can't find a link for that, unfortunately.

[smatovic]

...
However, one tester reported Google Deep Mind have trained a chess engine without any search at all to hit 2900 Elo and this may be incorporated into Gemini soon. I can't find a link for that, unfortunately.

Vincent posted it in front of your nose

viewtopic.php?f=2&t=83320&p=958369#p958355
Grandmaster-Level Chess Without Search. Large-scale attention-based architectures and datasets of unprecedented scale.

--
Srdja

[Vinvin]

One more complete video : " Gemini Ultra is Here! (Google's "ChatGPT Killer")" https://www.youtube.com/watch?v=yr_OAkGIG7k