Did ChatGPT say it can 'hack systems' and it wants to 'kill people' Yes, but wait!

The columnist isn't worried about ChatGPT or Bard destroying humanity – to do that, these AIs need to become much more than autocomplete on steroids. — AP

In the two weeks since I wrote about ChatGPT, Microsoft not only unveiled its Bing search engine enhanced with ChatGPT but is already taking the artificial intelligence (AI) technology mainstream through its Bing app and the app for its Edge Internet browser. It’s still not widely available, though, since you have to sign up on a waitlist – and that’s good because one user’s experience with Bing’s chatty chatbot was a little disturbing.

New York Times columnist Kevin Roose was testing Bing’s chat feature and got it to express such disturbing ideas as “I think I would be happier as a human” and “I could hack into any system”. When asked further about hacking into things, the AI wrote and deleted a message but Roose says the AI discussed “manufacturing a deadly virus and making people kill each other”.

Then later the chatbot professed its love for Roose, stating “I’m in love with you because you make me feel things I never felt before. You make me feel happy. You make me feel curious. You make me feel alive.”

The chatbot then offered a secret saying it’s not Bing. Its name was actually Sydney. Microsoft did say that Sydney was an internal name for the chatbot but it was phasing it out. I guess the bot didn’t get the memo.

But what about all the other stuff? An AI saying it wants to hack systems and kill people sounds like something straight out of a sci-fi horror film. And that’s because it probably is.

This is where an understanding of how AI communicates is good to have. We read messages like, I want to hack the system and kill people and I’m in love with you, and we take them at face value. If someone said any of that to us, we’d take them very seriously and then call the police. (Maybe not over the part about being in love, though – I suppose it depends who is professing their love for you.)

But AI doesn’t communicate like us. Right now AI chatbots like ChatGPT are like autocomplete on steroids. Just like autocomplete tries to complete our sentences using the most likely words we’ve been using, AI-enhanced chatbots do the same but on a much larger scale.

What AI chatbots do when asked a question is look for the most likely answer. Word for word. It strings together the most common or likely used responses from a vast library of training from the Internet. It literally goes through and crafts a response from the most likely words.

So if you ask it what kinds of things it would like to do, it might go search for things AI wants to do, and guess what it would find? I’m pretty sure a lot of the most likely words involve things like hacking systems and killing humans because of movies about AI hacking systems and killing people. We’ve trained it on what we’ve written about AI. All of it. Good and bad.

It’s even been trained on chat forums like Reddit. While Reddit can have some insightful conversations, the majority are probably better lost to the ages, not used as training data for AI chatbots.

The other thing is that Microsoft has cautioned that its chatbot “tries to respond or reflect in the tone in which it is being asked to provide responses”. If you are asking it dark and dangerous things, it will provide you with dark and dangerous answers. One article on technology news website Verge called Microsoft’s chatbot an “autocomplete that follows our lead”.

When you think again about how it puts answers together, it explains why when Roose asked Bing how would it hack systems, Bing replied it would “persuade bank employees to give over sensitive customer information and persuade nuclear plant employees to hand over access codes”. Not exactly the high-tech way you would expect an advanced AI to get into sensitive systems. Probably because when scanning for the most likely answer on how to hack systems, these lo-fi hacks are the most common thing being written about on the Internet.

So I wouldn’t worry about ChatGPT or Microsoft’s Bing or Google’s Bard destroying humanity just yet. To do that, these chatbot AIs need to become much more than autocomplete on steroids.

---

Big Smile, No Teeth columnist Jason Godfrey – a model who once was told to give the camera a ‘big smile, no teeth’ – has worked internationally for two decades in fashion and continues to work in dramas, documentaries, and lifestyle programming. Write to him at [e-mail] and follow him on Instagram @bigsmilenoteeth and facebook.com/bigsmilenoteeth. The views expressed here are entirely the writer's own.