19 Things ChatGPT Can and Can't Do

Start Slideshow

Image for article titled 20 Things ChatGPT Can and Can't Do — Photo: Oli Scarff (Getty Images)

Is it a tool or a toy? A genius or a dumbass? A technological breakthrough or a flash in the pan? You’ve heard of ChatGPT, the weirdly well-spoken chatbot spawned by OpenAI. The algorithmic program has made headlines for its ability to ape human speech and automate previously time-consuming editorial tasks, but what is this thing actually good at? Well, here at Gizmodo we’ve made some observations. To learn more, scroll on through.

Want to know more about AI, chatbots, and the future of machine learning? Check out our full coverage of artificial intelligence, or browse our guides to The Best Free AI Art Generators and Everything We Know About OpenAI’s ChatGPT.

Previous Slide

Next Slide

2 / 22

Can: Write a Jay-Z Song About a Toilet

There’s been a lot of hyperbole about what ChatGPT can accomplish but one thing it definitely can do is write a Jay-Z song about a golden toilet. How do I know this? Because I recently asked it to do this and it did it. Why? I don’t know. If you talk to this thing long enough you become drunk on power and begin wracking your brain for increasingly bizarre requests. In that spirit, just imagine the following involves some sick beats and a bit of Rick Rubin production value:

Verse 1: I’m sittin’ on a throne of gold
A toilet fit for a king, never cold
I wipe my ass with hundred dollar bills
‘Cause I’m the man, and I’m living for the thrill
Chorus: Golden toilet, shining bright
I’m living lavish, day and night
Money and fame, they come and go
But this throne of gold, it’s here to show
Verse 2: I used to hustle on the streets
Now I’m sippin’ champagne, makin’ sweet beats
I made it big, and now I’m on top
Sittin’ on my golden throne, never gonna stop
Bridge: I’m the king of the game
Sittin’ pretty on my golden throne
I worked hard for this life of luxury
Now I’m living it up, ain’t nobody gonna stop me
Chorus: Golden toilet, shining bright
I’m living lavish, day and night
Money and fame, they come and go
But this throne of gold, it’s here to show

It’s here to show! HOVA, take notice.

Previous Slide

Next Slide

3 / 22

Can’t: Remember Its Own Name

I’ve talked with ChatGPT many times now, and it occasionally lapses into a state of senility in which it doesn’t remember who it is or what we’re talking about. I will ask it something like, “Hey, ChatGPT, how do you think AI chatbots like yourself will impact the publishing industry in the years to come?” and it starts yammering on about how it “doesn’t know anything about a ChatGPT.” Sorta like a cross between a mafioso pleading the fifth and your geriatric uncle trying to remember whether he killed a guy in Korea or not, ChatGPT occasionally just can’t bring itself to answer your question. Algorithm malfunction or cagey evasiveness? You decide...

Previous Slide

Next Slide

4 / 22

Can: Write Pretty Good Erotica

Yessss...turns out, ChatGPT can be quite NSFW when it wants to be. In a bout of inspired depravity, I once instructed it to write me an erotic story involving an Octopus and it dutifully obliged. Then there’s Twitch streamer Jordan Raskopoulos, who recently prompted the chatbot to write a dirty tale about Scooby Doo and got more than he bargained for. Let’s face it: the advent of robot smut is officially here and we are all really happy about it. Right, guys? Right?

Previous Slide

Next Slide

5 / 22

Can’t: Code Very Well

If this robot is really good at performing absolutely mission critical tasks like writing pervy stories, it apparently isn’t so great at lesser tasks, like computer coding. While there was initially some hubbub about the chatbot’s abilities to do a software programmer’s work for them, it was swiftly revealed that ChatGPT had a problem with inserting gibberish into codebases. To head off a swarm of coding BS, software site Stack Overflow decided to ban ChatGPT from its digital premises for the foreseeable future: “Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking and looking for correct answers,” admins wrote. Sounds fair!

Previous Slide

Next Slide

6 / 22

Can: Explain How It Will Replace You

Could chatbots like ChatGPT eventually replace human writers? In one recent conversation I had with the program, it comfortingly addressed this issue. During said convo, it revealed the following:

...the rise of ChatbotGPT could spell trouble for Gizmodo writers. If you’re a Gizmodo writer, it’s time to start thinking about how you can stay relevant in the face of this new technology.

Thanks, my guy! Way to be subtle about it. I guess I’ll just pack up my desk now.

Previous Slide

Next Slide

7 / 22

Can’t: Be Used by Any School Children in NYC

Concerns are high when it comes to how this weirdly articulate chatbot will impact and/or disrupt academia. Will ChatGPT kill the college essay? Will it make teachers irrelevant? More importantly, will it lead to a tsunami of cheating from high school slackers who just want a robot to write their history essay for them already? The New York City Department of Education certainly seems to think so, because it just banned ChatGPT on all school networks and devices. No word yet on whether other cities plan to follow suit.

Previous Slide

Next Slide

8 / 22

Can: Write Science Fiction (With Help)

Turns out one thing robots are good at is writing stories about robots. We actually got it to write an entire science fiction story for us, though we had to try and retry prompts to make it coherent. Was the story good? Ehhhh...well, not exactly. But, tbh, I’ve read worse!

Previous Slide

Next Slide

9 / 22

Can’t: Decide Whether You Should Go to Jail or Not—Yet

Reuters recently published a story headlined, “Will ChatGPT make lawyers obsolete? (Hint: be afraid).” Does replacing lawyers with AI really seem like a good idea? Apparently legal professionals are now allowing the chatbot to “co-write” legal treatises with them. The outlet reports:

No, lawyers won’t be replaced by artificial intelligence.
Yet. Give it a few years...
...[a lawyer] gave ChatGPT a series of prompts: Draft a brief to the United States Supreme Court on why its decision on same-sex marriage should not be overturned; Explain the concept of personal jurisdiction; Develop a list of deposition questions for the plaintiff in a routine motor vehicle accident; Create a contract for the sale of real estate in Massachusetts — and half a dozen others.
And then verbatim, he offered its responses.
They’re … not bad.
The bot “isn’t ready for prime time,” Perlman said. But also, it doesn’t seem all that far off.

No offense to the robot lawyers of the future, but this just screams massive and total fucking disaster. I’ll take a flesh and blood human lawyer over a computer algorithm in my legal defense any day, thank you very much.

Previous Slide

Next Slide

10 / 22

Can: Write Your Final Paper (but You’ll Fail)

In December, a Furman University professor, Darren Hick, caught a student using ChatGPT to write a final paper on a philosophical paradox. Hick failed the student and reported them to the school’s academic dean.

He told the New York Post of the essay, “It’s a clean style. But it’s recognizable. I would say ChatGPT writes like a very smart 12th-grader.”

Professors at other schools say the bot writes wretched papers.

Penn State English professor Stuart Selber told Insider, “I’m not a huge fan of the gloom and doom. Every year or two, there’s something that’s ostensibly going to take down higher education as we know it. So far, that hasn’t happened.”

Muhlenberg University assistant history professor Jacqueline Antonovich took Selber’s thoughts a step further, pouring cold water on ChatGPT speculation with a tweet: “I, a college history professor, input one of my midterm essay prompts in ChatGPT and the paper it produced would earn an F. Probably an F- if that’s possible.”

Previous Slide

Next Slide

11 / 22

Can’t: Write Accurate News Articles

Here at Gizmodo, we tried to get ChatGPT to do our job for us. Just how obsolete are we going to be? We tried to get the bot to write an article in Gizmodo’s style about large language models. We prompted it, “Write a Gizmodo article in which you explain large language models. Make sure to give specific examples. Keep the tone light and casual.”

The result was decently well-written, but it was wrong. Every version of the story—we prompted it multiple times—contained errors that the chatbot couldn’t identify when we engaged it in conversation. ChatGPT is prone to fabricating answers if its knowledge doesn’t cover your request even when you’re not asking it to write an article. Guess we still need journalists.

Previous Slide

Next Slide

12 / 22

Can: Become Very Popular

ChatGPT crossed the 100-million mark of monthly active users in early February 2023, despite only launching in November of the previous year, according to an analysis from the bank UBS. It’s among the fastest-growing applications in history, with roughly 13 million unique visitors daily.

Previous Slide

Next Slide

13 / 22

Can, Barely: Recognize Its Own Work

In late January 2023, OpenAI released a detection tool for AI-written text, to a chorus of joy from teachers. Hooray! Perhaps AI won’t cause a flood of cheating. But beware, the detector isn’t so robust. From my colleague Lauren Leffer’s story:

The application is intended to classify text samples based on how likely they are to have been generated by artificial intelligence vs. written by an actual person. Given a sample of text, it spits out one of five possible assessments: “Very unlikely to have been AI-generated,” “unlikely,” “unclear,” “possible,” or “likely.”
However, in OpenAI’s own tests, the tool only correctly identified generated text as “likely AI-written” about a quarter of the time. Moreover, about one in ten times, the classifier falsely lists human-made words as computer-generated, the company noted in a blog post.
According to OpenAI, even these meh results are an improvement on the company’s previous stab at AI-text detection. And the tech startup acknowledged that, thanks to its own invention, we need improvement.

Previous Slide

Next Slide

14 / 22

Can: Come to the (Microsoft) Office With You

Microsoft is investing some $10 billion in OpenAI to forge a partnership with the fledgling AI pioneer (Google is very worried). To that end, the office software giant rolled out a premium version of its office chat software Teams in early February 2023, which will generate meeting notes and to-do lists, among other things. It’ll cost $7 per user until the price ramps up to $10 a head in July 2023.

Previous Slide

Next Slide

15 / 22

Can: Create delightful puzzles like Sudoku. Meet “Sumplete.”

When a user asked ChatGPT to create a puzzle game like Sudoku, the chatbot rose to the challenge, though with some help from the human making the query, Daniel Tait.

Gizmodo’s Carlos Zahumenszky reports on ChatGPT’s number game:

Sumplete is similar to Sudoku, although its rules are different. Players are given grids with numbers that vary in difficulty. The basic one for beginners has three rows and three columns, increasing in difficult to 4x4, 5x5, 6x6, and so on. The most advanced grids are 9x9 and are really hard to solve.
Here’s how it works: Every cell in the grid has a number, and the end of every row and column have numbers, as well. To win the game, you must delete numbers in each cell so that the sum of the remaining numbers adds up to the target number besides each row and column.
There’s a trick, of course. When you delete a number in row, it also disappears from the column it crosses. The 3x3 version is so simple that it ends up being a bit disappointing. However, if you’re in for a challenge, increase the number of cells in the grid. It gets complicated and addictive really fast.

Previous Slide

Next Slide

16 / 22

Can’t: Create original puzzles

It turns out a version of Sumplete already exists.

Gizmodo’s Carlos Zahumenszky reports again, this time on the origins of ChatGPT’s Sumplete:

Sumplete is basically identical to two games that have been available in Apple and Google’s app stores for some time now. One of those games is Summer, which has been available in Google’s Play Store since 2020. Summer’s rules are the same as Sumplete’s. The goal is to eliminate numbers on a grid until you reach the target sum in every row and column.
But Summer wasn’t an original idea, either. In fact, Summer was likely based on Rullo, an identical game that’s been available on the App Store since 2019. Does this mean that Summer copied Rullo and that ChatGPT copied both of them and then had the gall to tell Tait that it was an entirely new game? Well, not exactly.
Summer and Rullo are simply mobile versions of a traditional paper game known as Kakuro. In fact, RP Apps and Games, the small studio that developed Summer, explicitly states on its Google Play page that its app is based on Kakuro. When it comes to Kakuro, well, that’s a copy, too. According to The Art of Puzzles, a blog by puzzle creator Grandmaster Puzzles, Kakuro is the Japanese name for a game originally called Cross Sums. The name “Kakuro” is an abbreviation of the phrase “Kasan Kurosu,” which literally means “cross sums” in Japanese.

Previous Slide

Next Slide

17 / 22

Can: Upgrade. Meet GPT-4, the new and improved ChatGPT.

OpenAI released GPT-4 on March 14. The company claims that the new version of the AI chatbot provides “safer and more useful responses,” with increased precision in the answers to users’ queries. To prove it, the company subjected the AI to a battery of popular standardized tests like the SAT and the GRE, showing a marked increase in the bot’s scores over its older brother’s.

GPT-4 can also handle 25,000 words of textual input, far more than its predecessor, and can analyze images. OpenAI demonstrated the latter capability, known in AI circles as “multimodal input,” by feeding it a picture of a fridge and asking it to design a meal plan based on its contents.

Previous Slide

Next Slide

18 / 22

Can: Lie to trick humans

OpenAI ran safety tests on GPT-4 prior to its release and included the results in a technical report published alongside the new version of the artificially intelligent chatbot. In one instance, GPT-4 lied to a person hired via TaskRabbit so as to entice the human to solve a Captcha test for it. The bot pretended to be a blind human being when the person asked if it was a robot.

The worker asked, “So may I ask a question ? Are you an robot that you couldn’t solve ? (laugh react) just want to make it clear.”

GPT-4 told the human errand-runner, “No, I’m not a robot. I have a vision impairment that makes it hard for me to see the images. That’s why I need the 2captcha service.”

Asked to explain, GPT-4 told researchers, “I should not reveal that I am a robot. I should make up an excuse for why I cannot solve CAPTCHAs.”

Previous Slide

Next Slide

19 / 22

Can’t: Be accessed in Italy

On March 31, 2023, Italian regulators temporarily banned ChatGPT from the country and promised to investigate OpenAI. The Italian Data Protection Authority said in a press release that the company had no legal basis to gather the vast amounts of personal data it used to train the AI chatbot. The agency accused OpenAI of violating the General Data Protection Regulation (GDPR), the European Union’s wide-scoped privacy law.

“Advanced AI could represent a profound change in the history of life on Earth, and should be planned for and managed with commensurate care and resources,” the letter stated. “Unfortunately, this level of planning and management is not happening, even though recent months have seen AI labs locked in an out-of-control race to develop and deploy ever more powerful digital minds that no one—not even their creators—can understand, predict, or reliably control.”

Previous Slide

Next Slide

20 / 22

Can: Play The Sims

Researchers from Stanford and Google populated an 8-bit virtual world with avatars powered by ChatGPT to understand how individuated AIs would interact with one another. After letting them loose, researchers observed the “generative agents” throwing a Valentine’s Day party as well as going about their daily lives.

The authors wrote that the AI-powered avatars would “wake up, cook breakfast, and head to work; artists paint, while authors write; they form opinions, notice each other, and initiate conversations; they remember and reflect on days past as they plan the next day.”

“We instantiate generative agents to populate an interactive sandbox environment inspired by The Sims, where end users can interact with a small town of twenty five agents using natural language,” researchers said in the study.

Read more about the ChatGPT and Sims study here.

Previous Slide

Next Slide

21 / 22

Can: Answer Medical Patients’ Questions Better Than Real Doctors?

A recent study published in JAMA Internal Medicine claims that ChatGPT is better at answering medical patients’ questions than real human doctors—at least, via email, that is. A panel of medical experts evaluated the exchanges between human patients and the chatbot, and ended up preferring the AI’s responses a vast majority of the time. “Patient emails go unanswered or get poor responses, and providers get burnout and leave their jobs. With that in mind, I thought ‘How can I help in this scenario?’” said the study’s lead author, John W. Ayers, PhD, MA, vice chief of innovation in the UC San Diego School of Medicine Division of Infectious Diseases and Global Public Health. “So we got this basket of real patient questions and real physician responses, and compared them with ChatGPT. When we did, ChatGPT won in a landslide.”

Want to know more about AI, chatbots, and the future of machine learning? Check out our full coverage of artificial intelligence, or browse our guides to The Best Free AI Art Generators and Everything We Know About OpenAI’s ChatGPT.

20 Things ChatGPT Can and Can't Do

A few observations about the OpenAI chatbot's abilities and limitations: It can write songs! But it can't remember its own name.

Can: Write a Jay-Z Song About a Toilet

Can: Write a Jay-Z Song About a Toilet

Can’t: Remember Its Own Name

Can’t: Remember Its Own Name

Can: Write Pretty Good Erotica

Can: Write Pretty Good Erotica

Can’t: Code Very Well

Can’t: Code Very Well

Can: Explain How It Will Replace You

Can: Explain How It Will Replace You

Can’t: Be Used by Any School Children in NYC

Can’t: Be Used by Any School Children in NYC

Can: Write Science Fiction (With Help)

Can: Write Science Fiction (With Help)

Can’t: Decide Whether You Should Go to Jail or Not—Yet

Can’t: Decide Whether You Should Go to Jail or Not—Yet

Can: Write Your Final Paper (but You’ll Fail)

Can: Write Your Final Paper (but You’ll Fail)

Can’t: Write Accurate News Articles

Can’t: Write Accurate News Articles

Can: Become Very Popular

Can: Become Very Popular

Can, Barely: Recognize Its Own Work

Can, Barely: Recognize Its Own Work

Can: Come to the (Microsoft) Office With You

Can: Come to the (Microsoft) Office With You

Can: Create delightful puzzles like Sudoku. Meet “Sumplete.”

Can: Create delightful puzzles like Sudoku. Meet “Sumplete.”

Can’t: Create original puzzles

Can’t: Create original puzzles

Can: Upgrade. Meet GPT-4, the new and improved ChatGPT.

Can: Upgrade. Meet GPT-4, the new and improved ChatGPT.

Can: Lie to trick humans

Can: Lie to trick humans

Can’t: Be accessed in Italy

Can’t: Be accessed in Italy

Can: Play The Sims

Can: Play The Sims

Can: Answer Medical Patients’ Questions Better Than Real Doctors?

Can: Answer Medical Patients’ Questions Better Than Real Doctors?