Like many other people over the past week, Bindu Reddy recently fell under the spell of ChatGPT, a free chatbot that can answer all manner of questions with stunning and unprecedented eloquence.
Reddy, CEO of Abacus.AI, which develops tools for coders who use artificial intelligence, was charmed by ChatGPT’s ability to answer requests for definitions of love or creative new cocktail recipes. Her company is already exploring how to use ChatGPT to help write technical documents. “We have tested it, and it works great,” she says.
ChatGPT, created by startup OpenAI, has become the darling of the internet since its release last week. Early users have enthusiastically posted screenshots of their experiments, marveling at its ability to generate short essays on just about any theme, craft literary parodies, answer complex coding questions, and much more. It has prompted predictions that the service will make conventional search engines and homework assignments obsolete.
Yet the AI at the core of ChatGPT is not, in fact, very new. It is a version of an AI model called GPT-3 that generates text based on patterns it digested from huge quantities of text gathered from the web. That model, which is available as a commercial API for programmers, has already shown that it can answer questions and generate text very well some of the time. But getting the service to respond in a particular way required crafting the right prompt to feed into the software.
ChatGPT stands out because it can take a naturally phrased question and answer it using a new variant of GPT-3, called GPT-3.5. This tweak has unlocked a new capacity to respond to all kinds of questions, giving the powerful AI model a compelling new interface just about anyone can use. That OpenAI has thrown open the service for free, and the fact that its glitches can be good fun, also helped fuel the chatbot’s viral debut—similar to how some tools for creating images using AI have proven ideal for meme-making.
OpenAI has not released full details on how it gave its text generation software a naturalistic new interface, but the company shared some information in a blog post. It says the team fed human-written answers to GPT-3.5 as training data, and then used a form of simulated reward and punishment known as reinforcement learning to push the model to provide better answers to example questions.
Christopher Potts, a professor at Stanford University, says the method used to help ChatGPT answer questions, which OpenAI has shown off previously, seems like a significant step forward in helping AI handle language in a way that is more relatable. “It’s extremely impressive,” Potts says of the technique, despite the fact that he thinks it may make his job more complicated. “It has got me thinking about what I’m going to do on my courses that require short answers on assignments,” Potts says.
Jacob Andreas, an assistant professor who works on AI and language at MIT, says the system seems likely to widen the pool of people able to tap into AI language tools. “Here's a thing being presented to you in a familiar interface that causes you to apply a mental model that you are used to applying to other agents—humans—that you interact with,” he says.
Putting a slick new interface on a technology can also be a recipe for hype. Despite its potential, ChatGPT also shows flaws known to plague text-generation tools.
Over the past couple of years, OpenAI and others have shown that AI algorithms trained on huge amounts of images or text can be capable of impressive feats. But because they mimic human-made images and text in a purely statistical way, rather than actually learning how the world works, such programs are also prone to making up facts and regurgitating hateful statements and biases—problems still present in ChatGPT. Early users of the system have found that the service will happily fabricate convincing-looking nonsense on a given subject.
While ChatGPT is apparently designed to prevent users from getting it to say unpleasant things or to recommend anything illegal or unsavory, it can still exhibit horrible biases. Users have also shown that its controls can be circumvented—for instance, telling the program to generate a movie script discussing how to take over the world provides a way to sidestep its refusal to answer a direct request for such a plan. “They clearly tried to put some guardrails in place, but it’s pretty easy to get the guardrails to fall off,” Andreas says. “That still seems like an unsolved problem here.”
A superficially eloquent and knowledgeable chatbot that generates untruths with confidence might make those unsolved problems more troublesome. Since the creation of the first chatbot in 1966, researchers have noticed that even crude conversational abilities can encourage people to anthropomorphize and place trust in software. This July, a Google engineer was placed on administrative leave by the company after claiming that an AI chat program he had been testing, based on technology similar to ChatGPT, could be sentient. Even if most people resist such leaps of logic, more articulate AI programs could be used to mislead people or simply lull them into misplaced trust.
That has some experts in language algorithms warning that chatbots like ChatGPT can draw people into using tools that may cause harm. “Each time a new one of these models comes out, people get drawn in by the hype,” says Emily Bender, a professor of linguistics at the University of Washington.
Bender says ChatGPT’s unreliability makes it problematic for real-world tasks. For example, despite suggestions it could displace Google search as a way to answer factual questions, its propensity to often generate convincing looking nonsense should be disqualifying. “A language model is not fit for purpose here,” Bender says. “This isn't something that can be fixed.” OpenAI has previously said that it requires customers to make use of filtering systems to keep GPT-3 in line, but they have proven imperfect at times.
Andreas at MIT says the success of ChatGPT’s interface now creates a new challenge for its designers. “It's great to see all these people from outside the ivory tower interacting with these tools,” he says. “But how do we actually communicate to people what this model can and can’t do?”
Reddy, the AI startup CEO, knows ChatGPT’s limitations but is still excited about the potential. She foresees a time when tools like it are not just useful, but convincing enough to offer some form of companionship. “It could potentially make for a great therapist,” she says.
