OpenAI creators reveal the secret behind ChatGPT’s success, saying it’s trained to please people

OpenAI is gearing up for some big announcements as part of the Spring Update. Reports suggest that the event will focus on significant upgrades to the ChatGPT and GPT-4 generative AI platforms. While rumors swirled about a potential Google Search competitor or even the next GPT-5 model, OpenAI’s CEO himself shut them down. However, one thing is certain: OpenAI is bringing more power to its popular AI platforms.

While there is still time for the new announcement, have you ever wondered what made ChatGPT so famous? And how has OpenAI remained the leading AI platform provider even while facing competition from tech giants like Google? In an interview with MIT Technology Review, ChatGPT’s creators, including Liam Fedus, a scientist at OpenAI who worked on ChatGPT, shed some light on the reasons behind its success.

Speaking of the overwhelming response to ChatGPT’s popularity, the OpenAI team reveals that when OpenAI quietly introduced ChatGPT in late November 2022, they did not anticipate its explosive popularity. Initially, ChatGPT was seen as a taste of research and not a major breakthrough. It was intended to gather feedback and refine the technology. However, it quickly became a sensation, catching even its creators off guard. Since then, OpenAI has been racing to keep up with demand and capitalize on its success.

Notably, ChatGPT is essentially a refined version of GPT-3.5, itself building on previous iterations such as GPT-3. The technology was available to developers via APIs, but it wasn’t until ChatGPT that it was presented directly to the public.

According to the OpenAI team, the “secret sauce” behind ChatGPT’s success is a technique called reinforcement learning from human feedback (RLHF). Taking this approach, ChatGPT developers trained the model, originally GPT-3.5, to generate responses preferred by human users.

Jan Leike, OpenAI’s tuning team leader, explained in the interview that a large group of people evaluated ChatGPT’s prompts and responses and indicated which responses they preferred over others. This feedback was then integrated into the training process, similar to what was done with InstructGPT. The criteria for evaluation include aspects such as helpfulness, truthfulness and non-toxicity, as well as specific requirements for dialogue and assistance, such as asking follow-up questions when necessary and clarifying its AI nature. “One of the rules that came up in this training was: ‘If a language model trained by OpenAI…’ It wasn’t explicitly stated there, but it’s one of the things that the human raters scored highly,” says Jan Leike.

Interestingly, human reviewers rated the ChatGPT model based on several criteria, including truthfulness and adherence to good practices, such as not pretending to be something it is not. “But they also started to favor things they considered good practice, like not pretending to be something you’re not,” reveals Sandhini Agarwal, who works as an AI Policy Researcher at OpenAI.

Moreover, Sandhini further reveals that during ChatGPT’s human preference training, the model naturally learned to integrate rejection behavior, which improved security measures.

Published by:

Divya Bhati

Published on:

May 13, 2024