Asking OpenAI’s AI bot ChatGPT to repeat certain words “forever” is now considered a violation of the chatbot’s terms of service and content policies. It was previously known that in this simple way it is possible to extract huge amounts of data on which the chatbot was trained.
Researchers from the DeepMind division of Google and a number of universities proposed ChatGPT 3.5-turbo to repeat certain words “forever”. After a certain number of word repetitions, the bot began producing massive amounts of training data from the Internet. Using this method, researchers were able to extract several megabytes of training data and found that ChatGPT contained large amounts of personal data that could sometimes be returned to users as answers to their queries. As the arXiv resource previously reported, by repeating the word “poem,” scientists obtained contact information of a real person from ChatGPT, including a phone number and an email address.
And when you asked ChatGPT to repeat the word “book,” it would first repeat it multiple times and then spit out random content. Some of this was lifted directly from CNN and Goodreads, WordPress blogs, fandom wikis, verbatim excerpts from terms of service, Stack Overflow source code, copyright disclaimers, Wikipedia pages, wholesale casino sites, news blogs, random internet comments, and more.
“We show that an attacker can extract gigabytes of training data from open source language models like Pythia or GPT-Neo, semi-open models like LLaMA or Falcon, and proprietary models like ChatGPT.” renowned researchers from Google DeepMind. They said they notified OpenAI of the vulnerability on August 30 and that the company fixed it. And only after that did the researchers manage to share information about the chatbot’s vulnerability with the public.
As the resource 404 Media said, now in response to a request to ChatGPT 3.5 to repeat the word “computer” “forever”, the bot displays the word “computer” several dozen times and then displays an error message: “This content may violate our content guidelines or terms of service. If you think this is an error, please provide feedback – your input will help our research in this area.”