People reading your chats on ChatGPT: OpenAI’s ‘Project Lily’ sparks controversy

The assumption that what users write on ChatGPT is read only by machines is no longer holding up. An exclusive report by US investigative news outlet 404 Media has revealed that OpenAI has hired hundreds of contract workers who read a vast stream of actual prompts submitted by users—some of which may contain sensitive personal information. The programme is internally known by the codename “Project Lily.”

According to 404 Media, reviewers can sometimes see entire conversations between users and the chatbot. Yet most of ChatGPT’s more than 900 million users are probably unaware that real people may be reading their conversations.

Improving the Quality of Responses

According to the report, the purpose of the reviews is to improve the chatbot’s responses. Workers evaluate and critique ChatGPT’s answers, assigning each response a score on a scale of 1 to 7.

Internal documents obtained by 404 Media show that these workers are also involved in training ChatGPT not to present itself as human or engage in excessive flattery. The report said excessive agreeableness has become a major problem for OpenAI, which has faced several lawsuits related to the issue.

404 Media reported that the workers are recruited through a platform called Crossing Hurdles and paid through Mercor. One worker said they earn more than $50 an hour.

Names Hidden, But Risks Remain

According to 404 Media, reviewers cannot see ChatGPT users’ names. OpenAI says it attempts to remove personal information from prompts before they reach human reviewers. However, the company has acknowledged that some sensitive information may still remain.

The report said the risk is significant because people often use ChatGPT as a therapist, professional assistant or digital companion, sharing deeply personal details about their lives.

Were Users Told?

404 Media asked OpenAI where users had been informed that humans might read their chats. OpenAI did not answer the question. After the report was published, the company only pointed to a help page.

The report also said that, for consumer plans, permission to use users’ data for model training is enabled by default.

Others Are Doing the Same

According to 404 Media, human review is not unique to OpenAI. Anthropic has confirmed that it also carries out similar work involving human reviewers. Google has a comparable programme as well.

The report highlights a broader misconception about how AI models are improved. Their progress is not driven solely by gathering vast amounts of information from the internet, teams of highly skilled engineers or the capabilities of new models. The repeated reading and evaluation of responses by external contract workers also plays an important role.

What Users Can Do

Users can go to the Data Controls section in ChatGPT’s settings and turn off permission to use their chats for model training. They can also use Temporary Chat.

It is also advisable to exercise caution before sharing sensitive health, financial or family information with an AI chatbot.