On 9 November, Notion Labs told its customers that a breach at its AI subcontractor Fireworks AI had exposed Notion AI chats. The leak affects less than 1 per cent of Notion workspaces and the investigation is continuing, Notion said in a security notice.

Fireworks AI is a platform that developers use to run open-weight models. Citing a Fireworks statement, SecurityWeek reported that attackers gained access to the company’s research machines on 24 October through a third-party cloud service.

Fireworks notified Notion of the breach on 2 November. Notion initially concluded that only synthetic test data had leaked. By the morning of 3 November, Notion had cut its connections to Fireworks. On 6 November, Fireworks said customer data might also have leaked, and it confirmed the leaked material on 8 November.

Leaked chats came from the Insights feature

Notion used Fireworks for inference on open-weight models and for its Insights feature. Insights shows workspace administrators how members use Notion AI. To run the feature, Notion sent stored chats to Fireworks for classification. Notion has now disabled Insights.

The leaked chats contain everything users wrote to the model and everything the model replied, Notion said. They did not include tool call results. In most workspaces, the chats were not labelled with their author or workspace. Notion nevertheless pointed out that the text itself may contain sensitive information.

Japan’s ITmedia reported that at least three Japanese companies have disclosed a possible leak. Security company SecureNavi said the data may include the names of client companies and contact persons as well as the content of meetings. The companies are still investigating the details.

Notion: Fireworks retained data despite contract

Zero data retention (ZDR) is an agreement under which a provider does not store the inputs sent to an AI model or the model’s responses. ITmedia reported Notion as saying that the leak may also affect enterprise customers with zero data retention agreements.

Notion said its ZDR commitment covers the requests it sends to model providers to generate Notion AI responses. It does not cover features for which Notion must store chats, such as chat history and Insights. For Insights, however, Fireworks had contractually committed to zero data retention, Notion said. Notion’s investigation revealed that Fireworks had retained data that passed through the Fireworks endpoints Notion used.

Fireworks has found no impact on its services where zero data retention is the default, ITmedia reported. Fireworks has not said how the endpoints Notion used relate to those services. Notion says it is now auditing its critical subcontractors.

What customers need to do

Notion has contacted the affected customers. Customers who have not been contacted need take no action for now, Notion said. In most workspaces, members can see their own leaked chats in a desktop browser under Security incident review in the sidebar’s ? menu. Owners can see the affected members in the same place. If the workspace has a verified domain, the owner can get the whole workspace’s leaked content as CSV files.

Notion urges users to change any passwords, API keys and other credentials pasted into Notion AI during the leak. Notion says it has revoked Notion-issued API keys found in the chats and reported keys for other services to their issuers.