ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial3 min readDisconfirmAI Editorial

Fireworks AI CEO Lin Qiao: Two Standard Modes for AI API Data Retention, Advocating Ephemeral Processing When Workflows Allow

Lin Qiao, co-founder and CEO of Fireworks AI, outlines two standard lifecycle designs for AI inference and training APIs: customer-managed upload-use-delete and ephemeral send-compute-return. She argues that the latter should be prioritized when workflows permit, as it eliminates human errors such as forgotten datasets or missed deletion calls.

2026-11-14 10:46 GMT+8

In AI infrastructure, data retention is one of the most misunderstood terms. In a November 2026 post on X, Lin Qiao, co-founder and CEO of Fireworks AI, proposed a clear framework for thinking about this: AI inference and training APIs follow two standard design patterns, distinguished by who manages the data lifecycle and where.

The first pattern is customer-managed retention: upload, use, delete. Users create datasets or files, submit jobs, and then explicitly delete the data. The platform keeps the data stored and available until it receives a deletion instruction. Batch APIs are a typical example—OpenAI Batch, AWS Bedrock Batch, GCP Gemini Batch, and Fireworks Batch all follow this standard. The platform cannot know when the user no longer needs the data unless the user defines it; deletion is the user's responsibility.

The second pattern is ephemeral processing: send, compute, return. Users send data, the platform uses it during request processing, and returns the result without retaining the content afterward. Standard inference APIs and serverless training fall into this category. Applications own their conversation history, with no separate platform cleanup step required.

Lin Qiao used an intuitive analogy: saving a spreadsheet to a cloud drive versus passing numbers to a calculator. The former requires the service to keep the file for later use, while the latter only requires computing the answer and returning it immediately.

She advocates that ephemeral processing should be prioritized whenever workflows allow, not for performance or cost reasons, but to avoid human error. Ephemeral processing leaves no forgotten datasets and no missed deletion calls. Every retained copy of data adds something that must be protected and eventually deleted. If data never reaches disk, it cannot be taken from disk later.

Sign in / Sign up
ReadingResearchRadarInvestment framework

The practical implication of this framework is that zero data retention (ZDR) is not a vendor label but an endpoint attribute. A single platform can provide endpoints for both modes. OpenAI’s Chat Completions defaults store to false, falling under ephemeral processing; however, its Batch and Files endpoints involve customer-managed retention, with data persisting until deletion. Amazon Bedrock’s Converse API uses content solely to generate responses, whereas its Batch inference reads and writes to user-specified S3 locations. Google’s Gemini text generation is stateless when store is false, but its Interactions API has store defaulting to true.

However, this framework has clear boundaries. Fine-tuning based on datasets requires data at rest, making customer-managed retention indispensable in these scenarios. Additionally, even when zero retention is declared, both OpenAI and AWS note that content flagged as CSAM may still be retained. Default settings, approval requirements, and exception conditions vary significantly across platforms, so developers must verify settings per endpoint.

It is worth noting that the article explaining this framework on MarkTechPost is explicitly labeled as sponsored content, with the Fireworks team providing thought leadership and resources. While Lin Qiao’s framework serves as a clear technical classification, her preference for ephemeral processing aligns with Fireworks’ own product positioning for its ephemeral APIs, indicating a potential conflict of interest.

Practical advice for developers: map every endpoint you call to one of the two modes; prioritize ephemeral processing endpoints when you only need results; plan deletion calls for each upload and record documented expiry times in your documentation; audit persistence flags like store; define ZDR by processing path in contracts, then verify the settings in your account.

Sources

  • x.comX (formerly Twitter)↗
  • marktechpost.comMarkTechPost↗
Continue researching →