ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial2 min readDisconfirmAI Editorial

Open-source database Valkey releases 9.2 with distributed KV caching to cut AI inference costs

Valkey 9.2 introduces shared KV caching across GPU fleets, claiming up to 40% lower AI inference costs and 80% reduced time-to-first-token.

2026-11-19 23:47 GMT+8

Valkey, an open-source in-memory datastore governed by the Linux Foundation, released version 9.2 on November 19, 2026. The core highlight of this release is the introduction of distributed key-value (KV) caching, designed to address the waste of GPU compute caused by repeatedly processing the same prompt instructions, tool definitions, and conversation history in agentic applications.

According to official release notes and early adopter reports, establishing a shared cache layer across GPU clusters allows inference engines to reuse previous computations rather than recomputing them. Valkey claims this mechanism can reduce AI inference costs by up to 40% and shave time-to-first-token (TTFT) by up to 80%. Deployment requires no application code changes and supports cloud-agnostic environments.

Multiverse Computing, a compressed model provider, provided real-world data points: after integrating Valkey-based distributed KV caching, they processed 50% more requests on the same GPUs while maintaining a median TTFT under one second. They reported a 23% reduction in median TTFT at normal traffic and a 95% reduction at peak traffic. These performance gains are currently based on vendor self-tests and early user feedback, lacking widespread independent third-party benchmark verification.

Additionally, Valkey 9.2 introduced "forkless snapshots," eliminating the memory usage doubling traditionally associated with in-memory backups. This allows operators to size hardware based on actual application needs rather than worst-case backup windows. The new version also optimizes cross-region replication traffic, claiming to reduce network transfer fees by approximately 50% to 65%.

Sources

Sign in / Sign up
ReadingResearchRadarInvestment framework
  • diginomica.comdiginomica↗
Continue researching →