LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasTema · QuePaxa共识算法Material4 min de lecturaEditorial de DisconfirmAI

Cloudflare Engineers Explain: Why Raft Stalls on the Global Internet, and How QuePaxa Bypasses Fault Detection with Optional Leaders

Cloudflare’s Meerkat global consensus service selected the QuePaxa algorithm, published in 2023, over Raft or Paxos. The three authors explain that leader-based consensus suffers from systematic stalls in wide-area networks due to false fault detection and destructive interference during elections; QuePaxa achieves continuous request processing even when leaders fail through three design choices—optional leaders, constructive interference, and hedging timeouts unconstrained by network latency—reducing decision rates to a minimum of 65% rather than zero.

2026-11-11 22:24 GMT+8

The default choice for distributed consensus systems is Raft or Paxos; they are mature, understandable, and work well in local area networks. However, Cloudflare’s Meerkat service needs to operate across continents, where connection links often exhibit high and volatile latency. The three authors, João Pedro Leite, Bob Halley, and James Larisch, explain that this environment led them to abandon all mainstream algorithms in favor of QuePaxa, published in 2023 by Tennage, Băsescu, et al.

The root of the problem lies in two structural flaws of leader-based consensus. The first is destructive interference: when a leader fails or slows down, multiple replicas simultaneously initiate elections, blocking each other from obtaining a majority vote, causing elections to repeatedly fail. Even if a network fault lasts only two seconds, an election storm can persist far longer than the fault itself. The authors term this metastable failure—a failure pattern that sustains itself even after the initial cause has disappeared.

The second flaw is fault detection itself. In asynchronous networks, a process cannot directly observe whether a remote process is dead; it can only observe whether expected messages arrive on time. Latency spikes and connection interruptions make healthy leaders appear failed. Raft typically deploys an election timeout at five to ten times the network round-trip time; on transoceanic links with round-trip times in the hundreds of milliseconds, detecting a failed coordinator may take nearly a second—during which time the system is completely stalled. Cloudflare experienced this firsthand in 2020: a partial failure of a core network switch caused an internal consensus system to degrade, slowing dashboard operations to one-eightieth of normal speed for six hours. No data was lost, but writes were entirely unavailable.

Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión

QuePaxa’s response consists of three design choices. First, the leader becomes optional. Any proposer can submit proposals; the leader is simply the one with the highest priority and the most likely to complete a decision in a single round-trip. When the leader goes down, clients continue sending requests to other proposers, and the system does not stop. Second, constructive interference replaces destructive interference. Proposers do not compete against each other’s proposals but instead collaborate to converge on the known highest-priority proposal—when a proposer receives a higher-priority proposal, it adopts it rather than fighting it. Third, hedging timeouts are not constrained by network latency. Clients send requests to all replicas, and non-leader replicas wait for a short local timeout before proposing their own. This timeout is not for fault detection and does not need to exceed network round-trip time, allowing it to be set very small without freezing the system.

Execution is divided into four phases. In Phase Zero, proposers randomly select priorities and submit proposals, adopting the highest-priority proposal received from a majority of recorders; if the leader confirms its proposal reached a majority of recorders first, it can decide immediately. Phase One spreads the adopted proposal to a majority of recorders. Phase Two allows proposers to query a majority of recorders for the highest-priority proposal seen in Phase One; if it matches their own, they decide. Phase Three prepares proposers who failed to decide for the next round by adopting the best proposal from a majority of recorders. The majority intersection property guarantees safety: any two majorities share at least one recorder, ensuring that decided proposals cannot be overturned by subsequent rounds.

Cloudflare reported internal evaluation results: in an environment where servers span nearly all continents, Meerkat continuously processes requests and never fully stalls. Under normal conditions, more than 95% of decisions are completed in Phase Zero by the leader in a single round-trip; when the leader fails, the decision rate drops to approximately 65%, recovering once a replacement leader is installed after one minute. When decisions are not completed in Phase Zero, throughput briefly drops to one-third of normal because three round-trips are required instead of one. The authors emphasize this is a trade-off, not a free lunch: more active proposers mean more broadcasting and a lower probability of deciding in a single round, but they believe trading bandwidth for uninterrupted operation is worthwhile.

Boundaries to retain: The QuePaxa paper predicts a 50% decision probability per round, though actual performance is higher due to convergence effects, with no guaranteed upper bound on the number of rounds. Cloudflare’s performance data is self-reported internally, and the algorithm itself has not undergone independent real-world testing on the same scale as Raft or Paxos. The authors also note they are extending the original crash-stop design to support crash-recovery, which increases engineering complexity and requirements for correctness verification.

Practical implications for readers: If your consensus system runs in a local area network with predictable latency, Raft remains the correct choice. But if your replicas span continents, network latency is unpredictable, and the cost of false fault detection is system-wide stall, QuePaxa’s optional leaders and constructive interference mechanisms offer an alternative path with specific design rationale and preliminary production evidence. The core insight is not that QuePaxa is faster than Raft, but that it does not stop completely when the network deteriorates—it is fast when the network is normal, and it continues to advance when the network is not.

Fuentes

  • blog.cloudflare.comCloudflare Blog↗
Tema

QuePaxa共识算法

【背景】在广域网环境下,传统基于领导者的共识算法(如Raft)常因网络延迟波动导致故障检测误判和选举风暴,从而引发系统性停摆。2026年11月,Cloudflare工程师公开解释其内部实验性全球控制平面服务Meerkat为何放弃Raft,转…

3 pasos1 piezasActualizado 2026-11-16 20:28

Cómo llegó hasta aquí

  1. 2023-10-23Tennage & Băsescu等人在SOSP 2023发表QuePaxa论文,提出首个不依赖超时的状态机复制协议,旨在解决领导者型共识在恶劣网络条件下的可用性问题。bford.info ↗
  2. 2026-07-08Cloudflare发布博客介绍Meerkat,指出其为管理全球330+数据中心控制平面状态而设计,并明确说明该系统由QuePaxa驱动,以应对互联网不可预测的延迟和连接中断。blog.cloudflare.com ↗
  3. 2026-11-11Cloudflare工程师João Pedro Leite等人发布详细技术分析,解释为何在广域网中Raft会因破坏性干扰和故障检测误判而停摆,并展示QuePaxa如何通过随机优先级和四阶段协议保持活性。blog.cloudflare.com ↗

Lo que aún está pendiente

  • QuePaxa目前仍被描述为“实验性”且“未经充分实战检验”,其在更大规模生产环境中的长期稳定性如何?
  • Cloudflare提到正在对QuePaxa实现进行形式化验证,具体的验证结果和潜在的安全边界何时公布?
  • 除了Cloudflare,是否有其他大型科技公司跟进采用QuePaxa或类似的无超时共识算法?
Leer el tema completo →
Continuar investigando →