Cloudflare Chooses an Unusual Consensus Algorithm for Meerkat

Cloudflare says its Meerkat control-plane service uses QuePaxa, an unusual consensus algorithm designed to keep decisions moving across a global network. The company described the choice in a technical post about the system it built for its internal global control plane.

QuePaxa was published in 2023 by researchers Tennage & Băsescu et al. Cloudflare said it selected the protocol because it can reach consensus in a single round trip under favorable conditions, while continuing through latency spikes, connectivity problems and server failures.

Why Cloudflare Chose an Unusual Consensus Algorithm

Many consensus systems depend on a leader that coordinates decisions. In protocols such as Raft, replicas use timeouts to decide when that leader may have failed. Cloudflare explained that a delayed message on an asynchronous network can look like a failed server, even when the server remains active.

A mistaken failure suspicion can trigger an election and pause work.

Cloudflare said Raft deployments commonly use an election timeout five to ten times the network round-trip time. A round trip halfway around the world can take hundreds of milliseconds, and a timeout spanning several round trips can take around a second to detect a failed coordinator.

Proposers Cooperate Instead of Competing

QuePaxa reduces reliance on failure detection by allowing any proposer to submit work. It assigns priorities to proposals using random numbers, while an optional leader receives the highest possible priority. Recorders share proposals with active proposers, which adopt higher-priority proposals they learn about.

This design encourages proposers to converge on the highest-priority proposal they collectively know. Random priorities make agreement likely within a round, though not guaranteed. Enough rounds guarantee agreement, the article says. The leader can decide in one round trip if its proposal reaches a majority of recorders first.

Clients can also send requests to every replica and let non-leader replicas wait briefly, giving the leader time to use its faster path. Cloudflare said its implementation differs from the paper: clients wait through that hedge timeout, reducing messages between replicas. The timeout does not detect failure, so lowering it can make the system react faster to leader changes, at the cost of more proposal traffic.

The protocol runs in four phases. Proposers first submit and adopt proposals, then spread the adopted choice. They next check recorder reports and decide when the highest-priority proposal matches their own. A final carry-forward phase prepares undecided proposers for another round.

Meerkat Keeps Processing During Leader Problems

Cloudflare said its Meerkat evaluations used servers spanning nearly all continents. The service continued processing requests when its leader crashed, disconnected or became slow, and the company said it had not seen the system completely stall.

More than 95% of decisions were made by the leader in the first phase, according to the post. When a decision missed that phase, throughput briefly fell to at most a third of its usual rate. During a leader failure, the decision rate dropped to around 65% of its original rate and returned to 100% when the faulty leader was replaced, a minute after failing.

The design also lets Cloudflare change which replica acts as leader while a cluster runs. The company said cooperative proposers make a temporarily non-optimal leader less limiting than in systems where one leader drives decisions.

Cloudflare noted that QuePaxa has not been battle-tested as extensively as Raft or Paxos, and said it is extending the original crash-stop design to support crash recovery. The company is formally verifying QuePaxa and Meerkat, alongside simulation and fault-injection testing. It plans future posts on verification, failure testing and benchmarks.

For related context on Cloudflare’s infrastructure work, see its AI agent decision models, content delivery network policy coverage and blog migration. The source article, Choosing QuePaxa, gives Cloudflare’s account of the protocol and its Meerkat evaluation.