How GitHub Is Rebuilding Git Storage for Agent-Scale Traffic

Agents and developers pushing to their own branches don’t actually conflict, but GitHub’s Git storage has been making them wait in one line anyway. GitHub’s latest engineering post explains how it is fixing that while moving its Git service to Azure under exponential traffic growth. For platform teams running their own Git servers, CI fleets or artifact stores, it is a detailed case study in what agent-driven write load does to infrastructure built for humans.

What Happened

In GitHub’s engineering post, the company says 99.99999% of its Git repositories now live both in its own data centers and in Azure. It points to what it calls the agentic shift: fast traffic growth, a hard ceiling on data center power in its region, and physical supply chain limits. The first repository copy landed in Azure’s Central US region in August 2025. GitHub laid out the two principles behind the new design, minimizing coordination and decoupling storage from compute, in an earlier post. Today most repositories keep two of their five replicas in Azure, and Azure serves about half of Git traffic.

The remaining repositories are among the busiest, held back by one number: a 25ms round trip between GitHub’s data centers and Azure, about 1,000 miles away.

Why It Matters

The bottleneck was the write quorum in Spokes, GitHub’s replication system. A write needs only a majority to commit, but Spokes waits for every healthy replica (up to a timeout) and holds a repository-wide lock until the write commits. The slowest replica therefore set both write latency and the repository’s throughput ceiling, and a far-away replica made that ceiling much lower.

GitHub’s fix is to apply the quorum per reference instead of per repository. Git’s consistency rules apply to each reference on its own, and most writes touch different references, so they no longer wait on each other. In internal testing with realistic traffic, writes to different references saw 3 to 4 times higher throughput at lower latency.

Reads come next. Repositories see between 100 and 1,000 reads per write, yet reads and writes share replicas today. GitHub is adding a “virtual write replica,” an ephemeral node that streams writes into Azure Blob Storage, as a quorum member. Read replicas can then warm from object storage and scale on demand without joining the quorum. Against them, GitHub’s load-testing harness hit its own limits first.

What This Means for Platform Teams

The lesson carries beyond GitHub: lock scope that works for human traffic can become the bottleneck when agents create many parallel branches. GitHub says the commit protocol that took it to a billion repositories won’t take it to the next billion, and promises a final post covering the end state at GitHub Universe.

Stack Archive’s look at how Datadog scaled Git serving 20x for CI covers a similar read-heavy problem from inside one company.

Source: The GitHub Blog

Content created with AI assistance and reviewed for accuracy.