PUBLIC INDEX zcourts.com ZCOURTS - VOL. 02
LONDON - 2026
Keldra: Git at any scale, on your own infrastructure
Git does not need a distributed filesystem or a distributed reimplementation of Git. Keldra stores immutable repository history, publishes it through one exact compare-and-swap, and treats native Git repositories as disposable execution caches.
Static article
Imported body
Legacy aliases
Article archive
FIG. 02
notes
surface
research
surface
Article
app model
publish
surface
Imported writing rendered as native Fission Markdown content.
SECTION
Source and context.
The static release keeps the original post body locally while the backend content pipeline is still being built.
keldra
git
distributed-systems
object-storage
Published 2026-08-24 on zcourts.com. Estimated reading time: 16 min.
Original routes are preserved as local aliases so older links keep resolving to this static archive.
There is an odd kind of satisfaction in discovering that, independently, you have been walking towards the same answer as somebody else.
For some time Kestral had been part way towards a very different way of storing Git repositories. The pieces were there, but I had not quite assembled them into the simplest possible design. Then Cursor published <a href="https://cursor.com/blog/git-at-any-scale" rel="nofollow">Git at any scale</a>. It filled in some of the missing reasoning and, more importantly, confirmed something I had been circling for a while: the durable copy of a Git repository does not have to be a Git repository sitting on a particular server.
That sounds like a small observation. It changes almost everything.
The result is now part of Keldra, my distributed object store for application state. Keldra can host ordinary Git repositories, speak Git's standard smart HTTP protocol, and scale repository reads across a cluster without making a local filesystem the source of truth. It does not depend on S3, a cloud provider, or a separate routing database. You can run it on your own hardware, add nodes as you need them and continue using the normal git command line.
This post is about how that works, but first it is worth understanding why Git hosting becomes difficult in the first place.
A Git repository is already a very good data structure
Git is a content-addressed store. Blobs, trees and commits are identified by a hash of their contents and linked together into a graph. On the surface this looks perfect for a distributed key-value store:
object id -> object bytes
The temptation is to split every Git object across a cluster and fetch each one when Git asks for it. That is also where things go wrong.
Git does not know the next object it needs until it has read the current one. A commit points to a tree and its parents. A tree points to more trees and blobs. Packfiles make this more efficient on disk by compressing objects together and representing many objects as deltas from other objects, but that means a single logical walk may also require several physical reads.
Turn every one of those reads into a network round trip and a fast graph traversal becomes death by a thousand tiny requests.
The opposite idea, putting the repository on a network filesystem, is not much better. Git quite reasonably expects local filesystem behavior. Its locking, renaming, syncing and random access patterns were designed for the disk in a developer's machine, not a filesystem pretending that a network is a disk.
The answer is not to throw away Git's storage format. Packfiles, multi-pack indices, commit graphs, bitmaps and years of work on repacking are extremely good at what they do. The answer is to stop treating one particular copy of those files as the durable authority.
Authority and execution are different things
This distinction turns out to be the centre of Keldra's Git architecture.
The authoritative repository is stored as:
•
immutable Git packfiles
•
an ordered sequence of accepted reference changes
•
periodic immutable checkpoints
•
one small mutable current pointer
A native bare Git repository on a Keldra node is an execution copy. It is materialised from those objects, used by normal Git to serve a clone, fetch or push, and can be thrown away whenever the cache needs the space.
That means a repository can be executed anywhere without its authority moving anywhere.
If a node disappears, no repository has disappeared with it. Another node reads current, downloads the latest checkpoint and its short tail of changes, then reconstructs an ordinary bare repository. If every Git cache in the cluster is deleted, acknowledged repositories still exist. The cache is useful, sometimes enormously useful, but it is not precious.
I like this separation because it gives each part of the system one clear job:
•
Git remains responsible for Git semantics.
•
Keldra remains responsible for durable bytes, placement, publication and authorization.
•
The local filesystem remains responsible for making native Git operations fast.
There is no distributed reimplementation of Git hiding in the middle.
A push is data first, visibility second
A Git push naturally has two parts.
The first is a pack containing the new blobs, trees and commits. The second is a reference transaction saying, for example, move refs/heads/main from commit A to commit B.
The pack can be written before it is visible. Nothing can reach those objects until a reference points to them. Keldra uses that fact as the publication boundary.
When a push arrives, the selected node:
1.
brings a local bare repository to the exact current generation;
2.
lets native Git receive the pack into quarantine;
3.
asks Git to validate the pack, object connectivity and reference transaction;
4.
streams accepted pack bytes into Keldra as immutable objects;
5.
writes an immutable record describing the accepted reference change; and
6.
advances the repository's small current object with PutIfVersion.
That final compare-and-swap is the instant at which the push becomes visible.
If two nodes race, both may do some preparation work, but only one can replace the exact version of current they read. The loser reads the new state and re-evaluates its reference transaction. It cannot overwrite a push it did not see because PutIfVersion makes that state impossible.
This is substantially smaller than running a consensus protocol over every packfile or keeping several mutable repository directories in lockstep. The large bytes are immutable and can be distributed independently. Agreement is only required for the tiny decision that says which complete generation is current.
That pattern has followed me through years of work on distributed systems: first identify the smallest thing which represents a decision, then make that correct. Do not force every byte involved in the decision through the same coordination machinery.
Keldra already had the pieces
Keldra was not originally built as a Git hosting system. It was built around opaque bytes stored at stable paths, with a small set of primitives applications can compose instead of pretending every workload is a relational database.
Those primitives are exactly what made this design practical.
Immutable objects
Packfiles, push batches and checkpoints are immutable Keldra objects. They go through the same storage path as every other object. Small records are kept inline; larger objects are streamed into the distributed byte plane and erasure coded according to the cluster's configured profile.
Content addressing also means identical stored bytes share their underlying payload. Git already does an excellent job of avoiding unnecessary object duplication inside packs; Keldra's own content identity applies independently to the bytes it stores across the cluster.
Exact compare-and-swap
PutIfVersion is the entire publication mechanism. The caller says, in effect, "replace version 41 only if version 41 is still current". Success publishes the new generation. Failure means somebody else won and the operation must be evaluated against their result.
No repository assignment, reference list or pack payload goes into Raft. Raft remains reserved for small cluster decisions. Repository truth lives in ordinary Keldra objects and the exact-path CAS which publishes them.
Streaming writes
Git packs can be large, so the gateway does not collect a whole request in memory before doing anything useful. Request and response bodies stream end to end. Backpressure travels from storage to the Git client instead of becoming an arbitrary file-size limit or an ever-growing memory allocation.
Weighted rendezvous hashing
Any active node can receive a Git request. Keldra uses weighted rendezvous hashing to choose the preferred node for a repository from the current membership.
The useful property here is that placement is calculated rather than catalogued. Given a repository ID and the active nodes, every node arrives at the same ranking. Add or remove a node and only the minimum necessary assignments move. There is no enormous repository-to-server table to maintain, replicate or repair.
The weighting matters too. A node with more disk, memory and CPU can carry a proportionally larger share of the work than a smaller machine. A cluster can be heterogeneous without pretending every server has the same capacity.
Most importantly, this ranking is an optimisation. If membership changes at an awkward moment and two nodes briefly believe they should coordinate a repository, the current CAS still decides the winner. Fast when healthy, correct when the topology is moving.
Zanzibar authorization
The Git gateway is not a separate security island. Clone, fetch, push and reference advertisement are authenticated and authorized using the same Zanzibar relationships as Keldra's object APIs.
Repositories are private by default. A bucket owner can explicitly grant read access to the anonymous principal when a public clone is wanted. An invalid credential never quietly becomes an anonymous request.
Ordinary durability and accounting
Packs use Keldra's normal durability path. On a single-node installation they are durably accepted by that node. As the cluster grows, the same objects follow the configured replication and erasure-coding rules without Git needing its own storage topology.
Git traffic also flows through ordinary tenant and bucket accounting. Client ingress and egress are counted once; internal replication, cache population and compaction remain operational traffic rather than being charged as if the client sent them again.
This is why implementing Git as a gateway over Keldra is more interesting than building a standalone Git server. The difficult distributed pieces were already primitives, not Git-specific inventions.
Reads remain local and native
The durable representation is distributed, but a Git operation is deliberately not.
For a clone or fetch, Keldra first reads the authoritative current pointer. It then acquires a local materialisation at exactly that generation. If the cached repository already matches, the rest of the operation is ordinary local Git work. If it is behind, Keldra downloads and applies only the missing push batches. If it is too far behind, it starts from the latest checkpoint and applies the bounded tail.
Only when the local repository's packs, references and generation marker agree is it handed to the Git process.
This preserves the thing local Git is exceptionally good at: walking its graph and packfiles with filesystem latency. The distributed store is used to move coarse immutable objects, not inserted into every object lookup.
It also gives Keldra an uncomplicated way to scale reads. A busy repository can be materialised on more nodes. Clone and fetch traffic is spread across those nodes, each doing native local work. A quiet repository may have no warm copy at all and cost almost nothing until somebody accesses it again.
The number of execution copies is therefore independent of the number of durable payload fragments. We can add read capacity without multiplying the authoritative repository and we can evict idle repositories without reducing durability.
Compaction happens once
An immutable sequence cannot grow forever. Every push may introduce another pack and every pack has an index which Git may have to inspect. Eventually an efficient lookup repeated across hundreds of packs is no longer efficient.
Git already has the answer: multi-pack indices, commit graphs, bitmaps and repacking. Keldra uses them rather than inventing a new pack format.
The preferred node compacts one exact repository generation using native Git, writes the resulting packs and checkpoint as new immutable Keldra objects, then attempts to publish that checkpoint with the same current CAS.
If a push arrived in the meantime, compaction loses the CAS and nothing half-finished becomes visible. If it succeeds, every other serving node can download the compacted result. They do not all spend CPU independently rediscovering the same answer.
This trades network bandwidth for repeated compute, which is generally the right trade for immutable artifacts. Expensive work is performed once, its result is distributed, and the old generation ages into normal garbage collection after active readers have finished with it.
Why not just use S3?
You can build this architecture on S3. Cursor did, and it makes sense for their environment.
Keldra's distinction is that the object store is the product, not an external dependency. The Git gateway composes the same primitives whether Keldra is running in a cloud, across dedicated servers in a data centre, or on three machines in an office.
An organisation that needs source code to remain on premise does not have to recreate the architecture around an S3-compatible service, a reference database, a repository routing database and a separate authorization system. Keldra supplies the byte plane, mutable publication record, cluster membership, weighted placement, credentials, Zanzibar checks, accounting and Git protocol on one coherent foundation.
Start with one node and a complete local copy. Add a second and Keldra gains another durable copy. At the configured erasure width, large immutable objects are placed as data and parity fragments across distinct nodes. The Git representation does not change as the installation grows.
It is the same repository and the same API throughout.
The shape of the result
The complete architecture can be reduced to this:
git push
|
v
native Git validation
|
+----> immutable pack objects --------+
| |
+----> immutable push batch |
v
PutIfVersion(current)
|
v
published generation
|
+---------------------+---------------------+
| | |
v v v
native Git cache native Git cache native Git cache
node A node B node C
The top half is authority. The bottom half is disposable execution.
Once those are separated, several hard operational problems simply stop being special cases:
•
A node failure loses a cache, not a repository.
•
Adding serving nodes does not add participants to every push.
•
Cache eviction does not alter durability.
•
A missed notification does not create stale reads because every request checks current.
•
Compaction produces an immutable generation instead of mutating every replica in place.
•
Two writers cannot silently overwrite one another because publication is an exact CAS.
There is still real work in receiving packs, validating references, materialising repositories and moving bytes. The point is not that distributed Git becomes free. The point is that each cost now belongs to the layer best able to pay it.
Git handles Git. Local disks handle graph traversal. Keldra handles durable distributed state and the one decision which makes a generation visible.
Available now
Keldra is open source and its Git smart HTTP gateway works with the standard Git command line. It shares the same public listener as Keldra's gRPC and S3 APIs, so an installation does not need another public server merely to host repositories.
The project, container image and Rust client are available from the Keldra repository. There is also a broader overview of Keldra's storage, indexing and application-state capabilities.
I started this journey thinking about how to distribute repository storage. The useful answer was to stop distributing the execution of Git and distribute its durable history instead.
That final distinction is what turned the pieces Kestral had already uncovered into Keldra's Git architecture: normal Git repositories where they are fastest, immutable history where it is safest, and one small pointer deciding what the world can see.
SECTION
Related writing.
More imported posts from the current and legacy zcourts archives.
2026-04-09
Should you write your own database?
We run a data pipeline that ingests very large product snapshots, turns them into downstream feed artifacts, and then keeps those feeds fresh with smaller incremental updates. The pipeline has a weekly "start from a new baseline" phase a...
2026-02-13
AI, Energy, and the Infrastructure Cycle: What Rising Power Bills Tell Us About the Next Economic Phase
For many households in the UK, the story begins not with artificial intelligence, but with a monthly or quarterly energy bill.
2026-01-28
Safety Without Friction: Why “Responsible AI” Fails Under Commercial Incentives
Over the past several years, a new genre of writing has emerged alongside the rapid development of large-scale AI systems: the AI safety essay. These essays are typically long-form, reflective, and morally serious. They acknowledge uncer...