durable-git README

The project's README from github.com/littledivy/durable-git. As one markdown file: README.md.

dgit

Durable git.

dgit is a git server for Cloudflare Workers and for your own machines with celld. Each repository is a Durable Object: a small server with a name and a private SQLite database that holds the repository's objects and refs, speaks the git smart HTTP protocol to a stock git client, and renders a cgit-style web interface. There is no origin server, no filesystem, and no GitHub in the critical path. A repository nobody touches costs almost nothing, and applications shard by construction: one hot repository cannot slow another. Reads are public; pushes authenticate; pushing to a name that does not exist creates the repository.

How it works

dgit implements git in TypeScript: pkt-line framing, packfile parsing with ofs- and ref-delta resolution, pack generation over a streaming SHA-1, commit/tree/tag codecs, and a Myers diff. The one dependency is pako, for zlib.

A push streams into the repository's cell and is stored as the packfile the client sent; an index maps each object id to its pack, offset, and delta base, so the client's compression is preserved rather than re-derived. When an R2 bucket is bound the pack bytes are written to R2 and only the index stays in the cell's SQLite, so pack storage is no longer capped by the per-cell database. A clone walks the closure of the requested refs and copies the stored compressed bytes verbatim into the outgoing pack; a full clone, once built, is cached in R2 and afterwards streamed straight from the Worker, so repeat clones never load the cell. Fetch negotiation excludes the closure of the client's haves, cut correctly at shallow boundaries, so an incremental fetch downloads only what is missing. Shallow clones (--depth, deepening, --unshallow), thin packs, side-band progress, forced updates, and ref deletion behave as they do against any git server.

The web interface is the cgit surface: summary, refs, log with search and per-path history, tree, blob with syntax highlighting, blame, commit and arbitrary-range diffs, format-patch output that applies cleanly with git am, tar.gz and zip snapshots of any ref, about pages rendered from the README, atom feeds, and commit-activity statistics. Repositories carry a description, an owner, a section on the index page, and a private flag that hides them and gates every read behind the push token.

Deploy to Cloudflare

npm install
npx wrangler r2 bucket create dgit-pack-cache   # optional: R2 pack offload + clone cache
npx wrangler deploy
npx wrangler secret put GIT_TOKEN   # the push password

The PACK_CACHE R2 binding in wrangler.jsonc is optional: with it, pushed pack bytes live in R2 (off the cell's SQLite) and full clones are served straight from R2 by the Worker without loading the cell. Remove the binding and everything falls back to the SQLite-only path.

Then push anything:

git remote add origin https://<your-host>/myrepo.git
git push -u origin main

A Workers request is bounded at 128MB of memory and five minutes of CPU, so a very large history lands as a series of smaller pushes rather than one; day-to-day pushes, clones, and fetches fit comfortably. Building a full-history clone of a repository with millions of objects can exceed the CPU bound the first time — once such a clone is cached in R2 it streams from the Worker without rebuilding, and shallow and incremental fetches of the same repository are fine regardless. The largest repositories belong on celld.

Self-host on celld

celld runs the same Worker against a bucket you own, with none of the managed platform's request bounds. Set a real GIT_TOKEN var in wrangler.celld.jsonc first:

celld deploy wrangler.celld.jsonc --bucket s3://my-cells --endpoint https://...
CELLD_V8_HEAP_LIMIT_MB=4096 CELLD_LTX_DURABILITY_TIMEOUT_SECS=180 \
celld --bucket s3://my-cells --endpoint https://... \
  --listen 0.0.0.0:8080 --internal-listen 10.0.0.1:8081 --advertise 10.0.0.1:8081

Each repository's SQLite database replicates to the bucket; nodes are disposable, and a killed node's repositories come back bit-identical. The heap and durability-deadline variables give large single-cell ingests the room the defaults do not.

Build on dgit

dgit is also a library, published as durable-git. The deployed Worker above is three lines around it:

import { createDurableGit, secretsEqual } from "durable-git";
export { RepoCell, Registry } from "durable-git";

export default createDurableGit({
  async authorize({ repo, op, private: priv, credentials, env }) {
    if (op === "read" && !priv) return true;
    return secretsEqual(credentials?.pass ?? "", await lookupDeployKey(env, repo));
  },
  onPush(event, env) {
    return fetch("https://ci.example.com/hook", { method: "POST", body: JSON.stringify(event) });
  },
});

The package ships TypeScript source, which wrangler bundles directly; bind the Durable Object classes as wrangler.jsonc does and re-export them from your entry module.

authorize gates every repository-scoped request and replaces the stock GIT_TOKEN policy entirely (the private flag is yours to consult); onPush fires with the ref updates a push applied. Every repository also serves a JSON content API, gated like its pages, and the same surface as typed RPC on the cell. ui: false runs it headless behind your own frontend, cors opens the API cross-origin, and namespaces: true routes GitHub-style owner/name repositories. See docs at docs/api.md.

Operate

curl -X PUT  -u x:$GIT_TOKEN -d '{"description":"...","section":"tools","private":false}' \
  https://<host>/myrepo/config                       # describe and place a repository
curl -X POST -u x:$GIT_TOKEN https://<host>/myrepo/gc    # prune unreachable objects
curl -X DELETE -u x:$GIT_TOKEN https://<host>/myrepo     # delete a repository

Garbage collection also runs by itself, from a Durable Object alarm, after a forced update or a ref deletion. GIT_TOKENS holds additional comma-separated tokens; MAX_PUSH_MB caps a single push. Setting SHA1DC=1 screens every pushed object for a SHA-1 collision attack on ingest; by default objects are hashed with native SHA-1 — the same object ids, without the check.

Contributions

Pull requests are disabled. Send a git format-patch attachment to [email protected].

durable-git (littledivy/durable-git)

Searching Database…

263 ·GitHub
Durable-git
What is this?dgit is a git server for Cloudflare Workers and for your own machines with celld. Each repository is a Durable Object with its own SQLite database: it speaks the git smart HTTP protocol to a stock git client and renders a cgit-style web interface.
So every repository is its own little server — what does that mean in practice?Each repository is a Durable Object: a small server with a name and a private SQLite database holding its objects and refs. Repos shard by construction — one hot repository cannot slow another, and a repository nobody touches costs almost nothing.
How do I get it running?Configure wrangler.jsonc — optionally binding an R2 bucket as PACKCACHE — and run 'wrangler deploy'. After that, pushing to any name that does not exist yet creates that repository on the server.
Does it need anything heavy to run — a real database server, lots of dependencies?No. Git itself is implemented in TypeScript — pkt-line framing, packfile parsing, commit/tree/tag codecs, a Myers diff — and the one dependency is pako, for zlib. SQLite is built into Durable Objects.
Are there limits for very large repositories on Cloudflare?A Workers request is bounded at 128MB of memory and five minutes of CPU, so a very large history lands as a series of smaller pushes rather than one. The first full clone of a repo with millions of objects can exceed the CPU bound; once cached in R2 it streams without rebuilding, and the largest repositories belong on celld.
Can I self-host it on my own machines instead of Cloudflare?Yes — celld runs the same Worker against a bucket you own, with none of the managed platform's request bounds. Each repository's SQLite database replicates to the bucket, so nodes are disposable and a killed node's repositories come back bit-identical; set a real GITTOKEN first.
Can repositories be private?Yes. Repositories carry a private flag that hides them and gates every read behind the push token, and the authorize hook lets you replace the stock GITTOKEN policy entirely.
How is repository data actually stored?A push streams into the repository's cell and is stored as the packfile the client sent, with an index mapping each object id to its pack, offset, and delta base — so the client's compression is preserved rather than re-derived. When an R2 bucket is bound, pack bytes go to R2 and only the index stays in the cell's SQLite.
Does it protect against SHA-1 collision attacks?By default objects are hashed with native SHA-1 — the same object ids, without the check. Setting SHA1DC=1 screens every pushed object for a SHA-1 collision attack on ingest.
Can I build my own app on top of it?Yes — it is published as the durable-git npm package, shipping TypeScript source that wrangler bundles directly. You get an authorize hook for auth, onPush for ref updates, ui:false for headless mode behind your own frontend, CORS, GitHub-style owner/name namespaces, and a JSON content API plus typed RPC on each cell.
What are similar projects?open-compute, walgit, hono, vinext, git-ai, tardigrade, computer, agentic-inbox
Which topics is it in?edge-computing, hosting, object-storage, cloudflare-workers, version-control, cloudflare, http, git, sqlite, cloud, npm, api
Creator
littledivy
Size
0.3 MB
Stars
263
Updated
CommitAug 21
Age
1mo
Commits · 90 Days