durable-git README

The project's README from github.com/littledivy/durable-git. As one markdown file: README.md.

dgit

Durable git.

dgit is a git server for Cloudflare Workers and for your own machines with celld. Each repository is a Durable Object: a small server with a name and a private SQLite database that holds the repository's objects and refs, speaks the git smart HTTP protocol to a stock git client, and renders a cgit-style web interface. There is no origin server, no filesystem, and no GitHub in the critical path. A repository nobody touches costs almost nothing, and applications shard by construction: one hot repository cannot slow another. Reads are public; pushes authenticate; pushing to a name that does not exist creates the repository.

How it works

dgit implements git in TypeScript: pkt-line framing, packfile parsing with ofs- and ref-delta resolution, pack generation over a streaming SHA-1, commit/tree/tag codecs, and a Myers diff. The one dependency is pako, for zlib.

A push streams into the repository's cell and is stored as the packfile the client sent; an index maps each object id to its pack, offset, and delta base, so the client's compression is preserved rather than re-derived. When an R2 bucket is bound the pack bytes are written to R2 and only the index stays in the cell's SQLite, so pack storage is no longer capped by the per-cell database. A clone walks the closure of the requested refs and copies the stored compressed bytes verbatim into the outgoing pack; a full clone, once built, is cached in R2 and afterwards streamed straight from the Worker, so repeat clones never load the cell. Fetch negotiation excludes the closure of the client's haves, cut correctly at shallow boundaries, so an incremental fetch downloads only what is missing. Shallow clones (--depth, deepening, --unshallow), thin packs, side-band progress, forced updates, and ref deletion behave as they do against any git server.

The web interface is the cgit surface: summary, refs, log with search and per-path history, tree, blob with syntax highlighting, blame, commit and arbitrary-range diffs, format-patch output that applies cleanly with git am, tar.gz and zip snapshots of any ref, about pages rendered from the README, atom feeds, and commit-activity statistics. Repositories carry a description, an owner, a section on the index page, and a private flag that hides them and gates every read behind the push token.

Deploy to Cloudflare

npm install
npx wrangler r2 bucket create dgit-pack-cache   # optional: R2 pack offload + clone cache
npx wrangler deploy
npx wrangler secret put GIT_TOKEN   # the push password

The PACK_CACHE R2 binding in wrangler.jsonc is optional: with it, pushed pack bytes live in R2 (off the cell's SQLite) and full clones are served straight from R2 by the Worker without loading the cell. Remove the binding and everything falls back to the SQLite-only path.

Then push anything:

git remote add origin https://<your-host>/myrepo.git
git push -u origin main

A Workers request is bounded at 128MB of memory and five minutes of CPU, so a very large history lands as a series of smaller pushes rather than one; day-to-day pushes, clones, and fetches fit comfortably. Building a full-history clone of a repository with millions of objects can exceed the CPU bound the first time — once such a clone is cached in R2 it streams from the Worker without rebuilding, and shallow and incremental fetches of the same repository are fine regardless. The largest repositories belong on celld.

Self-host on celld

celld runs the same Worker against a bucket you own, with none of the managed platform's request bounds. Set a real GIT_TOKEN var in wrangler.celld.jsonc first:

celld deploy wrangler.celld.jsonc --bucket s3://my-cells --endpoint https://...
CELLD_V8_HEAP_LIMIT_MB=4096 CELLD_LTX_DURABILITY_TIMEOUT_SECS=180 \
celld --bucket s3://my-cells --endpoint https://... \
  --listen 0.0.0.0:8080 --internal-listen 10.0.0.1:8081 --advertise 10.0.0.1:8081

Each repository's SQLite database replicates to the bucket; nodes are disposable, and a killed node's repositories come back bit-identical. The heap and durability-deadline variables give large single-cell ingests the room the defaults do not.

Build on dgit

dgit is also a library, published as durable-git. The deployed Worker above is three lines around it:

import { createDurableGit, secretsEqual } from "durable-git";
export { RepoCell, Registry } from "durable-git";

export default createDurableGit({
  async authorize({ repo, op, private: priv, credentials, env }) {
    if (op === "read" && !priv) return true;
    return secretsEqual(credentials?.pass ?? "", await lookupDeployKey(env, repo));
  },
  onPush(event, env) {
    return fetch("https://ci.example.com/hook", { method: "POST", body: JSON.stringify(event) });
  },
});

The package ships TypeScript source, which wrangler bundles directly; bind the Durable Object classes as wrangler.jsonc does and re-export them from your entry module.

authorize gates every repository-scoped request and replaces the stock GIT_TOKEN policy entirely (the private flag is yours to consult); onPush fires with the ref updates a push applied. Every repository also serves a JSON content API, gated like its pages, and the same surface as typed RPC on the cell. ui: false runs it headless behind your own frontend, cors opens the API cross-origin, and namespaces: true routes GitHub-style owner/name repositories. See docs at docs/api.md.

Operate

curl -X PUT  -u x:$GIT_TOKEN -d '{"description":"...","section":"tools","private":false}' \
  https://<host>/myrepo/config                       # describe and place a repository
curl -X POST -u x:$GIT_TOKEN https://<host>/myrepo/gc    # prune unreachable objects
curl -X DELETE -u x:$GIT_TOKEN https://<host>/myrepo     # delete a repository

Garbage collection also runs by itself, from a Durable Object alarm, after a forced update or a ref deletion. GIT_TOKENS holds additional comma-separated tokens; MAX_PUSH_MB caps a single push. Setting SHA1DC=1 screens every pushed object for a SHA-1 collision attack on ingest; by default objects are hashed with native SHA-1 — the same object ids, without the check.

Contributions

Pull requests are disabled. Send a git format-patch attachment to [email protected].

durable-git (littledivy/durable-git)

Searching Database…

265 ·GitHub
Durable-git
这是什么?dgit 是一个面向 Cloudflare Workers 以及你自己的装有 celld 的机器的 git 服务器。每个仓库都是一个 Durable Object,带有自己的 SQLite 数据库:它能与原生 git 客户端通过 git smart HTTP 协议通信,并渲染一个 cgit 风格的网页界面。
那每个仓库都是一个小型独立服务器——实际用起来是什么感觉?每个仓库都是一个 Durable Object:一个有名字的小服务器,带有私有的 SQLite 数据库,存放它的对象和引用。仓库天然就是分片的——一个繁忙的仓库不会拖慢另一个,没人访问的仓库几乎不占用任何资源。
我怎么把它跑起来?配置 wrangler.jsonc——可选地把一个 R2 存储桶绑定为 PACKCACHE——然后运行 'wrangler deploy'。之后,推送到任何还不存在的名称,就会在服务器上创建对应的仓库。
运行它需要什么重量级的东西吗——比如真正的数据库服务器、一大堆依赖?不需要。git 本身用 TypeScript 实现——包括 pkt-line 帧、packfile 解析、commit/tree/tag 编解码,以及一个 Myers diff——唯一的依赖是用于 zlib 的 pako。SQLite 内置于 Durable Objects 中。
在 Cloudflare 上,特别大的仓库有什么限制?一个 Workers 请求的内存上限是 128MB,CPU 时间上限是五分钟,所以特别大的历史会以多次较小的推送形式落地,而不是一次完成。对拥有数百万个对象的仓库做首次完整克隆时,可能超出 CPU 限制;一旦缓存到 R2,之后就能直接流式传输而无需重建,而最大的那些仓库更适合放在 celld 上。
我可以不用 Cloudflare,在自己机器上自行托管吗?可以——celld 会让同一个 Worker 运行在你自己的存储桶上,没有托管平台的那些请求限制。每个仓库的 SQLite 数据库会复制到存储桶,因此节点是随时可丢弃的,被杀掉的节点上的仓库会以完全一致的比特恢复;不过要先把 GITTOKEN 设置成一个真正的值。
仓库可以设为私有吗?可以。仓库带有一个 private 标志,会把它们隐藏起来,并且让所有读取都要通过推送令牌验证;authorize 钩子还允许你完全替换默认的 GITTOKEN 策略。
仓库数据实际上是怎么存储的?推送会流入仓库的 cell,并以客户端发来的 packfile 原样存储,同时建立一个索引,把每个对象 id 映射到它所在的 pack、偏移量和 delta 基准——因此客户端的压缩得以保留,而不必重新推导。绑定了 R2 存储桶时,pack 的字节会存到 R2,只有索引留在 cell 的 SQLite 里。
它能防范 SHA-1 碰撞攻击吗?默认情况下对象使用原生 SHA-1 计算哈希——对象 id 与原生 git 相同,但没有碰撞检查。设置 SHA1DC=1 后,会在每个对象写入时检测是否属于 SHA-1 碰撞攻击。
我可以在它之上构建自己的应用吗?可以——它以 durable-git 这个 npm 包发布,附带 TypeScript 源码,wrangler 会直接打包。你可以获得用于鉴权的 authorize 钩子、用于引用更新的 onPush、在自己前端后面以 ui:false 运行无界面模式、CORS、GitHub 风格的 owner/name 命名空间,以及每个 cell 上的 JSON 内容 API 和带类型的 RPC。
What are similar projects?open-compute, walgit, hono, vinext, git-ai, tardigrade, computer, agentic-inbox
Which topics is it in?edge-computing, hosting, object-storage, cloudflare-workers, version-control, cloudflare, http, git, sqlite, cloud, npm, api
Creator
littledivy
Size
0.3 MB
Stars
265
Updated
CommitAug 21
Age
2mo
Commits · 90 Days