openfuse README

项目的 README,来自 github.com/CopilotKit/openfuse。合并为一个 markdown 文件:README.md。

OpenFuse

A personal agent with a browser, terminal, files, and work that keeps going. Compatible with any agent harness.

Ask for an outcome. Follow the plan, review actions, and come back to the result. Built with CopilotKit React Native for iOS, Android, and web.

Quick start · Demo · Features · Architecture · Docs · Contributing

CI MIT license

CopilotKit%2Fopenfuse | Trendshift

Deploy to Render

Clone this template and customize it however you want.

Building on OpenFuse? Meet with the CopilotKit team →

Alpha, for self-hosting and building on. Open-ended reasoning, live Google accounts, and CopilotKit Rich Threads require their own configuration. See what is verified and the roadmap.

Demo

On iPhone, ask OpenFuse to find interesting stories on Hacker News and summarize CopilotKit. On desktop, ask it to check the school-trip email, open the message, and research exhibits at Monterey Bay Aquarium. The agent shows email and browser results inline. Take control opens that same browser session when you need it.

The send arrow becomes a stop square inside the input pill while the agent replies, then switches back. Stopping keeps your draft intact. See the recording notes for the model setup and reproduction steps.

Recording details and reproduction

The Jev aquarium-trip demo walks through a fictional school email, clarification choices, sourced exhibit cards, hands-on preference refinement, and a confirmed selection.

What it is

OpenFuse is a personal-agent application with an agent computer, visible work, and rich results. It runs its own server, task worker, and browser worker. You can inspect and change the source under the MIT license.

The computer combines persistent Chromium and an optional Linux workspace. The agent can browse public pages, run commands in its own container, work with files, and move PDFs between the computer and the app. You can open its browser or terminal and continue the work. Graphical desktops and autonomous checkout remain future work.

Features

Surface What runs in this alpha
Chat CopilotKit headless chat with streamed AG-UI events, mailbox search and reading, send/stop in one input pill, a visible follow-up queue, retained drafts, delegated tasks, and inline email, browser, PDF, plan, and finance cards.
Agent computer Persistent browser profiles and takeover console; optional isolated Linux terminal, saved command receipts, editable workspace files, and PDF transfer.
Activity Durable task plans, progress, input requests, pause/resume/cancel/retry, approvals, and saved receipts. SQL leases recover interrupted work.
Ideas Suggestions with source evidence; edit, accept, or dismiss. Sent replies and completed matching work are excluded.
Goals & Tracking Goals and milestones; recurring public-page checks for changes, text availability, or USD price thresholds, with deduplicated alerts and failure backoff.
Documents Email attachment → PDF → requested form values → filled copy → reviewed reply → receipt. Native/web PDF viewing, paging, zoom, supported fields, and sharing.
Finance Import transaction CSV to create a spending summary with categories, transactions, and a savings-goal action.
Gmail & Calendar Google OAuth adapters, complete mail threads, drafts/attachments, calendar discovery, and reviewed event creation/update/deletion. Live credentials required.
Personal context Editable name, tone, avatar, and memories. Background-update preferences and durable in-app notifications.
Rich Threads CopilotKit Intelligence persistence in every mode, with a stable main conversation, side chats, renaming, archiving, restoring, and replay. A server-only project key is required.

The feature inventory describes implemented capabilities and planned extensions. Health/bank/social connectors, device push, voice, generated executable tools, and automatic reservations/payments are on the roadmap.

Quick start

Requirements: Node 24 LTS, pnpm 11.19.0, and a CopilotKit Intelligence project key. The local sample app needs no model, Google account, or Docker.

git clone https://github.com/CopilotKit/OpenMuse.git openfuse
cd openfuse
pnpm install --frozen-lockfile
cp .env.example .env
npx copilotkit@latest login
npx copilotkit@latest project select
# Set CPK_INTELLIGENCE_API_KEY in .env to the generated server-only project key.
# Keep the CLI-generated CPK_TELEMETRY_ID too, including in deployed server environments.
pnpm dev

In another terminal:

pnpm dev:web

Open localhost:8081. The API runs at localhost:8787/api/health.

Try it

  1. In Chat, send “Complete the permission slip”. Open the task, supply fictional form values, inspect the saved PDF, and review the prepared reply. This writes only to the local mailbox.
  2. In Goals → Track, create a built-in availability watch, then change the built-in test page to trigger an alert.
  3. In Menu → Delegate task → Finance, use Try example transactions to create an interactive spending tracker.
  4. Start the browser worker and configure a model, then ask “Check out Hacker News for cool stuff” or “Summarize copilotkit.ai”. Follow the browser inline and use Take control to open its session. For a model-free version of this flow, follow the AI Mock demo setup.

For iOS or Android, use pnpm --dir apps/mobile ios or pnpm --dir apps/mobile android. Xcode or Android tooling is required. The PDF reader needs an Expo development build; use native setup.

Deploy on Render

Deploy to Render

render.yaml deploys three services: the API, the web app, and a private browser. The API answers / with JSON, so the UI is its own static site.

First run

  1. Click Deploy to Render. Wait until openfuse-api, openfuse-web, and openfuse-browser are live.
  2. On openfuse-api, open Environment and copy OPENFUSE_ACCESS_KEY.
  3. Open the openfuse-web URL and sign in with that key.
  4. Send a message.

The deploy form asks for three values you provide. Render generates the other two.

Variable Set by If it is missing
CPK_INTELLIGENCE_API_KEY You. Run npx copilotkit@latest login, then npx copilotkit@latest project select. Keep it on the server. Chat cannot open a thread.
CPK_TELEMETRY_ID You. Keep the identity generated by the same project selection. Runtime usage cannot be attributed to the Intelligence signup account.
OPENAI_API_KEY You. Used by the default openai/gpt-5. Change MODEL and supply the matching provider key for Anthropic or Google. The model call fails.
OPENFUSE_ACCESS_KEY Render You cannot sign in.
TOKEN_ENCRYPTION_KEY Render The API refuses to start in live mode.

Health check: https://<openfuse-api>/api/health.

Services

Service Plan What it runs
openfuse-api Standard, with a 1 GB disk at /var/data The Hono API and the in-process task worker. DATA_DIR is /var/data/openfuse.
openfuse-web Static site The Expo web export. EXPO_PUBLIC_API_URL is baked in at build time.
openfuse-browser Private service, Standard, 1 GB disk at /data Playwright and Chromium. The API calls it on the private network.

Standard is the smallest plan that stays up. At 512 MB the process runs out of memory before it binds a port, because PGlite loads an embedded Postgres build.

The disk holds the database, PDFs, and the signing key. A redeploy without it wipes that data. Chat threads are stored by CopilotKit Intelligence, so a thread can still load after you sign back in even when the disk was never attached.

Live mode is required. Render binds 0.0.0.0, and sample mode rejects any host that is not loopback. The Blueprint sets WORKSPACE_MODE=live.

Browsing is included, and you can take it out. openfuse-browser is a private service, so it has no public URL. The API reaches it at http://openfuse-browser:8790 with a token Render generates. If the private hostname is not openfuse-browser, set BROWSER_WORKER_URL to http://<that-host>:8790. To deploy without it, delete the openfuse-browser service and the BROWSER_WORKER_URL and WORKER_TOKEN entries on openfuse-api. Chat, drafts, and tasks still run. Page reads, screenshots, and Take control do not.

The Docker computer and Google mail or calendar need the setup in the sections below. This Blueprint does not start them.

Configure the agent and Google

Copy the commented settings in .env.example into your private .env:

  1. Set AGENT_BACKEND=model, MODEL=provider/model-id, and the matching provider key. CopilotKit supports the configured OpenAI, Anthropic or Google provider. Fictional data can still be used with a real model. Provider keys stay on the server.
  2. Create or select a CopilotKit Intelligence project with npx copilotkit@latest login and npx copilotkit@latest project select. Keep the generated CPK_INTELLIGENCE_API_KEY on the server.
  3. For personal mail/calendar, set WORKSPACE_MODE=live, the generated CPK_INTELLIGENCE_API_KEY, a random OPENFUSE_ACCESS_KEY of at least 24 characters, and TOKEN_ENCRYPTION_KEY containing 32 random bytes encoded as base64. Restart the API.
  4. Configure a Google OAuth web client with Gmail and Calendar APIs enabled. Set GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET; register ${PUBLIC_API_URL}/api/google/callback as its redirect URI. Configure consent/test-user access in your Google project.
  5. Open Apps → Gmail (or Google Calendar), connect read access, and grant write access when needed. Every send or calendar change still requires its own stored review. Changing/disconnecting the account invalidates pending connection-bound work.

Google credentials are encrypted at rest. File URLs and browser consoles use short-lived signatures. This deployment uses one owner protected by a shared access key; it is not a multi-tenant authentication system. Use HTTPS and restricted network access for a remote host. Keep the default local-data mode on loopback.

Web search is enabled by default. Set WEB_SEARCH_ENABLED=false on the API and any separate task worker to disable it. With a configured model, delegated tasks and built-in model chat (AGENT_BACKEND=model) use Parallel's free, keyless Search MCP as their built-in search provider. Ask, for example, “Find the official CopilotKit React Native setup instructions and cite the sources.” Results include source URLs and excerpts; delegated tasks save them as evidence. Scripted sample chat and external AG-UI conversations keep their existing tools.

Search sends model-generated queries and context, which may include information from your conversation or task, plus a random per-chat/task session identifier to Parallel. See Parallel's privacy policy. Free access is rate limited; failures are reported without a paid fallback. Stopping chat or interrupting a task cancels its search.

Browser worker

Set BROWSER_WORKER_URL=http://127.0.0.1:8790 and a random WORKER_TOKEN of at least 32 characters in .env.

pnpm --dir apps/worker exec playwright install chromium
pnpm dev:browser

Or use docker compose --env-file .env -f infra/compose.yaml up --build -d. The same token must reach the API and worker. Sessions have persistent Chromium profiles; the app can open a live screenshot console and import PDF downloads. Agent tools can read public pages and hand interactive work to the person. Worker setup and boundaries.

Persistence and operation

Linux terminal and workspace

Build the computer image, enable it on the API, then open Computer → Terminal → Start computer:

docker build -t openfuse-computer:local apps/computer
COMPUTER_ENABLED=true pnpm dev

The API needs access to the Docker CLI and engine. Commands run in a nonroot container with no host-directory mounts or credentials. A named /workspace volume retains files when stopped. Terminal networking is disabled; public web access uses the browser worker. Commands have a 30-second limit and saved output/exit receipts. Files supports folders, text editing, and PDF transfer to/from Documents. This is a Linux container, not a full operating-system VM. Setup, Colima option, and boundaries.

Application storage

By default, embedded PGlite, documents and the signing key live in .openfuse/; browser profiles live in .openfuse/browser-profiles/. Keep that directory private and back it up. The API hosts the task worker. The host must remain running for background work.

For a separate task worker, configure the same DATABASE_URL, secrets and shared DATA_DIR for both processes, then set TASK_WORKER_ENABLED=false on the API and run pnpm dev:worker. PGlite cannot be opened by separate processes. Production commands are pnpm build:server, pnpm start and pnpm start:worker. Run one API instance; task workers coordinate through SQL leases.

No hidden retry occurs after an uncertain external write. Review its provider outcome before creating a replacement. Pausing/cancelling prevents subsequent task steps; an already approved in-flight provider request may finish.

CopilotKit Rich Threads

Every deployment requires CPK_INTELLIGENCE_API_KEY on the API server for CopilotKit Intelligence conversation persistence and replay. Create or select a project with npx copilotkit@latest login and npx copilotkit@latest project select, set the generated server-only key, and restart the API. The native menu uses useThreads; rich tool results link back to saved tasks, documents, and browser sessions.

Intelligence is a separate service and is not included in this repository's MIT license. No project key is shipped. Configuration and validation boundaries.

Signup and runtime tracking

The CopilotKit SDK sends runtime telemetry through the shared CopilotKit sink to PostHog with accessibility_title: "OpenFuse". Telemetry defaults to full capture; explicit SDK opt-outs and sampling settings take precedence. Keep the CLI-generated CPK_TELEMETRY_ID alongside the server-only project key when deploying. Intelligence records completed account signups separately; OpenFuse runtime usage can be joined to those accounts after the sink resolves the identity. Setup, opt-out and PostHog validation.

Architecture

flowchart TD
  Client[Expo / React Native / Web] -->|AG-UI and authenticated API| API[Hono + CopilotKit runtime]
  API --> Tasks[Durable task worker]
  API --> Threads[CopilotKit Intelligence required in every mode]
  API --> Store[(PGlite or PostgreSQL)]
  Tasks --> Store
  Tasks --> Review[Stored action review]
  Review --> Google[Gmail / Calendar adapters]
  Tasks --> Browser[Chromium worker + persistent profiles]
  API --> Browser
  API --> Computer[Optional Docker Linux computer]
  Tasks --> Computer
  Computer --> Volume[(Persistent workspace volume)]
  Tasks --> Files[PDF files + structured artifacts]
  API -. future adapter .-> OpenBot[OpenBot]
Directory Purpose
apps/mobile Shared iOS, Android, and web UI with CopilotKit headless hooks.
apps/server API, CopilotKit runtime, identity boundary, task engine, reviews, files, and persistence.
apps/worker Token-protected Playwright browser service with persistent profiles.
apps/computer Nonroot Linux image, bounded filesystem helper, and real container verification.
packages/domain Shared types and request validation.
packages/integrations Google and browser protocol adapters.
packages/backends Optional OpenBot HTTP adapter and its identity boundary.
tests Workflow, runtime, persistence, provider-contract, and authorization tests.

OpenBot compatibility

OpenFuse's native client and personal-agent workflows are independent of OpenBot. The disabled OpenBot adapter is pinned and contract-tested against upstream interfaces. Live user/session bridging, routine mapping, and computer backend wiring remain future work. OpenBot's Intelligence runtime is not a raw AG-UI endpoint. Integration contract.

Development

pnpm lint
pnpm typecheck
pnpm test
pnpm build:server
pnpm build:web
pnpm build:ios
pnpm build:android
pnpm --dir apps/worker typecheck
pnpm test:browser
pnpm test:computer

Platform build scripts export JavaScript/Hermes bundles; they do not produce signed app binaries. Browser checks use controlled HTTP fixtures; the local check requires installed Chromium, and the container check requires Docker and registry access to build its image. CI also exercises the browser and Linux computer containers. See contribution guidance and verification results.

Contributing and license

Issues and pull requests are welcome. Start with CONTRIBUTING.md, ROADMAP.md, and the security policy.

MIT licensed. Built by CopilotKit. Its original interface and fictional assets are included. Website, email, and document content supplies evidence, not permission to act.

openfuse (CopilotKit/openfuse)

正在搜索数据库…

这是什么?OpenMuse 是一款个人智能体应用:一个拥有自己的浏览器、终端、文件和可持续后台工作的 AI 智能体。你提出想要的结果,跟着计划走,审核它的操作,然后回来收取成果。
这是手机应用、网页应用,还是需要我自己部署的东西?三种都是,外加服务器端。它是一个基于 CopilotKit 的 React Native 应用,支持 iOS、Android 和网页端,而它的 API 服务器、任务执行器和 Playwright 浏览器执行器由你自己托管。
我该怎么在本地把它跑起来?克隆模板,安装 Node 24 LTS 和 pnpm 11.19.0,把 .env.example 复制为 .env 然后启动;示例应用会在 localhost:8081 打开,API 在 localhost:8787。本地示例不需要模型、Google 账号或 Docker。
试用它需要模型服务商的密钥吗?任何部署都需要一个 CopilotKit Intelligence 项目密钥来保存对话记录。要用真实模型,需设置 AGENTBACKEND=model、MODEL=provider/model-id 以及对应的密钥;支持 OpenAI、Anthropic 和 Google 服务商,密钥只保存在服务器上。AI Mock 配置则完全不需要模型。
它还是 alpha 版——目前有哪些做不到的事?图形化桌面和自动结账是未来的工作。Linux 终端限制单条命令最长 30 秒且禁用网络;一个部署通过共享访问密钥只服务单一所有者,不支持多用户。健康、银行、语音和支付相关的连接器还在路线图上。
我的数据和 Google 凭据会存在哪里?默认情况下一切都留在你自己的服务器上:内嵌的 PGlite、文档和签名密钥存放在 .openmuse/ 里,浏览器配置文件存放在 .openmuse/browser-profiles/ 里。Google 凭据静态加密存储,文件 URL 使用短期有效的签名,默认的本地数据模式只在环回地址上运行。
代码是 MIT 协议——里面的所有东西都可以随便用吗?仓库采用 MIT 协议,但支撑 Rich Threads 持久化和回放的 CopilotKit Intelligence 服务是独立的,不在该协议覆盖范围内,仓库里也不附带任何项目密钥。
它日常到底能帮我做什么?它通过流式 AG-UI 事件进行对话,并内联展示邮件、浏览器、PDF、计划和财务卡片;可以运行带审批、暂停/恢复和重试的持久任务计划;监视公开页面的变化、文字出现或美元价格阈值并去重告警;起草 Gmail 回复和日历事件供你审核;填写 PDF 表单;还能把交易 CSV 变成消费总结。
智能体的电脑是什么?一个持久的 Chromium 浏览器,外加一个可选的非 root Linux 容器。智能体可以浏览公开页面、执行命令并保存执行记录,文件保存在重启后依然存在的 /workspace 卷里;你可以打开它的浏览器或终端,在任务中途接管操作。它是一个容器,不是完整的操作系统虚拟机,而且终端无法访问网络。
我关掉应用后工作还会继续吗?会。API 托管一个任务执行器,计划是持久的:SQL 租约可以恢复中断的工作,后续任务会排队并清晰可见,外部写入结果不确定时也不会偷偷重试。后台任务需要宿主机保持运行,你也可以用 pnpm dev:worker 单独运行一个工作进程。
怎么连接我真实的 Gmail 和日历?设置 WORKSPACEMODE=live,创建一个启用了 Gmail 和 Calendar API 的 Google OAuth 网页客户端,注册回调 URL,然后在 Apps 下连接账号。凭据静态加密存储,而且每次发送邮件或修改日历之前,都需要经过你事先保存的审核确认。
有哪些类似的项目?rankme.fast, midscene, laya, inbox-zero, stagehand, open-dot, gogcli, crm
大小
49.9 MB
星标
6.5K
更新
提交Nov 17
时长
2个月
提交 · 90 天