mesh-avatar-studio README

项目的 README,来自 github.com/shinshin86/mesh-avatar-studio。合并为一个 markdown 文件:README.md。

Mesh Avatar Studio

日本語 · 简体中文

Turn a single illustration into an animated 2D mesh avatar: it blinks, talks, turns its head, breathes and sways its hair. A coding agent prepares the avatar from your image; you fine-tune it in a local editor with a live preview.

See it move on the landing page. To move it with your own face and voice without installing anything, open the Player in Chrome or Edge.

The editor with the Miko sample

Try the sample

Requires Node.js 22.17+ and Git.

npx create-mesh-avatar-studio
cd mesh-avatar-studio
npm run dev

The first command downloads the repository, installs its packages and checks the tools Studio needs. If you cloned the repository yourself, run npm install, then npm run doctor for the same check. Only Node.js is needed for the editor and the Live page. uv and Python rebuild layers, and Playwright Chromium renders the motion review when an agent creates an avatar; the check warns about each one that is missing.

Open the URL printed in the terminal (for example http://127.0.0.1:5173/). The editor opens the bundled sample, Miko in a qipao. Pick a part on the left, drag its dots on the image and watch the preview on the right. The sample is read-only; when you change it, choose Copy and keep editing to continue in your own copy.

Using Windows

Use git clone to download the repository when possible. If you use a ZIP, open its Properties in File Explorer and select Unblock, if shown, before extracting it.

Create projects/ yourself under your Windows account before asking an agent to work in it (npx create-mesh-avatar-studio creates it for you). Folders created by another account or an agent running in a sandbox can have permissions that prevent the editor from reading them. If a project cannot be read, use the folder's Properties → Security settings to grant your account access to the whole project, including its files and subfolders through permission inheritance.

How it works

Making an avatar from your own illustration takes four steps. Steps 1 and 4 are done by a coding agent (Claude Code or Codex) working in this repository; steps 2 and 3 happen in the editor.

1. Ask your agent to create the avatar

Open Claude Code or Codex in this repository folder and give it your illustration:

Create an avatar from this image following docs/agent-guide.md: /path/to/your-image.png

The agent follows the agent guide: it checks how accurately it reads coordinates, places the rig on zoomed grids, cuts the image into layers and reviews the avatar in fixed poses. The project is saved in projects/<name>/, which is never committed. Best results come from a front-facing, head-and-shoulders PNG with a transparent background. When no project is open, the editor shows this request ready to copy.

The request for a new avatar

Recommended setups:

Agent Model
Claude Code Claude Opus 5.5
Codex GPT-6.1 Sol

One command with Codex

With the Codex CLI installed and signed in, one command creates the project, has Codex follow the agent guide without questions and opens the editor on the result:

npm run create -- /path/to/your-image.png

Codex also draws closed eyes, mouth shapes and expressions with OpenAI image generation, and first closes an open mouth in the illustration so the avatar rests with it closed. This sends the image and masks to OpenAI, so the command asks first; add --yes to agree in advance, or --no-variants to keep everything on your machine. It usually takes tens of minutes. Codex writes its notes to projects/<name>/HANDOVER.md. It runs with --approve-for-me, so its requests to run tools outside its sandbox, such as Chromium for the pose review, are reviewed automatically instead of by you. npm run create -- --help lists the options.

Without an illustration, --random has Codex draw a new anime-style girl first and builds the avatar from her. Her looks are chosen at random; --prompt describes what you want instead, and --seed repeats the same random choices. The drawing is kept in work/random/, and the choices in projects/<name>/work/character.json.

npm run create -- --random
npm run create -- --random --prompt "silver short hair, school uniform"

From your phone with a cloud agent

Send one illustration to an agent with its own cloud computer, such as Dot, Muse or Grok Bot, and ask it to create an avatar and, where supported, a page you can open on your phone. See Create an avatar with a cloud agent for a message to copy and what has been tried.

2. Open the project and check it

Start the editor (npm run dev), choose Open project and pick your project from the list. The pencil next to a project renames it, together with its folder under projects/. If the avatar has overlay layers such as glasses (built/overlays/), the preview lists them under Overlays so you can switch them on and off. In the preview, use Pose test to turn and tilt the head and close the eyes. In Lip sync, choose a mouth style and try it with kana text. Open Fine-tune to adjust movement, size and closing, or preview a local audio file.

Lip sync check

3. Fix what looks wrong by dragging

Select a part on the left. Its dots are highlighted on the image; drag them to fix the placement. The preview follows your changes immediately. Some values also decide how the image is cut into layers: eye and hand outlines, and the head, hair and bun areas used for the hair mask. After changing those, a banner appears; choose Save and rebuild layers there. It takes a few seconds.

Editing the eye outline

Rebuilding layers after changing an outline

Zoom with a pinch or Ctrl/Cmd+scroll, pan with two fingers or by dragging empty space, and use Fit selected part to jump to the current part.

4. Get drawn eyes and mouths

Without drawings, the eyes and mouth move by mesh deformation alone. Drawn closed eyes and vowel mouths make blinking and talking look much more natural. In Drawn variants, check Eyes, Mouth or both, copy the message and paste it into Codex opened in this repository folder. Codex draws them with its built-in image generation and imports them; the editor reloads them automatically. (Claude Code cannot generate images; its message prepares the masks and prompts for you to hand to an image generator.)

Expressions is optional: it adds drawn sad and surprised eyes and closed angry and sad mouths. The avatar switches to them for those expressions and emotion tags and still blinks and speaks; avatars without them look the same as before.

Requesting drawn mouths

Share your avatar

Choose Export .mavatar in the editor to download one file containing the saved rig, built layers, drawn expressions and avatar metadata. Include source image is on by default so the recipient can edit it in Studio. If you have unsaved edits, use Save and export; changed outlines need a layer rebuild first. Read-only views export the original files.

In Open project, choose Import .mavatar or drop the file there. Studio creates a new folder under projects/, using a new name if one already exists. Import requires npm run dev and accepts archives up to 128 MiB. Without the source image, the imported avatar can play but cannot be edited or rebuilt. Omitting the source does not remove the built images. See the avatar format for the file contents.

Live and streaming

The Live page

Open your project in the editor and choose Live. Start the camera, face forward with relaxed eyes and a closed mouth, then choose Calibrate. You can adjust mirroring, sensitivity and smoothing, or enable microphone lip sync. Camera video and microphone audio stay on your machine; only numeric avatar motion values reach the stream view.

Track body movement (experimental) is off by default. While it is off, only the face is tracked, as before. When it is on, Live also uses visible shoulders, hips, elbows and wrists, and lists what the loaded avatar follows: the upper body always, legs and arms only when its rig sets them up. Body points from a webcam are less stable than the face, so expect some extra sway, especially in skirts and other cloth. See Full-body tracking.

Choose a background and Copy OBS URL, or Open stream view in a new tab to check it first. Keep the Live page open and add the URL as an OBS Browser Source, for example at 1080 × 1080. The transparent background works directly in OBS. For other capture software, choose green and apply a chroma key. Both pages use the local development server (npm run dev). The stream view shows idle motion when the Live page stops sending updates.

Optional Lighting adds shading that follows a light you drag over the avatar. It is off by default; open its section near the bottom of the Live page, or below the preview in the editor. The OBS URL includes the lighting settings. See Lighting.

If the avatar has overlay layers such as glasses (built/overlays/), the Overlays section of the Live page switches them on and off. The stream view follows at once, and the OBS URL includes the overlays that are shown.

Keep the Live page open in its own window. Open the OBS URL inside OBS or in another tab; do not paste it into the tab running Live. If the page reports that tracking has stopped or slowed while hidden, bring its window to the front.

The stream view with a green background

Switch expressions such as a smile, surprise or a wink with the Expression buttons or keys 1–8 while tracking continues. A Stream Deck button or a keyboard shortcut can switch them while another app such as OBS is in front; macOS Shortcuts keys do not work in every app (details).

Save up to eight lighting setups, such as a red light from below for a horror game, as presets, and switch them with Shift+1–8 or the same kind of shortcut (details).

The camera is optional: with only the microphone on, the avatar keeps its idle motion and your voice moves the mouth. To appear as the avatar in Zoom, Google Meet and similar apps, use OBS's Virtual Camera (steps).

Stream view options, troubleshooting and privacy details are in the reference.

Without installing anything

The Player runs face tracking, microphone lip sync, expressions and lighting entirely in the browser. Open the sample or your own .mavatar file; nothing is uploaded. It has no stream URL: choose Open avatar window (Chrome or Edge) and add that window to OBS with Window Capture on a green background and a Chroma Key filter. Transparency, the OBS Browser Source and switching expressions from other apps need Studio's Live page.

Experimental native runtime

A Rust runtime, PNG renderer and transparent macOS desktop avatar are available in the native guide. They load the same .mavatar files and built project folders as the JavaScript runtime. The native version is experimental; its APIs and controls may change.

Releases

For changes to mesh-avatar, run npm run changeset and include the generated file in your PR. On main, GitHub Actions opens or updates a chore: version packages PR with the next version and changelog; merging it publishes mesh-avatar to npm with provenance. The Studio root package stays private. Publishing uses npm trusted publishing: on npmjs.com, add a trusted publisher for mesh-avatar that names this repository and the release.yml workflow; no npm token is stored in the repository. A publish that npm cannot match to a trusted publisher waits in the Staged Packages tab until a maintainer approves it with 2FA. In repository Settings → Actions → General, enable Allow GitHub Actions to create and approve pull requests so the workflow can open version PRs. Run npm run e2e locally before opening a PR; CI runs the other checks and does not install Playwright browsers.

More

License

The code is released under the MIT License.

The sample character Miko (samples/miko-qipao/) is not covered by the MIT License. Miko is the character of AITuber OnAir, © Yuki Shindo (AITuber OnAir), and her images are provided under the Miko Character Usage Guidelines; see samples/miko-qipao/MIKO_ASSET_TERMS.md. They may be used and modified as part of your own works, but not redistributed on their own or as an asset collection. This project is not an official AITuber OnAir product.

Face tracking and the experimental body tracking use MediaPipe (@mediapipe/tasks-vision and the Face Landmarker and Pose Landmarker lite models in vendor/mediapipe/), © The MediaPipe Authors, licensed under the Apache License 2.0; see vendor/mediapipe/LICENSE.

mesh-avatar-studio (shinshin86/mesh-avatar-studio)

正在搜索数据库…

这是什么?Mesh Avatar Studio 可以把一张插画变成一个会动的 2D 网格形象,它会眨眼、说话、转头、呼吸,还会让头发摇摆。编码智能体(Claude Code 或 Codex)会根据你的图片准备形象,然后你可以在本地编辑器里通过实时预览进行微调。
这是独立应用,还是需要依赖 AI 编码智能体?它是一个本地网页编辑器(通过 npm run dev 运行),再配合一套工作流:在你自己微调之前,由 AI 编码智能体完成绑定骨骼和图片分层等主要工作。
我该怎么让它跑起来?使用 npx create-mesh-avatar-studio 下载仓库、安装依赖包并运行依赖检查;也可以自己克隆仓库,然后运行 npm install,再运行 npm run doctor。需要 Node.js 22.17+ 和 Git。
它支持哪些编码智能体?做一个形象还需要其他什么?Claude Code 或 Codex 会根据你的图片生成骨骼绑定和图层;uv 和 Python 用于重新生成图层,Playwright Chromium 则在制作过程中渲染动作预览;npm run doctor 会检查上述所有组件,并提示缺少的部分。
Claude Code 也能处理画好的眼睛和嘴巴吗?不能。Claude Code 无法生成图片,所以在这一步它只会准备蒙版和提示词,交给你去用图像生成工具。Codex 则可以通过其内置的图像生成功能,直接绘制并导入眼睛和嘴巴。
不安装任何东西,能驱动这个形象吗?可以。Player 完全在 Chrome 或 Edge 中运行人脸追踪、麦克风口型同步、表情和灯光,不上传任何内容。不过它没有直播地址,所以 OBS 浏览器源的透明效果仍需使用 Studio 完整的 Live 页面。
进行 Live 追踪时,我的摄像头和麦克风数据会去哪里?摄像头画面和麦克风音频都保留在你的电脑上;只有形象动作的数值会发送到直播视图。
导出完成的形象后会得到什么?在编辑器中选择「Export .mavatar」,会下载一个单独的文件,其中包含保存的骨骼绑定、已生成的图层、绘制的表情以及形象元数据(最大 128 MiB)。如果包含了原始图片,其他人可以导入并重新编辑。
直播时,即使在其他应用里,也能切换表情吗?可以。点击表情按钮或按数字键 1-8,就能切换微笑、眨眼等表情,同时追踪持续进行。即使 OBS 或其他应用在前台,也可以通过 Stream Deck 按键或键盘快捷键触发表情。
Windows 上安装时有哪些需要注意的坑?ZIP 压缩包需要先在文件资源管理器的「属性」中勾选「解除锁定」,再解压;另外,由其他账户或沙盒智能体创建的 projects/ 文件夹,可能需要通过文件夹的「属性」→「安全」手动修改权限。
大小
50.4 MB
星标
2.4K
时长
1个月
提交 · 90 天