ffmpeg-skill
Give your coding agent a video editor.
Local FFmpeg · No cloud · No API keys · Python standard library
Claude Code · Cursor · Codex · MCP
npx ffmpeg-skill
ffmpeg-skill is an Agent Skill for Claude Code, Cursor, Codex and any agent that reads SKILL.md. It teaches the agent a fixed workflow (probe → edit losslessly where possible → check → verify) and ships 40 tools that do the actual work with ffmpeg / ffprobe: cut, join, silence removal, fit to duration and aspect, captions and karaoke, overlays and motion graphics, HDR → SDR and LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, whole-edit project rendering, batch folders. Every tool is also an MCP tool, and the whole set is described by a machine-readable contract.
If ffmpeg and python3 are on your PATH, it works: offline, on footage you would rather not upload.
SPEC (Self-Producing Execution Contract), coined by this project's author
kajisho5: each tool's input_schema — the part of its contract
and MCP tool definition that has to track the CLI flag-for-flag — is never hand-authored beside
the code. It's derived, at run time, from the same argparse parser that already defines the
CLI, and CI fails the build if any of it drifts. → full explanation
Standalone, and in an ecosystem
Standalone, this is a local FFmpeg engine: probe → edit → verify, npx ffmpeg-skill and nothing else. No API key, no account, no other repo required. Everything above and below this section describes that standalone tool, and none of it changes if you never read the rest of this one.
In kajisho5's wider video-production ecosystem, this repo is the hands: it cuts, measures and exports files, and reports back in structured JSON. It does not decide what to cut, whether a deliverable is approvable, or what makes a highlight interesting — those are a brain's job, sitting in front of this engine, not inside it.
Other repos in the ecosystem — media-analysis-skill, transcription-skill, subtitle-skill, thumbnail-skill, color-grading-skill, motion-graphics-skill, qc-skill — read this repo's contract --json, its tools' --json output and doctor, the same way any agent framework would; this repo does not call into any of them. The dependency runs one way.
Contents
Standalone, and in an ecosystem · Why · Quick start · How it works · Design principles · Tools · Audio · Built for agents · FFmpeg compatibility · Tested on real footage · Install · Requirements · Development · Docs
Why
An agent that "knows FFmpeg" still guesses: it assumes a frame rate, picks a codec the container cannot hold, re-encodes a file that only needed a stream copy, and reports "done" without opening the result. ffmpeg-skill exists to take the guessing out:
- Real files first. Every job starts with
probe.py; the agent decides from the measured duration, fps, resolution, colour and audio layout, not from the file name.
- Structured tools, not shell strings. Each operation is a script with typed arguments. Nothing runs through a shell; no filter graph is accepted from the caller.
- A contract the agent can read.
contract --json states, for every tool, what it takes, what it writes, which FFmpeg components it needs and how the result is verified. The MCP surface is derived from it.
- Verification after execution. The result is probed, checked against the destination's spec and, when the picture changed, looked at as a contact sheet.
- Local first. No cloud, no API keys, no Python dependencies. Optional local transcription is used when a whisper is installed, never required.
Quick start
# 1. install the skill for Claude Code (Cursor: --cursor, Codex: --codex, all three: --all)
npx ffmpeg-skill
# 2. check the machine: ffmpeg, ffprobe and every FFmpeg component the tools need
npx ffmpeg-skill doctor
# 3. (for agent frameworks) read the machine-readable contract
npx ffmpeg-skill contract --json | head -40
Already installed? re-run npx ffmpeg-skill to refresh ~/.claude/skills/ffmpeg-skill. Copies are not updated automatically.
doctor's overall ok and a single tool's usable: no are different signals: ok means nothing required by every tool is missing, but a plain Homebrew ffmpeg on macOS can still be ok while caption.py specifically can't run (no subtitles filter) — check doctor --json's tools field for the per-tool answer, not just ok.
Then talk to your agent:
"Take interview.mp4, keep 0:45–3:10 and 5:00–6:30, and make it exactly 60 seconds for Reels."
The agent runs probe.py, cut.py --segments 0:45-3:10,5:00-6:30, fit.py --duration 60 --aspect 9:16 --fit crop, export.py --preset reels, check.py --platform reels and look.py, then reports "final.mp4: 59.98 s, 1080×1920, 30 fps, AAC stereo" with the contact sheet it inspected.
The tools also work on their own, from any shell:
S=~/.claude/skills/ffmpeg-skill/scripts
python3 $S/probe.py input.mp4 --compact
python3 $S/fit.py input.mp4 --duration 60 --aspect 9:16 --dry-run # print the plan, run nothing
python3 $S/export.py input.mp4 --preset reels --json # structured result with a probe of the output
On Windows in Git Bash, python3 is only on PATH if Python was installed from the Microsoft Store; a python.org install exposes python (or the py launcher) instead — replace python3 with python above if you see a "command not found". bin/install.js and doctor/contract already handle this for you; only the raw script examples above need it spelled out manually.
More requests and the commands behind them: examples/README.md. To see everything run end-to-end on generated footage: npm run demo.
How it works
flowchart TD
U[User request] --> A[AI agent<br/>Claude Code · Cursor · Codex]
A -->|reads| S[SKILL.md<br/>workflow, request → tool map, report format]
A -->|runs| T[Structured tool<br/>scripts/<name>.py, typed argparse flags]
T --> C[Contract<br/>input schema · role · capabilities · verification policy]
C --> D[Capability detection<br/>doctor: available / missing / unknown]
D --> F[FFmpeg execution<br/>no shell, stream copy when possible]
F --> V[Verification<br/>probe · check · look.py contact sheet]
V --> R[Structured result<br/>--json: status, output, commands, probe]
R --> A
Over MCP the same tools are reached through a transport that holds no tool table of its own:
flowchart LR
M[MCP client<br/>Claude Desktop · Cursor · any client] --> P[mcp/server.py<br/>stdio JSON-RPC]
P -->|tools/list| C[Contract-derived ToolSpecs<br/>names · order · inputSchema]
P -->|tools/call| T[scripts/<name>.py]
C -.derived from.-> K[scripts/_contract.py]
T -.described by.-> K
Names, order and inputSchema in tools/list are translated from each tool's argparse parser at start-up, so a new flag or a new script appears in MCP with no edit to mcp/. A test copies the skill, adds, removes and edits a script, and reads tools/list again to prove it.
Design principles
These are the rules the skill file gives the agent and the code enforces. Together they are what separates this from a list of FFmpeg one-liners.
- Probe first. No tool decides from the file name.
probe.py measures duration, fps (with variable-frame-rate detection), resolution, rotation, bit depth, HDR format including Dolby Vision, colour tags and every audio stream before anything is cut.
- Lossless when possible.
cut.py, join.py and loudness.py stream-copy what they do not need to touch. Re-encoding happens only when it must: frame-accurate cuts, filters, format changes, or a keyframe farther than the tolerance.
- Plan before render. Every tool takes
--dry-run (print the ffmpeg command lines, write nothing), --json (structured result with a probe of the output), --fast (preview quality) and --progress (percent and ETA). A test runs every tool under --dry-run behind a fake ffmpeg and asserts that no ffmpeg call happened and no file appeared.
- Machine-readable contract.
contract --json describes all 40 tools: input schema generated from the parser, output schema, role, required and conditional FFmpeg capabilities, dry-run support, the verification tools to run afterwards, whether a visual check is required, mutates_input: false. provides lists all 40 by a cross-repository Capability id (ffmpeg-skill.cut, ffmpeg-skill.loudness, ...) for kajisho5/AI-video-production-OS's CapabilityContract.provides — see docs/contract.md.
- Contract-derived MCP.
mcp/server.py builds its tools/list from the contract. Tool names, order and inputSchema cannot drift from the scripts; a test keeps the two byte-identical.
- Capability detection.
doctor reads ffmpeg -encoders / -filters / -bsfs and reports which of the components the tools need are present on this build (libx264, libass, zscale, loudnorm, xfade, …), before a job fails inside ffmpeg.
- Unknown is not missing. When a listing cannot be read (a layout the parser does not know, ffmpeg exiting non-zero) the affected capabilities are
unknown: never missing, never silently available. An installed filter is not reported absent; a failed detection is not a pass.
- Verify the result. The output is probed, and when the picture changed (captions, overlays, crops, colour, transitions) the agent runs
look.py and inspects the PNG. The report is not finished until its Look: line names that image; audio-only jobs say Look: not needed. "Inspects" means the calling agent's own vision, not a feature of this skill: look.py only renders a PNG; nothing in this repository detects faces, products, subjects, or "the interesting part" of a frame or a scene. When a crop or reframe needs to keep a specific part of the frame (fit.py --fit crop --crop-x/-y, see Tools), it is the multimodal agent looking at that PNG and choosing the anchor — a non-visual caller (a script, a CLI user without eyes on the sheet) has to supply that decision itself, and the default is a plain centre crop. Likewise scenes.py --highlights ranks candidate scenes by a measured proxy (--rank-by audio or --rank-by duration), never by content; it is the agent that turns a look at the sheet into a judgement.
- Keep originals. No tool overwrites its input. Outputs are new files named
<input>_<operation>.<ext> unless told otherwise, and a test hashes every input after the run.
40 public tools, all Python 3.9 standard library, all with --help, --dry-run, --json, non-zero exit and a reason on stderr on failure.
Analysis and inspection
| Tool |
What it does |
probe.py |
Duration, fps (+ VFR detection), resolution, codecs, bit depth, HDR format incl. Dolby Vision, colour space, rotation, every audio stream; --analyze flags Log footage |
scenes.py |
Scene changes, audio peaks, highlight proposals (--rank-by audio loudest, or --rank-by duration longest — both proxies, not "best") and a per-scene sheet; cut list for cut.py --segments |
look.py |
Contact sheet, single frames, side-by-side comparison as PNG so the agent can see what it made |
Editing
| Tool |
What it does |
cut.py |
In/out or multi-segment cuts, lossless -c copy first, re-encode fallback, --accurate for frame-exact video and sample-exact audio; reports precision |
join.py |
Concatenate clips with xfade transitions, normalising size, fps and audio; audio-only inputs are joined as audio |
silence.py |
Detect and remove dead air (jump cuts) with a margin around speech; list or export the cut list |
fit.py |
Fit to a duration (pitch-preserving speed change or trim, smooth slow-mo) and/or aspect ratio (pad or crop, with --crop-x/--crop-y to keep an off-centre subject) and/or exact --width/--height; rotate 90/180/270, flip h/v; force constant fps |
crop.py |
Crop to an exact pixel rectangle (--x --y --width --height) — distinct from fit.py --fit crop, which crops to an aspect ratio it computes itself |
cropdetect.py |
Measure existing black letterbox/pillarbox bars and report the crop.py-ready rectangle that removes them — analysis only, writes no file |
deinterlace.py |
Deinterlace interlaced source footage (yadif), --mode frame/field, --parity |
denoise.py |
Reduce video noise/grain (hqdn3d), --strength low/medium/high or individual spatial/temporal overrides |
redact.py |
Blur or pixelate an exact pixel rectangle for the whole clip (privacy/compliance redaction) |
sphere.py |
Extract a flat rectilinear viewport from a 360/spherical video (--yaw --pitch --roll --h-fov --v-fov); no subject tracking, only the aim you give it |
straighten.py |
Rotate by an arbitrary angle for horizon correction (--degrees, --fit crop/pad) — distinct from fit.py --rotate's exact 90-degree turns |
insert.py |
Turn a still image into a silent, fixed-duration video clip (title card, end slate) at an exact frame size / fps, with an optional Ken Burns zoom/pan |
background.py |
Generate a solid-colour or two-colour gradient clip at an exact size/duration — no input file |
reverse.py |
Reverse playback (video and, unless --no-audio, audio) |
stabilize.py |
Two-pass motion stabilisation (vidstabdetect/vidstabtransform) |
sequence.py |
Numbered (frame_%04d.png) or glob-matched still images into a video |
waveform.py |
Render an audio track as a waveform or spectrum visualization video (showwaves/showspectrum) — for audio-only inputs with no picture worth showing |
freeze.py |
Hold a frame for N seconds (--at, --hold, --mode insert/extend) — an end-card hold or a comedic beat |
pad.py |
Add black/silent padding at the start and/or end of the timeline (--start, --end) — distinct from fit.py --fit pad's per-frame letterbox bars |
speedramp.py |
Step through different constant speeds across a clip via --segment START-END:FACTOR (repeatable) — distinct from fit.py's single whole-clip speed factor |
loop.py |
Repeat a clip --times N or to a target --duration — for background loops and filling a fixed slot length |
grid.py |
Composite --colsx--rows clips into one grid, each cell letterboxed and labelled with its filename by default (--label none to skip) |
Audio
| Tool |
What it does |
audio.py |
Voice clean-up chain, FFT denoise, typed compressor / limiter / gate, music bed with sidechain ducking, fades, 5.1 → stereo, track replacement, extraction (-o out.wav), --audio-stream N |
sync.py |
Offset between two recordings by audio cross-correlation (1 ms, pure Python), clock-drift correction; aligned video or audio out (audio-to-audio only — no lip-sync/face detection) |
loudness.py |
Two-pass EBU R128 loudnorm to −14 LUFS / −1 dBTP or any target, video stream-copied; --measure-only |
Picture
| Tool |
What it does |
caption.py |
Burn SRT/ASS with font, size, colour, outline, position; build SRT from timed plain text; animated and word-by-word karaoke timed to the speech energy; optional local transcription |
overlay.py |
Logos, watermarks and titles with position, time range, opacity, fades; --video for picture-in-picture, --chromakey for green-screen compositing |
graphics.py |
Lower-thirds, title cards, chapter chips, progress bars, countdowns, corner bugs drawn by FFmpeg from a brand kit |
color.py |
HDR10 / HLG / Dolby Vision → SDR BT.709 tone mapping, DV layer stripping, 3D LUT (.cube), colour-tag rewriting, typed primary correction (exposure/contrast/saturation/gamma/white balance/lift-gain/levels/curves) |
Delivery
| Tool |
What it does |
export.py |
Presets youtube, youtube4k, reels, x, prores, h265, gif, all tagged BT.709 |
proxy.py |
Small, low-bitrate proxy for downstream AI analysis/preview/editing decisions — resize by --width/--scale, proxy-grade --crf, --fps, --no-audio; not a delivery preset |
check.py |
PASS / WARN / FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, broadcast and podcast specs, with the fix for each failure and a format / judgement kind per row |
report.py |
Single-file HTML delivery report: before/after sheets, media facts, loudness, compliance, the commands run |
Orchestration
| Tool |
What it does |
render.py |
Render a whole edit from a declarative project.json (clips, transitions, captions, overlays, music, loudness, export, check); --init, --dry-run, --stop-after |
batch.py |
Apply a step recipe or a project to a folder with a content-hash cache; --watch |
multicam.py |
Align any number of cameras and recorders by audio (with drift correction) and cut between them from a switch list |
verify.py |
Run the toolchain on real device files and report PASS / FAIL per step |
Not tools, but part of the surface: mcp/server.py (the MCP transport) and scripts/_contract.py (contract --json, doctor). Per-flag reference for every tool: references/scripts.md.
WAV, FLAC, MP3, M4A/AAC, OGG and Opus go through probe, cut, join, silence, loudness, audio, sync and check --platform podcast with the same commands as video. The output extension picks the codec: `-o out.wa
ffmpeg-skill 是一个智能体技能（Agent Skill），为 Claude Code、Cursor、Codex 等编程智能体提供视频编辑器：它教会它们一套固定的“探测-剪辑-检查-验证”工作流，并附带 40 个 Python 脚本，实际调用 ffmpeg/ffprobe 来剪切、拼接、加字幕、适配尺寸、调色并导出视频。