🎬 LAST FRAME
A playable film. Every shot is generated as you watch, and the video
model is the dice: whatever happens on screen is what happened, whatever
you intended. You asked to leap the chasm; the frame shows you dangling from
the ledge by one hand? That's what happened. Take the damage.
Powered by fal + MiniMax H3 Max,
a video model faster than realtime, which makes a game rendered by a video
model possible: the next shots film while the current one is still playing,
so picking one plays with no gap.
Two engines live in this repo:
- Shots (EVERYDOOR, THE LOST TAPE, the quest worlds): discrete 10-second
clips chained frame to frame, three branches pre-filmed per beat, an
Adjudicator ruling on every landed shot.
- Live (the Horror category): one continuous
MiniMax H3 Max Director
stream over WebRTC. No clips, no cuts, no freeze-frames. You steer it with
text and the world keeps moving while you decide.
From a run of THE LOST TAPE, the found-footage horror world. Shot 19: the
rocking chair starts moving on its own while the next choices film over the
still-rolling clip.
Shot 26, 5 vitality left. It stopped keeping its distance.
The rules
- 100 vitality, one life. The Adjudicator (a vision LLM that watches
every shot's frames) rules on what it sees and charges you for it.
Every HP change carries an evidence still of the frame that justified it.
- Three choices per beat, or type your own. The options surface over
the clip while it is still playing, and all three branches film in
parallel behind it. Pick one mid-clip and it rolls the instant the shot
ends. Press 1, 2 or 3, or click. Typed moves are refereed against the
current frame and risky ones roll a visible d20: the roll decides whether
the success or the failure shot gets filmed.
- Hesitate and the world moves. If the clip ends before you pick, the
frame freezes and a timer runs. Let it run out and the world takes its
own shot, never in your favor.
- What you see is what you take. Objects your hero visibly grabs become
inventory cards (with the frame they came from). An item is a stored
prompt fragment: arm one and it is written into your next shot, guaranteed.
- Win on screen. Quest worlds have a goal the Adjudicator must see
accomplished. Open worlds (EVERYDOOR, the horror) have no goal: one life,
see how far you get. Death gets a final generated cinematic; either way
you get the full timeline of your run, shot by shot, with every roll and
wound.
Live Horror
Five worlds on one continuous Director stream. Each is a different fear
grammar, and each carries two hidden decks the Director GM draws from at
random, so nobody can predict the beat:
| World |
The rule |
| THE LOST TAPE: LIVE |
Night woods, a rotting farmhouse, a tall pale shape that is always somewhere it should not be. |
| BLACKOUT |
An apartment tower losing power. The shadow exists only where there is no light, and every light is failing. |
| MIRRORS |
A grand empty house of glass. The wrongness lives only in reflections, until it does not. |
| MILE MARKER |
A night drive that will not end. The same figure on the shoulder, a mile later, closer to the road. |
| FOGBOUND |
A fog-drowned coastal town that emptied mid-dinner and is slowly remembering its people. |
How a live run works:
- The hero is always moving. The video model is good at walking, driving
and opening doors, and bad at a person standing still while a monster
performs at them.
- The unknown does the work. The entity is never seen clearly and never
approachable; light only ever fails, never flares. For the first two
thirds of the tape the GM can only draw subtle cards (a switch clicking
off behind you, footprints appearing one step at a time). Spike cards,
the real danger and the jumpscare, unlock in the final act.
- Every ~9 seconds, or the instant you make a move, the Director GM grabs
the live frame, rules vitality from what is on screen, and sends the next
segment as a
prompt message, which the stream applies at its next chunk
boundary. Vitality drains slowly by design; the tape is meant to be
survivable.
- Director sessions run about two minutes. That cap is the premise: a REC
clock counts down, and the tape running out means you survived it.
Quick start
npm install
cp .env.example .env.local # put your fal API key in it
npm run dev
Get a key at fal.ai/dashboard/keys. Open
http://localhost:3000 and pick a world. The live
worlds use the alpha line of @fal-ai/client for fal.realtime.open.
How the shot engine works
opening ──► play ──► ADJUDICATOR rules mid-playback (zero wall-clock)
│ │
│ ▼
│ options surface OVER the rolling clip
│ 3 branches pre-film in parallel (i2v from the last frame)
│ each ready branch is pre-adjudicated too
▼
pick mid-clip: rolls the instant the shot ends ──► loop
no pick: freeze-frame + timer ──► the world's own shot
- Continuity is the frame, novelty is the text. Every shot chains from
the previous shot's last frame (
image_url), so the film never breaks;
prompts only ever describe change. A rare hard cut (at most one
option per beat, only when no exit is visible) films text-to-video so the
story can actually leave a scene.
- The Adjudicator (Gemini Flash Lite via fal's vision router) returns
strict JSON per shot: scene ground truth, narration, HP delta + reason,
items gained/lost, a short synopsis of completed facts, death/victory
flags, the next three options, and the world's move if you stall. It runs
while the clip plays, and ready branches are ruled on before you pick
them, so the next options usually appear the moment a shot starts.
- The Referee judges typed actions against the frame: risk tier
(safe / risky 8+ / deadly 13+ on a d20) plus a success shot and a failure
shot. Moderation (strict ALLOW/BLOCK, injection-hardened, fail-closed)
runs in parallel server-side.
- Speech is scripted. The video model babbles unless voices are declared
wordless, and pronounces a line cleanly only when it is quoted verbatim
(measured with Whisper; see
experiments/). Prompts with a quoted line
get the scripted sound clause, everything else stays wordless.
- Your
FAL_KEY stays server-side (@fal-ai/server-proxy), and clips
stream through a same-origin media proxy so canvas frame grabs never hit
CORS.
Cost note
A shot-engine run films ~3 branch clips per choice plus one shot that
plays, at 480P / 10s each, plus a few small LLM calls per beat. A live
horror run is a Director session, billed per second of stream with a
60-second minimum, so budget a couple of dollars per two-minute tape. Keep
an eye on your fal usage; dying is the cheapest way to stop.
这是 LAST FRAME：一款像实时生成的电影一样游玩的游戏。你观看时，视频模型会把每一个镜头实时渲染出来，而模型本身就是骰子——如果画面里你的主角没跳过去、而是挂在悬崖边上，那事情就是这样发生了。