PocketMen-with-you README

The project's README from github.com/six-nut/PocketMen-with-you. As one markdown file: README.md.

PocketMen with You companion capsule logo

PocketMen with You

Turn 2+ photos into a high-fidelity Codex companion — locally.

v0.3: Neural Local Studio. No hatch-pet. No OpenAI API key. Open-weight multi-reference editing when your GPU can run it.

GitHub stars GitHub issues Release

Try now · ⭐ Star · Share example

简体中文

Why v0.3 exists

The deterministic v0.2 engine was reliable and private, but it could only warp supplied pixels. It could not genuinely invent a new waving hand, a natural running stride, a faithful kitten pose, or a polished 3D chibi redesign.

v0.3 adds a real local generative engine. When supported hardware is present, PocketMen uses an open-weight multi-reference image editor to create a canonical master and state-specific animation key poses, then applies deterministic micro-motion, atlas assembly and QA. The result is far closer to a modern hosted image editor while remaining independent from OpenAI ImageGen.

Engine stack

2+ reference images
        ↓
Codex visual identity lock
        ↓
PocketMen hardware doctor
        ↓
┌─────────────────────────────────────┐
│ Neural Local Studio                 │
│ default: FLUX.2 [klein] 4B          │
│ optional: Qwen-Image-Edit-2511      │
│ multi-reference identity editing    │
└─────────────────────────────────────┘
        ↓
canonical master + semantic key pose per state
        ↓
flat chroma extraction / alpha cleanup
        ↓
deterministic micro-animation
        ↓
8 × 9 transparent Codex atlas
        ↓
validation + contact sheet + GIF previews
        ↓
pet.json + spritesheet.webp
        ↓
Codex local install

If the neural runtime or compatible GPU is unavailable, PocketMen falls back to the deterministic v0.2 engine instead of asking for OPENAI_API_KEY.

Local neural backends

FLUX.2 [klein] 4B — default

Qwen-Image-Edit-2511 — Identity-Max

PocketMen intentionally does not make FLUX.2 [dev] the default because its model license is non-commercial.

Quality profiles

Quick start in Codex

Upload at least two images and ask:

Use $pocketmen-with-you to create and install a companion from these references.
Use Neural Local Studio when my hardware supports it.
Style: hero-chibi.
Subject type: person.
Quality: max.
Preserve identity strictly and do not use hatch-pet or any OpenAI API key.

The skill should visually summarize stable identity traits from the references and pass them through --identity-notes.

CLI

Hardware check:

pocketmen doctor

Create with automatic engine selection:

pocketmen create \
  --reference photo-1.jpg \
  --reference photo-2.jpg \
  --name "Mochi" \
  --subject-type animal \
  --style soft-real \
  --engine auto \
  --quality balanced \
  --output ./pocketmen-output \
  --install

Force the default neural model:

pocketmen create ... --engine neural --backend flux2-klein-4b --quality max

Use the heavier identity backend:

pocketmen create ... --engine neural --backend qwen-image-edit-2511 --quality max

One-click setup

INSTALL_SKILL.bat installs the skill and creates an isolated runtime. On Windows, the installer checks NVIDIA VRAM; a compatible machine automatically receives the neural dependency profile.

To explicitly prepare and cache the default local neural model:

PREPARE_NEURAL_ENGINE.bat

For Qwen Identity-Max:

PREPARE_IDENTITY_MAX.bat

Model weights are downloaded from their original model hosts into the normal Hugging Face cache; they are not committed to this repository.

Codex pet contract

PocketMen emits an 8-column × 9-row atlas, 192×208 per cell, 1536×1872 overall. States are:

idle, running-right, running-left, waving, jumping, failed, waiting, running (Codex working, not physical running), and review.

What “ImageGen-like” means here

PocketMen narrows the quality gap specifically for reference-driven Codex companion creation by combining strong open-weight image editing, multi-reference identity anchors, structured prompts, dedicated state generation, deterministic alpha extraction and strict atlas QA.

It does not claim bit-for-bit or general-purpose parity with a proprietary frontier model. Broad world knowledge, typography, complex scene reasoning and difficult multi-object composition can still differ. For the narrow “2+ photos → consistent animated companion” workflow, however, v0.3 is designed to use the strongest local path available before falling back to simple warps.

Privacy

Raw personal references remain local by default and are never copied into public Git commits/releases. The final install package contains only pet.json and spritesheet.webp.

Community & maintenance playbook

For operation cadence (24h follow-ups, community sync templates, and weekly reports), use:

License

PocketMen code: MIT.

Optional model weights retain their own licenses. The default FLUX.2 [klein] 4B and Qwen-Image-Edit-2511 backends are referenced, not redistributed.

PocketMen with You is not affiliated with Nintendo, The Pokémon Company, Game Freak, OpenAI, Black Forest Labs, Alibaba/Qwen, Hugging Face, or GitHub.

PocketMen-with-you (six-nut/PocketMen-with-you)

Searching Database…

311 ·GitHub
PocketMen-with-you
What is this?PocketMen with You turns 2+ of your photos into a high-fidelity, animated 'Codex companion' pet that runs entirely locally using open-weight image models instead of OpenAI's API.
Is this a standalone app or a Codex skill?It is a Codex agent skill you call inside the Codex CLI/agent, not a separate GUI program; the one-click installer sets up an isolated runtime and the skill runs the generation pipeline for you.
How do I install it and create my pet?Run INSTALLSKILL.bat to install the skill and create an isolated runtime, then inside Codex upload at least two photos and ask it to build your companion so it can extract stable identity traits via --identity-notes.
Do I need an OpenAI API key or a specific GPU?No OpenAI API key is required; for the full neural pipeline you need a compatible NVIDIA GPU to run the default FLUX.2 [klein] 4B model or the heavier Qwen-Image-Edit-2511 backend, both fetched into the normal Hugging Face cache.
What happens if my GPU can't run the neural models?PocketMen automatically falls back to the deterministic v0.2 engine, which can only warp your supplied pixels and can't invent new poses like a wave or run cycle, instead of asking for an API key.
What happens to the photos I upload?Your raw reference photos stay local by default and are never committed to public Git history or releases; only the final pet.json and spritesheet.webp leave your machine.
Are the model weights free to use commercially?PocketMen's own code is MIT, but weight licenses differ: the default FLUX.2 [klein] 4B and Qwen-Image-Edit-2511 backends are Apache-2.0, while FLUX.2 [dev] is deliberately kept off by default because its weights are non-commercial.
What quality profiles can I pick?Draft generates one neural canonical master plus a deterministic nine-state animation; balanced adds neural key poses per state (reusing some by mirroring); max independently generates a neural key pose for every single state.
What does the final output look like?PocketMen produces an 8-column by 9-row sprite atlas (192x208 per cell, 1536x1872 overall) covering states like idle, running, waving, jumping, failed, waiting and review.
Why not just use OpenAI's image generation?PocketMen narrows the quality gap for the specific 'photos to animated companion' task using local open-weight editing, multi-reference identity anchors and strict QA, so you get a similar result without an API key, cost, or sending photos to a cloud service.
What are similar projects?WanGP, qwen21-nsfw-pose-skill, sprite-gen, dsh-image-gen, xxg-portrait-rebuild-light, Open-LLM-VTuber, stable-diffusion-webui, no-negative-echo
Which topics is it in?spritesheet, qwen-image, desktop-pet, openai-codex, neural-rendering, local-ai, image-editing, codex-skill, offline-first, agent-skill, multimodal, image-generation
Creator
six-nut
Size
0.2 MB
Stars
311
Updated
CommitOct 01Releasev0.3.0Aug 17
Age
2mo
Commits · 90 Days