ComfyUI-MiniMaxH3Mod - Changing the F#cking world with cigarretes and coffe.
🚧 Under construction — API and node schemas are still evolving. Mods
stay compatible, but expect node names/inputs to shift between versions.
What's new — v0.2.5
On main after v0.2.5: Refresh RefMods updates the current Loader/Axis
dropdowns from the RefMod folders without refreshing all ComfyUI models.
Selections and connected inputs are preserved. The library's Refresh also
updates that loader's dropdowns.
- Clearer creation controls — Create node titles, Full/Compressed Reference modes and Refinement Steps; old workflows remain accepted.
- Explicit visual budget policy — choose truncate or error before saving, including Master.
- Compatibility fixes — projected ClipProj encoders and batched H3 VAE decoding in Text Encode.
- Full visual inspection — optional full-video preview, with a lightweight first-frame default.
Since v0.2.0: extraction widgets accept token budgets above
65,536 and frame limits above 16, including Master. The motion_sequence
preset now respects the requested frame limit. Counts remain integers;
defaults are unchanged.
Experimental: H3 RefMod Text Encode presents saved references to the native
H3 text/vision encoder and reports their <Picture n>, <Video n> and
<Audio n> labels. Character binding and voice quality still require generation
tests; this is not a validated voice-cloning fix.
- RefMod Master — extract image/video and audio references in one node,
with separate VAE inputs, one output bundle and a combined token budget.
- Integrated audio — create and load audio RefMods directly in this pack;
no H3AudioMod installation required. Mix visual and audio refs in Apply.
- Known failure: voice transfer — audio support includes an observed music
reference result, but speaker-identity transfer failed in current tests.
This update does not provide working voice cloning. See
voice-transfer limitations.
- Library and inspector — search/filter your collection, select loader
slots, inspect token costs and optionally preview stored visuals or audio.
- Save H3 RefMods — a dedicated output node saves bundles without needing
a Preview, sampler or another downstream consumer.
- Linked loaders fixed — unresolved names from upstream nodes no longer
fail queue validation, including both sides of the Axis loader.
- Subfolders and external drives — recursive discovery and saving to the
registered RefMod root respect
extra_model_paths.yaml.
- Stale references and cache fixes — Step Curve no longer retains refs
from previous calls; loaders detect overwritten files. Bridge injection is
scoped to the MODEL branch, with cleanup after sampling errors.
- Extraction fixes — masks follow image crops, video sampling stays
bounded when frame counts are unknown, and runtime token/copy limits are
enforced. Saves use atomic file replacement.
- Fewer dependencies and repeated operations — removed sibling-pack imports;
shared extraction helpers and grouped multi-ref refinement reduce duplication.
Full history: CHANGELOG.md. Validation and remaining limitations:
REVIEW_FOLLOWUP.md.
RefMod in brief
RefMod saves MiniMax H3 image, video or audio references as reusable
.safetensors files. Load a saved reference, combine it with others, and pass
the bundle to Apply H3 RefMod. Create H3 RefMod Master brings visual
and audio extraction into one node.
Extraction uses the corresponding VAE. It does not train H3 weights or require
the diffusion model. The visual mode named training refines a compressed
latent; it is not LoRA training or supervised concept learning.
Saving avoids re-encoding the source on each run. Compression can reduce the
number of reference tokens processed during generation, at the cost of lost
information. Encode mode keeps more detail but does not remove the attention
cost of those tokens. Neither mode guarantees that only the desired attribute
will transfer: identity, clothing, background and composition can still mix.
Try it
A ready-made example mod ships in the repo: mods/vanellope_example.safetensors.
It appears as vanellope_example in the Load H3 RefMods dropdown after
install — plug it into Apply H3 RefMod at strength 1.0 and prompt for
a candy racer in a karting scene.
Install
Use a current ComfyUI with native MiniMax H3 support. RefMod does not
import or require ComfyUI-MiniMaxH3. Connect a standard H3 video VAE to
vae, the audio VAE to audio_vae, and native CONDITIONING to Apply.
Old av_encoder connections are supported by reading the VAERef video
checkpoint through ComfyUI's native VAE loader; they no longer load the
sibling pack's VAE implementation or its audio checkpoint. Prefer vae
to share an already loaded instance. Existing pack-conditioning objects
remain accepted by Apply without importing their package.
This pack: clone into custom_nodes/ and restart ComfyUI. Python
deps (safetensors, numpy, Pillow) are in requirements.txt and are
installed automatically by ComfyUI Manager (or pip install -r requirements.txt manually). opencv-python/imageio are optional video
backends for the folder loader.
git clone https://github.com/Luisacaotica/ComfyUI-MiniMaxH3Mod custom_nodes/ComfyUI-MiniMaxH3Mod
Tested on Windows; os.path-based paths so it should work on Linux/Mac, but
only Windows has been exercised so far.
Visual creation also offers budget_policy: truncate (the default) uses
existing frame reduction to fit max_tokens; error stops before saving if
the combined visual latent exceeds the budget after the multiplier. This applies
to both Create and Master. max_tokens=0 disables the cap. Master's audio policy
and combined budget remain separate.
Create/Master now print which visual controls the effective configuration ignores
and which values a preset replaces. Apply reports actual curve/retention overrides
and inactive scrambling controls. These are execution-time notices; widgets remain
visible and editable, and cached nodes do not rerun just to print a notice.
The preset tooltip lists exact assignments. motion_sequence preserves the frame
limit and Refinement Steps and disables frame-difference extraction (motion_only)
to retain the visual sequence. concept_type is metadata, not an algorithm selector.
On Apply, curve_shape still matters with constant for non-linear shapes; a missing
saved override does not disable manual controls or discard a valid graph preset.
The UI calls the modes Full Reference (encode internally) and
Compressed Reference (training internally). Refinement Steps retains
its historical input ID identity. Saved files and API workflows keep these
internal identifiers; the descriptions below use them when discussing code.
Node titles now use Create instead of Extract, with unchanged node IDs.
A saved visual latent enters H3 through the native reference-token path.
That shares the model's reference mechanism, but does not establish parity
with a complete native reference workflow: resizing, conditioning, prompt
processing, sampling and other references can differ.
| Visual mode |
Operation |
Tradeoff |
encode |
Resize/preprocess and store the VAE encode |
Keeps more reference detail; larger token cost. Useful as the baseline for identity comparisons. |
training |
Pool the latent, then optionally refine it against the original VAE latent |
Fewer tokens; detail and motion may be lost. |
The training loss is MSE between the full latent and a trilinearly enlarged
small latent. Only the small latent is optimized. No DiT, text instruction,
identity recognizer or motion objective participates. identity is the number
of refinement steps, despite its historical name; 0 means pooling only.
Increasing it does not teach the model which attribute to preserve or discard.
concept_type is descriptive metadata, not a separate learning algorithm.
The aliases full → encode and pooled → training remain supported.
Storage
Mods are saved in the first registered refmods folder, including mappings
from extra_model_paths.yaml. The fallback is ComfyUI/models/refmods/, created
when saving. Set is_default: true in the YAML mapping to prioritize that root.
A write failure is reported rather than silently switching drives. Config updates
keep the original path of a loaded file. Loaders search all registered roots and legacy locations, including
the pack's own mods/ folder, so existing files keep working. The standalone CLI
uses the folder registry available in its process; it does not load YAML mappings
on its own.
Both Load H3 RefMods and Load H3 RefMod Axis scan subfolders too:
models/refmods/celebs/person.safetensors appears as celebs/person.
Names use / without the .safetensors extension; linked Windows names
using \ work too. Only files with RefMod metadata (kind: image, video or audio) are listed. graph_presets/, .git/, and __pycache__/ trees
are skipped. Extract saves to the selected root by default; subfolder is optional.
You can convert mod_#, mod_a_#, or mod_b_# to inputs and connect a
STRING/combo output. Unresolved linked values are checked when the loader
runs. None, an empty string, "None", and "(none)" skip the slot;
a resolved missing filename raises a missing-file error. Fixed dropdown
values are still checked when queuing. After adding or moving files,
refresh ComfyUI's node definitions to update the visible dropdowns.
Loader regression checks (run with ComfyUI's Python from this pack):
python tests/test_loaders.py. They use temporary safetensors files and
ComfyUI's queue validator; no generation models are needed.
Integrated audio, library and inspection
Create H3 RefMod Master combines the visual extractor's inputs and controls
with optional audio and audio_vae inputs. Connect the visual H3 VAE to vae
(or the existing av_encoder) and the H3 audio VAE to audio_vae. Either modality
can be omitted. Video-frame inputs do not implicitly include an audio track;
connect the source loader's AUDIO output separately when needed.
The master extracts visual references first, then audio, returning one mods
bundle for Apply/Inspect and a details string with paths and token totals.
Visual max_tokens and audio_max_tokens are separate; max_total_tokens limits
their combined cost (0 disables that extra limit). A failed extraction or budget
check does not save either result. Each final file is saved atomically, but saving
the pair is not a filesystem transaction.
With name=hero and subfolder=characters, saving creates
characters/hero_visual.safetensors and characters/hero_audio.safetensors.
Select both in Load H3 RefMods to reconstruct the bundle after restarting.
The internal visual extractor reports (not saved) because Master postpones
saving until both extractions and the total-budget check succeed. Master's final
Created / Replaced messages show the actual paths. encode and training
both replace existing files at those paths; an older file without the modality
suffix is a different destination. With save=False, Master only returns the bundle.
These are appearance and voice references, not joint character training or a
guarantee of audiovisual synchronization. The individual extractors remain available.
Create H3 Audio RefMod accepts AUDIO and the H3 audio VAE (the
32 kHz codec, not the video VAE). The implementation follows the tested
ComfyUI-H3AudioMod encode path: stereo normalized [1,32,2,T] latents,
40 frames/second, 2 tokens/frame, bounded 10-second encode chunks.
max_seconds explicitly selects the initial duration of the clip. A token
budget overflow raises an error by default; optional truncate keeps a
contiguous prefix rather than resampling speech/music in latent time.
Chunk boundaries may affect codec continuity; this is reference conditioning,
not a promise of lossless audio reconstruction.
Save audio to the selected RefMod storage root with an optional subfolder. Existing files
from models/audio_refmods/ using audio_refmod_meta load directly. Both
common loaders, Apply, Step Curve, Config and the model-scoped bridge accept
mixed visual/audio bundles. Keep different voices in separate files for
independent strengths. New saves use format version 4; older visual mods
remain readable. No <Name> prompt triggers are implemented.
Both loaders now have a RefMod library button: search names, subfolders,
concepts and descriptions, filter image/video/audio, choose a slot and Use.
Refresh rescans the disk. Connected slots are protected from widget replacement.
Close and Escape dismiss the library. Registered extra model roots are honored.
Inspect H3 RefMod reports paths, shapes, metadata, saved config and total
cost after copies. Optional matching vae enables stored or
compare_strength previews: visual_preview=first_frame decodes the first
latent frame by default; full_video decodes the complete stored visual latent
as an IMAGE batch for a video-combine/save node. Full previews use more memory.
In comparison mode, the stored sequence is followed by the weakened sequence;
audio outputs preview up to two seconds. Comparison appends the weakened
preview after the stored preview. This shows stored information, not a
prediction of generated identity/style quality.
max_total_tokens on loaders/Apply/bridge limits the sum after copies
(0 disables it). Oversized bundles fail explicitly. Extraction also fails
when one frame alone exceeds its cap. Loaded-file cache checks file and
sidecar changes, and is bounded to 256 MiB and 24 entries.
Visual Extract offers opt-in presets: manual preserves current widgets;
identity_encode selects encode/1024px; style_experimental selects an
8×8 training grid; motion_sequence keeps an ordered temporal sequence.
These are starting settings, not quality guarantees or semantic disentanglement.
Both visual and audio Extract support an optional save subfolder.
Validation: python -m unittest discover -s tests -v runs production-code
regressions. python gauntlet_harness.py --device cuda compares resident,
streaming and grouped multi-ref refinement with numerical parity checks.
python tests/audio_smoke.py PATH_TO_H3_AUDIO_VAE exercises the real codec,
safetensors roundtrip and native H3 reference layout without loading the DiT.
Each file contains a latent and JSON metadata in the safetensors header.
Visual latents have shape [1,24,T,H,W]; audio latents use [1,32,2,T].
The Master saves two independent files when both modalities are present.
Visual token count is T × (H/2) × (W/2). H and W are even latent-grid
dimensions, not image pixels. Audio uses 2 × T tokens.
| Visual latent grid |
Tokens per latent frame |
| 8×8 |
16 |
| 16×16 |
64 |
| 32×32 |
256 |
| 64×64 |
1024 |
For example, a square 1024×1024 image encoded to a 64×64 latent costs 1024
tokens and about 192 KiB of fp16 tensor data, plus metadata. A rectangular
image with a 1024-pixel short edge can cost more. Four 16×16 latent frames
cost 256 tokens; sixteen 64×64 frames cost 16,384 tokens before budget fitting.
A small grid is spatial compression, not a concept extractor. It may retain
colors and large structures while losing face detail, texture or useful motion.
A larger grid preserves more information, including unwanted content. There
is no validated universal 8×8 concept / 16×16 identity sweet spot.
multiplier repeats the extracted visual latent along its time axis; loader
copies repeats a reference block. Both increase token cost. Neither adds new
information, and neither promises a proportional increase in influence.
Visual max_tokens defaults to 5120 (0 disables it). Budget fitting drops
near-duplicate latent frames, then resamples time if needed, after repetition.
This can damage motion timing; inspect the resulting frame/token count. If a
single spatial frame exceeds the cap, extraction raises an error. Audio instead
uses the explicit error/prefix-truncation policy described above.
Nodes (MiniMax-H3/mod)
| Node |
Purpose |
| H3 RefMod Text Encode |
Encode a prompt with numbered saved references; outputs conditioning and the reference map. |
| Create H3 RefMod Master |
Visual and/or audio extraction into one bundle, with separate VAE inputs. |
| Create H3 RefMod |
Visual extraction, masks, compression and optional refinement. |
| Create H3 Audio RefMod |
Audio extraction with duration and token limits. |
| Save H3 RefMods |
Save a bundle as an output node; no downstream connection required. |
| Load H3 RefMod Folder |
Ordered image/video references from a folder. |
| Load H3 RefMods |
Up to 8 slots with strength, copies and optional total budget. |
| Load H3 RefMod Axis |
Select A or B with a signed strength; 0 skips the slot. |
| Apply H3 RefMod |
Append refs to native CONDITIONING or an existing pack-conditioning object. |
| H3 RefMod Step Curve |
Change marked RefMod latents during denoising via the MODEL connection. |
| Fix H3 RefMod Config |
Persist Apply/Step Curve settings in mod metadata. |
| Inspect H3 RefMod |
Metadata, token totals and optional VAE previews. |
| Continuum RefMod Bridge |
Inject refs through the MODEL sampling hook. |
Older Apply H3 RefMod (Cond) workflows migrate to the unified Apply node.
The legacy Bridge Disarm node remains for compatibility; bridge state now
belongs to the MODEL branch.
Prompting with numbered RefMods (experimental)
Connect Load H3 RefMods → H3 RefMod Text Encode.mods, the native H3 CLIP
or an H3-compatible projected CLIP (current ClipProj Loader all-in-one),
and the H3 video VAE for visual references. Enter your prompt and connect the
CONDITIONING output to the sampler's positive input. Keep the workflow's normal
negative conditioning and generation latent. This node already attaches the
references: do not Apply or Bridge the same bundle again.
Connect reference_map to a text display to see the actual mapping. For example,
if it reports <Picture 1> = alice and <Picture 2> = beth, try:
[Shot1]
The woman in <Picture 1> stands on the left.
The woman in <Picture 2> stands on the right.
They turn toward each other and smile.
The map counts each modality separately, in bundle order, excluding zero-strength
entries. Copies receive additional labels. Stacked photos saved as one video-kind
RefMod receive one <Video n> label, not one Picture label per original photo.
Keep different characters in separate files. Neither the filename nor description
is a trigger, and <Subject n> is not automatically bound to a loader slot.
Visual presentation requires decoding the stored latent for Qwen; it adds VAE and
vision-encoder work. Compressed latents reconstruct less detail than the original
photos. Videos are decoded and then sampled for Qwen at 2 fps; reference_fps
sets reconstructed playback timing (default 24). Original timing is not recovered
from pooled or stacked refs. Audio presentation uses the native numbered label
without decoding audio. Loader strengths affect the latents; saved Apply curves
and overrides are not applied by this node. max_total_tokens limits DiT reference
tokens, not Qwen tokens or VAE decode memory.
Presentation/payload tests pass, but full GPU generation and identity/voice
quality remain unverified. Compare against the old Apply wor
A ComfyUI custom node pack that adds 'RefMod' reference adapters for the MiniMax H3 model. It compresses reference images, videos or audio into small reusable .safetensors files that can be loaded and applied to influence generation without any training of the H3 model.