MiniMax-Music3 README

The project's README from github.com/MiniMax-AI/MiniMax-Music3. As one markdown file: README.md.

MiniMax

MiniMax Agent API MiniMax Music 3 Live Demo MiniMax Website
ModelScope MiniMax AI WeChat Discord Hugging Face GitHub LICENSE

MiniMax Music 3

MiniMax Music 3 is a high-performance music generation model for creating complete songs up to five minutes long. Conditioned on lyrics and a detailed music description, it generates structurally coherent songs with expressive vocals, evolving arrangements, and stable long-form audio quality.

MiniMax Music 3 combines an 8B Global LLM for long-range musical structure, a 0.6B Local LLM for frame-level acoustic detail, and a continuous hidden-state synthesis system based on Flow Matching and Flow-VAE. The model produces 32 kHz, 16-bit stereo WAV audio.

Prompt Enhancement

A concise natural-language description can be used directly. For richer prompts and more precise control, use the provided music-caption-rewriter skill to expand it into a Structured Caption containing Global Metadata, Vocal Details, and Arrangement. The skill preserves musical instructions attached to lyric section tags in the arrangement description while keeping the lyric text in the lyrics input. See the skills page for installation instructions and a full feature overview.

npx skills add MiniMax-AI/MiniMax-Music3 --skill music-caption-rewriter

Demo

Explore music generation examples on the MiniMax Music 3 Demo.

MiniMax Music 3 architecture diagram

Complete Songs with Long-Range Coherence

MiniMax Music 3 natively supports full-song generation up to five minutes. The model maintains musical themes, rhythm, vocal identity, and arrangement progression across long sequences, enabling complete structures such as intro, verse, pre-chorus, chorus, bridge, instrumental break, and outro.

Fine-Grained Music Control

The model accepts two complementary inputs:

For precise control, we recommend using a Structured Caption with three sections:

This representation allows the model to follow not only a global style, but also the musical development of the song over time.

Hybrid-LM

MiniMax Music 3 uses a hierarchical autoregressive architecture that separates global musical modeling from local acoustic modeling.

The Global LLM is initialized from Qwen3-8B. During training, its embedding and output layers are first adapted to semantic music tokens. The Global and Local LLMs are then jointly trained to model all RVQ codebooks.

Continuous Hidden-State Synthesis

Instead of decoding only from discrete RVQ tokens, the synthesis module fuses the final hidden states of the Global and Local LLMs. These continuous representations preserve richer acoustic information for vocal articulation, instrumental texture, and temporal continuity.

The synthesis path is:

Global and Local LLM hidden states
                ↓
       Hidden-state fusion
                ↓
     Flow Matching (2.4B)
                ↓
        Flow-VAE latent
                ↓
    Flow-VAE Decoder (123M)
                ↓
       32 kHz stereo audio

The Flow-VAE architecture is adapted from MiniMax Speech and retrained for the dynamic range and spectral characteristics of music.

Music Tokenizer

The training tokenizer uses eight layers of Residual Vector Quantization (RVQ):

Training first optimizes the semantic codebook, then jointly trains all eight codebooks. At inference time, waveform synthesis uses the fused LLM hidden states and does not require the discrete tokenizer decoder.

How to Use

Download the Model

hf download MiniMaxAI/MiniMax-Music3 --local-dir /path/to/minimax_ttm

Serve with SGLang-Omni

MiniMax Music 3 is currently supported by SGLang-Omni. From the SGLang-Omni repository root, run:

sgl-omni serve --model-path MiniMaxAI/MiniMax-Music3 --port 8000

Inference uses two CUDA GPUs:

Generate Music

The service uses the shared speech API. Put the lyrics in input and the music description in instructions.

curl http://127.0.0.1:8000/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "minimax_ttm",
    "input": "[Verse]\nMorning light filtering through the pine\n[Chorus]\nSoftly the world begins to breathe",
    "instructions": "A warm acoustic pop song with intimate female vocals, fingerpicked guitar, soft piano, and a gradual emotional build into a wide final chorus.",
    "response_format": "wav",
    "seed": 7,
    "max_new_tokens": 9000,
    "stream": false
  }' \
  --output minimax_music3.wav

An example generated with this checkpoint is available at assets/minimax_ttm.wav.

Music Caption Rewriter Skill

This repository includes music-caption-rewriter, an agent skill that turns a brief music description and optional tagged lyrics into a detailed Music 3.0 Structured Caption. It uses a bundled text library to select compatible style references, combine requested influences, and build a coherent section-by-section arrangement without external APIs or runtime dependencies.

Then invoke $music-caption-rewriter in a supported coding agent with a music description and, optionally, lyrics containing section tags such as [Verse] or [Chorus]. The skill returns Global Metadata, Vocal Details, and Arrangement while preserving explicit musical constraints and keeping lyric text out of the rewritten caption.

Limitations

Contact Us

Contact us at [email protected].

MiniMax-Music3 (MiniMax-AI/MiniMax-Music3)

Searching Database…

961 ·GitHub
MiniMax-Music3
What is this?MiniMax Music 3 is a music generation model that creates complete songs up to five minutes long from lyrics and a music description, producing 32 kHz 16-bit stereo WAV audio with coherent structure and expressive vocals.
Is this a ready-to-use app or just a raw model?It is a raw model checkpoint, not a hosted app — you download the weights and run them yourself through a supported inference server.
How do I actually generate a song with it?Download the model, serve it from the SGLang-Omni repository root, then call the shared speech API putting your lyrics in the input field and the music description in instructions.
What do I need to run inference besides the weights?You need the SGLang-Omni serving stack, since it is currently the only supported way to run this model.
What are the main limits I should know about?Generation is non-streaming only, text prompts are capped at 5,000 tokens, audio output is limited to 9,000 acoustic frames, and requested tempo, key, instrumentation or structure may not always be followed exactly.
How many GPUs does inference need, and what do they do?Two CUDA GPUs are required: one handles the Qwen3-based eight-codebook RVQ autoregressive generation, and the other runs Flow Matching and waveform decoding.
What does the music-caption-rewriter skill do?It is a bundled agent skill that turns a brief description and optional tagged lyrics into a detailed Music 3.0 Structured Caption with Global Metadata, Vocal Details, and Arrangement, using an internal text library without external APIs.
How do I control the song's structure, like verses and chorus?Lyrics can include section tags such as [Intro], [Verse], [Pre-Chorus], [Chorus], [Bridge], [Instrumental], and [Outro], while the music description separately controls style, instrumentation, and arrangement across those sections.
How does the model keep a 5-minute song coherent?It uses a hierarchical Hybrid-LM: an 8B Global LLM (initialized from Qwen3-8B) models long-range musical structure, while a 0.6B Local LLM fills in frame-level acoustic detail, with both feeding a Flow Matching/Flow-VAE synthesis module for final audio.
What are similar projects?jeeves, YuE, audiocraft, vllm-omni, DramaBox, Qwen3.8-Flash-Next, vllm, TypeLLM
Which topics is it in?generative-model, text-to-audio, music-generation, sglang, model, inference-engine, model-serving, audio-generation, audio-processing, music, qwen, multimodal
Creator
MiniMax-AI
Size
36.1 MB
Stars
961
Updated
CommitAug 14
Age
2mo
Commits · 90 Days