We released our HeartMuLa-oss-3B-happy-new-year version. This version is currently the best open-sourced model in terms of lyrics controllability and music quality. We recommend using HeartMuLa-oss-3B-happy-new-year and HeartCodec-oss-20260123 for music generation.
⚖️ 03 Feb. 2026
We have released our HeartMuLa-Benchmark (referred to as HeartBeats Benchmark in our paper) as introduced in our paper. This benchmark comprises heterogeneous AI-generated lyrics and tags across diverse languages and genres, providing a rigorous and fair evaluation framework.
🚀 23 Jan. 2026
By leveraging Reinforcement Learning, we have continuously refined our model and are proud to officially release HeartMuLa-RL-oss-3B-20260123. This version is designed to achieve more precise control over styles and tags. Simultaneously, we are launching HeartCodec-oss-20260123, which optimizes audio decoding quality.
By default this command will generate a piece of music conditioned on lyrics and tags provided in ./assets folder. The output music will be saved at ./assets/output.mp3.
FAQs
How to specify lyrics and tags?
The model will load lyrics from the txt file --lyrics link to (by default ./assets/lyrics.txt). If you would like to use your own lyrics, just modify the content in ./assets/lyrics.txt. If you would like to save your lyrics to another path, e.g. my_awesome_lyrics.txt, remember to input arguments --lyrics my_awesome_lyrics.txt.
For tags it's basically the same.
CUDA out of memory?
If you have multi-GPUs (e.g. 2 4090s), we recommend placing the params of HeartMuLa and HeartCodec separately on different devices. You can do it by typing --mula_device cuda:0 --codec_device cuda:1
If you are running on a single GPU, use --lazy_load true so that modules will be loaded on demand and deleted once inference completed to save GPU memory.
All parameters:
--model_path (required): Path to the pretrained model checkpoint
--lyrics: Path to lyrics file (default: ./assets/lyrics.txt)
--tags: Path to tags file (default: ./assets/tags.txt)
--version: The version of HeartMuLa, choose between [3B, 7B]. (default: 3B) # 7B version not released yet.
--mula_device/--codec_device: The device where params will be placed. Both are set to cuda by default. You can use --mula_device cuda:0 --codec_device cuda:1 to explicitly place different modules to different devices.
--mula_dtype/--codec_dtype: Inference dtype. By default is bf16 for HeartMuLa and fp32 for HeartCodec. Setting bf16 for HeartCodec may result in the degradation of audio quality.
--lazy_load: Whether or not to use lazy loading (default: false). If turned on, modules will be loaded on demand to save GPU usage.
Recommended format of lyrics and tags:
[Intro]
[Verse]
The sun creeps in across the floor
I hear the traffic outside the door
The coffee pot begins to hiss
It is another morning just like this
[Prechorus]
The world keeps spinning round and round
Feet are planted on the ground
I find my rhythm in the sound
[Chorus]
Every day the light returns
Every day the fire burns
We keep on walking down this street
Moving to the same steady beat
It is the ordinary magic that we meet
[Verse]
The hours tick deeply into noon
Chasing shadows,chasing the moon
Work is done and the lights go low
Watching the city start to glow
[Bridge]
It is not always easy,not always bright
Sometimes we wrestle with the night
But we make it to the morning light
[Chorus]
Every day the light returns
Every day the fire burns
We keep on walking down this street
Moving to the same steady beat
[Outro]
Just another day
Every single day
Regarding tags, check this issue for reference.
Our different tags are comma-separated without spaces as illustrated below:
piano,happy,wedding,synthesizer,romantic
⚖️ License
This repository is licensed under the Apache 2.0 License.
📚 Citation
@misc{yang2026heartmulafamilyopensourced,
title={HeartMuLa: A Family of Open Sourced Music Foundation Models},
author={Dongchao Yang and Yuxin Xie and Yuguo Yin and Zheyu Wang and Xiaoyu Yi and Gongxi Zhu and Xiaolong Weng and Zihan Xiong and Yingzhe Ma and Dading Cong and Jingliang Liu and Zihang Huang and Jinghan Ru and Rongjie Huang and Haoran Wan and Peixu Wang and Kuoxi Yu and Helin Wang and Liming Liang and Xianwei Zhuang and Yuanyuan Wang and Haohan Guo and Junjie Cao and Zeqian Ju and Songxiang Liu and Yuewen Cao and Heming Weng and Yuexian Zou},
year={2026},
eprint={2601.10547},
archivePrefix={arXiv},
primaryClass={cs.SD},
url={https://arxiv.org/abs/2601.10547},
}
📬 Contact
If you are interested in HeartMuLa, feel free to reach us at [email protected]
Welcome to join us through discord or Wechat group.
Scan the QR code on the left to join our Wechat group. If it expires, feel free to raise an issue to remind us of updating.
If the number of group members exceeds 200, joining the group via directly scanning the QR code is restricted by WeChat. In this case, scan our team member's QR code on the right and send a request writing HeartMuLa Group Invite. We will invite you into the group manually.
heartlib (HeartMuLa/heartlib)
Choose a tag and it will appear here. With no tags selected, all catalog projects are shown.
Searching Database…
Choose a tag and it will appear here. With no tags selected, all catalog projects are shown.
What is this?HeartMuLa is a family of open-source music foundation models for generating music from lyrics and tags, shipped together with codec, transcription and audio-text retrieval companion models.
Is this one model or a set of models?A family of four: the main HeartMuLa music generator plus HeartCodec, HeartTranscriptor and HeartCLAP companion models.
How do I get it running?The README has no install commands — it links to a separate usage page that explains how to run the models.
What do I need installed before I can generate a song?A Python environment is the base, since the repo is Python code; the README lists no requirements inline and points to the usage page instead.
Is it really as good as Suno?The README claims the latest internal HeartMuLa-7B is comparable to Suno on musicality, fidelity and controllability; the published benchmark numbers are for the open oss-3B version.
Is there an app or API, or do I wire this into my own pipeline?It's a model repo for developers and researchers — you run the models in your own environment; no app or hosted service is mentioned.
Can it generate songs in my language?Yes — HeartMuLa is conditioned on lyrics and tags with multilingual support covering almost all languages, so non-English lyrics are in scope.
What do HeartCodec, HeartTranscriptor and HeartCLAP actually do?HeartCodec is a 12.5 Hz codec that reconstructs audio with high fidelity; HeartTranscriptor is a whisper-based model tuned for lyrics transcription; HeartCLAP unifies music and text descriptions in one embedding space for cross-modal retrieval.
So can I download the 7B, or only the smaller model?Published results cover the open oss-3B version, while the Suno-comparable HeartMuLa-7B is described as their latest internal version — so the 3B tier is what the repo openly offers.
HeartMuLa is a family of open-source music foundation models for generating music from lyrics and tags, shipped together with codec, transcription and audio-text retrieval companion models.