MOSS-Image README

项目的 README,来自 github.com/OpenMOSS/MOSS-Image。合并为一个 markdown 文件:README.md。

MOSS-Image MOSS-Image

Hugging Face · MOSS-Image-Base   Hugging Face · MOSS-Image-SFT   ComfyUI Models   ModelScope · MOSS-Image-Base   ModelScope · MOSS-Image-SFT   Hugging Face Spaces · Demo   MOSI API Platform   Homepage   Technical Report   arXiv: 2610.23374   Follow OpenMOSS on X   MOSS-Image QQ 交流群:755535143   MOSS-Builders Feishu Community   MOSS-Image Discord Community

English   |   简体中文

News

Community projects

User guide

Start with the English user guide or 中文用户指南 for recommended inference settings, prompt examples, layout input formats, and LoRA training guidance.

ComfyUI

MOSS-Image is supported natively in ComfyUI. Update to ComfyUI master/nightly with MOSS-Image support and install its dependencies, then use the built-in loaders with ComfyUI-format weights. No MOSS-Image plugin is required for this workflow.

The repository plugin remains available for loading a local Hugging Face / Diffusers directory and for existing plugin workflows.

Installation and usage · HF loader plugin workflow

Prompt enhancer skill

The MOSS-Image Prompt Enhancer turns image ideas into detailed captions or spatial Layout prompts. It is designed for GPT-6-family agents with local file and Python tools. The bundled script converts pixel-coordinate drafts into compact JSON with integer centered coordinates; it does not call an image-generation service.

Install the skill folder in your agent's skills directory, then invoke $moss-image-prompt-enhancer with your image idea, desired caption or Layout output, and any canvas dimensions. Paste the compiled result into MOSS-Image's positive prompt; for Layout, use the same generation dimensions.

MOSS-Image-Base and MOSS-Image-SFT

Contents

🤗 Models

Model Description
MOSS-Image-Base · ModelScope Base text-to-image model
MOSS-Image-SFT · ModelScope Fine-tuned from MOSS-Image-Base on curated data

Base vs. SFT

In this example, SFT produces clearer text grouping and a more organized whiteboard layout.

MOSS-Image-Base MOSS-Image-SFT
MOSS-Image-Base MOSS-Image-SFT

🚀 Quick start

🛠️ Installation

git clone https://github.com/OpenMOSS/MOSS-Image.git
cd MOSS-Image
python -m venv .venv
source .venv/bin/activate

pip install -r requirements.txt

Apple Silicon / MPS

Apple silicon Mac setup and limitations · M1 Max / 32GB

🐍 Python

import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "OpenMOSS-Team/MOSS-Image-Base",
    custom_pipeline="pipeline_moss_image",
    trust_remote_code=True,
    torch_dtype={"default": torch.bfloat16, "vae": torch.float32},
)
pipe.scheduler = type(pipe.scheduler).from_config(pipe.scheduler.config)
pipe.to("cuda")
image = pipe(
    "A wide view of a refrigerator beside a kitchen counter with a fruit bowl. A small sticky note on the refrigerator door is handwritten in Chinese:“\n记得买牛奶和鸡蛋!\n牛奶两盒,鸡蛋一打。\n顺便带一袋面包。”",
    height=1456,
    width=1456,
    num_inference_steps=50,
    guidance_scale=3.0,
    uncond_skip_layers=[22],
    generator=torch.Generator("cpu").manual_seed(42),
).images[0]
image.save("refrigerator_note.png")

🎨 Gradio

pip install -e '.[ui]'
python gradio_app.py --model OpenMOSS-Team/MOSS-Image-Base

🎛️ Sampling settings

Setting Default
Sampler Flow-matching Euler
Steps 50
CFG 3
Shift 5
Layer skipping Negative-branch skipping at layer 22
Negative prompt (Python API) Empty

See the user guide for sampling and prompt recommendations.

⚡ Memory and performance

Select Memory mode in Gradio's Advanced settings:

Mode Behavior
GPU resident Keep the entire model on GPU
Component Move model components between CPU and GPU as needed
Sequential · fallback Lower memory use with slower inference
Low memory Keep DiT weights in CPU RAM; load text encoder/VAE weights from disk as needed

If GPU memory is insufficient, select Sequential or reduce image size. For lower GPU and system memory use, select the minimal profile (Low memory mode):

python gradio_app.py --profile minimal --model OpenMOSS-Team/MOSS-Image-Base

The auto profile selects minimal below 12 GiB of GPU budget. Keep the model files on an SSD: DiT stays in CPU RAM, while the text encoder and VAE load weights as needed.

Measured peak usage for one 1024×1024 image with 50 steps:

Resource Peak usage
PyTorch GPU memory allocated / reserved 3.65 / 3.85 GiB
Task system RAM, including file cache 14.00 GiB

Compile DiT is disabled by default and unavailable in Sequential and Low memory modes.

📐 Layout-to-image generation

MOSS-Image uses JSON layout inputs to control text placement and object arrangement in complex text-to-image compositions. The image below was generated by MOSS-Image using layout control.

Layouts can also support controlled editing and user-friendly layout tools. By editing text, element descriptions, and bounding boxes, users can adjust image content and composition in a structured way.

Layout bounding boxes

Paste the layout JSON into Gradio’s Prompt field to specify the scene, text, objects, and their positions. The following example uses a summary/items layout for a 1024×1024 canvas. Save it as layout.json:

{
  "summary": {
    "meta": ["poster"],
    "canvas": "single",
    "caption": "A minimalist travel poster with a cream background, a blue mountain illustration, and the title EXPLORE at the top."
  },
  "items": [
    {
      "label": "text",
      "caption": "Large dark-blue uppercase title centered at the top.",
      "text": "EXPLORE",
      "bbox": [-416, -384, -288, 384]
    },
    {
      "label": "object",
      "caption": "A flat blue mountain illustration in the lower center.",
      "bbox": [-192, -288, 384, 288]
    }
  ],
  "bbox_coord_space": "abs_centralized",
  "bbox_format": "yxyx"
}

Keep the coordinates consistent with the requested output dimensions. In Gradio, paste the JSON into the prompt field and set the matching width and height. With the Python API, pass json.dumps(layout, ensure_ascii=False) as the prompt string.

🖼️ Showcases

Examples overview

Examples overview 2

📝 Citation

If you use MOSS-Image in your research, please cite our technical report:

@misc{ai2026mossimagetechnicalreport,
      title={MOSS-Image Technical Report},
      author={MOSI. AI},
      year={2026},
      eprint={2610.23374},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.23374},
}

📄 License and acknowledgements

The standalone pipeline code is provided under the Apache License 2.0.

We thank the Diffusers, Qwen and FLUX teams.

⭐ Star History

Star History Chart

MOSS-Image (OpenMOSS/MOSS-Image)

正在搜索数据库…

这是什么?MOSS-Image 是一个 50 亿参数的文生图扩散 Transformer(DiT)模型,擅长渲染文字,支持用 JSON 控制文字和物体的位置布局,还能高效微调(包括 LoRA),适配自定义任务。
这是现成的应用,还是需要我自己搭建的模型?它以基础版和 SFT 模型的形式发布,附带 Python 推理代码和 Gradio 演示,还提供原生 ComfyUI 插件,因此你可以通过脚本、网页界面或在 ComfyUI 中生成图片,无需自己搭建流程。
我该怎么把它跑起来?按照 README 的 Installation 部分配置 Python 环境,从 ModelScope 获取 MOSS-Image-Base 或 MOSS-Image-SFT 权重,然后通过 Python API 运行推理,或启动 Gradio 应用;使用 ComfyUI 的话,只需通过 ComfyUI Manager 安装插件即可。
在显存有限的 GPU 上运行有什么要注意的吗?显存紧张时,你需要切换到速度较慢的 Component 或 Sequential 内存模式;另外,DiT 编译默认关闭,在 Sequential 模式下则完全无法使用。
MOSS-Image-Base 和 MOSS-Image-SFT 有什么区别?Base 是预训练的文生图基础模型,SFT 则在精选数据上做了微调,文字分组更清晰,版面也更有条理,README 中的白板对比示例可以直观地看到这一点。
JSON 布局控制到底是怎么工作的?你需要编写一个 layout.json 文件,其中包含一段总体描述和一个条目列表,每个条目包括标签、描述、要渲染的文字,以及以像素为单位、以画布中心为基准的边界框。然后,将该 JSON 粘贴到 Gradio 的提示词输入框中,并设置相同的宽度和高度,即可控制文字和物体的位置。
Prompt Enhancer 技能是做什么用的?这是为 GPT 系列编程代理准备的一项技能,可以把粗略的图像构思转换为详细的描述文字,或转换为紧凑的 Layout JSON(使用整数形式的居中坐标),然后你再把结果粘贴到 MOSS-Image 的提示词中;它只负责整理提示词,本身不会调用任何图像生成服务。
我的文本提示词可以写多长?MOSS-Image 最多支持 4K 个 token 的提示词,因此你可以写详细的长篇场景描述,而不只是简短的图片描述。
作者
OpenMOSS
大小
85.5 MB
星标
2.9K
更新
提交Nov 08
时长
1个月
提交 · 90 天