Awesome-Multimodal-Large-Language-Models README

The project's README from github.com/BradyFU/Awesome-Multimodal-Large-Language-Models. As one markdown file: README.md.

Awesome-Multimodal-Large-Language-Models

✨ Highlights of NJU-MiG

🔥🔥 Surveys of MLLMs | 💬 WeChat (MLLM微信交流群)

🔥🔥 VITA Series Omni MLLMs | 💬 WeChat (VITA微信交流群)

🔥🔥 MME Series MLLM Benchmarks

[🍎 Project Page] [📖 Paper] [🤗 Dataset] [🏆 Leaderboard]

Table of Contents

Awesome Papers

Multimodal Instruction Tuning (& Latest Works)

Title Venue Date Code Demo
GPT-6 Astra: A new generation of intelligence Blog 2026-09-04 - -
Introducing Gemini 3.8 Flash and 3.8 Flash Cyber Blog 2026-09-02 - -
GLM-5.3-Flash: Frontier Intelligence, Flash Cost Zhipu 2026-08-26 Huggingface -
DeepSeek-V4-Flash-Vision-Exp DeepSeek 2026-08-21 - -
Qwen3.8-Max: A New Bar for Coding and Cowork Blog 2026-08-15 Huggingface Demo
Star
DeepSeek Harness
DeepSeek 2026-08-13 Github -
Star
Qwen-MM-Plugins
Qwen 2026-08-11 Github -
Star
Kimi K3: Open Frontier Intelligence
Kimi 2026-07-27 Github Demo
Star
VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding
arXiv 2026-07-16 Github Demo
Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity arXiv 2026-06-30 - -
Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models Blog 2026-06-15 - -
Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation arXiv 2026-06-15 - Demo
Star
OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains
arXiv 2026-06-12 Github Demo
GitHub Repo stars
Cosmos 3: Omnimodal World Models for Physical AI
arXiv 2026-06-05 Github Demo
π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities arXiv 2026-04-24 - Demo
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence DeepSeek 2026-04-24 Huggingface -
Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model Blog 2026-04-22 Huggingface Demo
Xiaomi MiMo-V2.5 Blog 2026-04-22 - Demo
Star
Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding
arXiv 2026-04-06 Github Demo
Introducing Muse Spark: Scaling Towards Personal Superintelligence Blog 2026-04-08 - Demo
Star
VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing
arXiv 2026-04-03 Github Local Demo
Gemma 4: Byte for byte, the most capable open models Blog 2026-04-02 - Demo
Qwen3.6-Plus: Towards Real World Agents Blog 2026-04-02 - -
Qwen3.5-Omni: Scaling Up, Toward Native Omni-Modal AGI Blog 2026-03-30 - Demo
Xiaomi MiMo-V2-Omni Blog 2026-03-18 - -
Star
InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing
arXiv 2026-03-10 Github Local Demo
Star
Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion
arXiv 2026-03-06 Github -
Beyond Language Modeling: An Exploration of Multimodal Pretraining arXiv 2026-03-03 - -
Gemini 3.1 Pro: A smarter model for your most complex tasks Blog 2026-02-19 - -
Star
Qwen3.5: Towards Native Multimodal Agents
Blog 2026-02-16 Github Demo
Star
MiniCPM-o 4.5
Blog 2026-02-06 Github Demo
Star
Kimi K2.5: Visual Agentic Intelligence
arXiv 2026-02-02 Github -
Star
DeepSeek-OCR 2: Visual Causal Flow
DeepSeek 2026-01-27 Github -
Seed1.8 Model Card: Towards Generalized Real-World Agency Bytedance Seed 2025-12-18 - -
Introducing GPT-5.2 OpenAI 2025-12-11 - -
Introducing Mistral 3 Blog 2025-12-02 Huggingface -
Star
Qwen3-VL Technical Report
arXiv 2025-11-26 Github Demo
Star
Emu3.5: Native Multimodal Models are World Learners
arXiv 2025-10-30 Github -
Star
VITA-E: Natural Embodied Interaction with Concurrent Seeing, Hearing, Speaking, and Acting
arXiv 2025-10-21 Github Local Demo
Star
DeepSeek-OCR: Contexts Optical Compression
arXiv 2025-10-21 Github -
Star
OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM
arXiv 2025-10-17 Github -
NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching arXiv 2025-10-16 - -
Star
InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue		 arXiv 2025-10-15 Github -
Star
VITA-VLA: Efficiently Teaching Vision-Language Models to Act via Action Expert Distillation
arXiv 2025-10-10 Github -
Star
LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training
arXiv 2025-10-09 Github Demo
Star
Qwen3-Omni Technical Report
arXiv 2025-09-22 Github Demo
Star
InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
arXiv 2025-08-27 Github Demo
MiniCPM-V 4.5: A GPT-4o Level MLLM for Single Image, Multi Image and Video Understanding on Your Phone - 2025-08-26 Github Demo
Star
Thyme: Think Beyond Images
arXiv 2025-08-18 Github Demo
Introducing GPT-5 OpenAI 2025-08-07 - -
Star
dots.vlm1
rednote-hilab 2025-08-06 Github Demo
Star
Step3: Cost-Effective Multimodal Intelligence
StepFun 2025-07-31 Github Demo
Star
GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
arXiv 2025-07-02 Github Demo
Star
DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World
arXiv 2025-06-30 Github -
Qwen VLo: From "Understanding" the World to "Depicting" It Qwen 2025-06-26 - Demo
Star
MMSearch-R1: Incentivizing LMMs to Search
arXiv 2025-06-25 Github -
Star
Show-o2: Improved Native Unified Multimodal Models
arXiv 2025-06-18 Github -
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities Google 2025-06-17 - -
Star
Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning
arXiv 2025-06-16 Github -
Star
MiMo-VL Technical Report
arXiv 2025-06-04 Github -
Star
OpenUni: A Simple Baseline for Unified Multimodal Understanding and Generation
arXiv 2025-05-29 Github -
Star
Emerging Properties in Unified Multimodal Pretraining
arXiv 2025-05-23 [Gi

Awesome-Multimodal-Large-Language-Models (BradyFU/Awesome-Multimodal-Large-Language-Models)

Searching Database…

18.1K ·GitHub
Awesome-Multimodal-Large-Language-Models
这是什么？这是一份经过整理、持续更新的列表，收录多模态大语言模型（MLLM）最新进展相关的论文、模型、数据集和基准测试。这些模型，比如 GPT-4o、Gemini 和 VITA，能够在文字之外理解图像、视频和音频。
这是论文还是可以直接运行的工具？这是一份参考列表（README），不是软件。它只是把论文、项目、代码仓库和演示的链接按主题整理在一起，主题包括指令微调、上下文学习、思维链和基准测试等。
这个列表是怎么组织的？列表分为“Awesome Papers”和“Awesome Datasets”两部分。“Awesome Papers”包括指令微调、幻觉、上下文学习、思维链、视觉推理、基础模型、评测和 RLHF；“Awesome Datasets”包括预训练、指令微调、上下文学习、思维链、RLHF 和评测基准。每部分都是一张表格，包含标题、发表平台、日期、代码链接和演示链接。
浏览这个列表需要准备什么吗？不需要。它只是一个在 GitHub 上阅读的 Markdown 文档，无需安装任何东西，也不需要注册账号。
条目经过审核，而且会持续更新吗？条目按时间顺序排列，并标注了发表平台和日期。其中既有经过同行评审的论文，也有 arXiv 预印本和公司博客文章，因此严谨程度因来源而异。列表会持续更新，条目日期已延续到 2026 年。
这个列表适合哪些人？适合从事或研究多模态人工智能的研究人员和工程师，他们可以借此快速追踪最新的 MLLM 论文、数据集和基准测试。
初学者应该从哪里开始？可以先看置顶的三篇综述（MME-Survey、统一多模态理解与生成综述，以及 NSR 2024 关于 MLLM 的综述），对领域有个大致了解，再深入具体的论文分类。
置顶提到的 VITA 系列是什么？VITA 是一个开源的交互式全模态大语言模型系列（包括 VITA、VITA-1.5 和 VITA-Audio），目标是实现接近 GPT-4o 的实时视觉和语音交互。该系列有自己的微信社群。
MME 是什么？为什么在这里很重要？MME 是一个被广泛引用的 MLLM 综合评测基准（NeurIPS 2025 DB Highlight）。列表还会追踪与之相关的 Video-MME 和 Video-MME-v2 基准，用于评估视频理解能力。
这个列表有社群吗？有。README 中附有 MLLM 综述和 VITA 模型系列的微信群链接。
What are similar projects?LLMsPracticalGuide, LLMSurvey, S-Space, AI-Papers-of-the-Week, Deep-Learning-Papers-Reading-Roadmap, GNNPapers, awesome-rsi, Awesome-LLM
Which topics is it in?in-context-learning, chain-of-thought, multimodal-ai, instruction-tuning, papers, academic-research, large-language-models, nlp, computer-vision, benchmark, awesome-list, list
Creator
BradyFU
Size
105.6 MB
Stars
18.1K
Updated
CommitOct 01
Age
3y
Commits · 90 Days