What is this?
A curated, continuously updated list of papers, models, datasets, and benchmarks tracking the latest advances in Multimodal Large Language Models (MLLMs) — models like GPT-4o, Gemini, and VITA that understand images, video, and audio alongside text.
A curated, continuously updated list of papers, models, datasets, and benchmarks tracking the latest advances in Multimodal Large Language Models (MLLMs) — models like GPT-4o, Gemini, and VITA that understand images, video, and audio alongside text.Is this a paper or a running tool?It is a reference list (README), not software — just organized links to papers, projects, code repos, and demos, grouped by topic such as instruction tuning, in-context learning, chain-of-thought, and benchmarks.
It is a reference list (README), not software — just organized links to papers, projects, code repos, and demos, grouped by topic such as instruction tuning, in-context learning, chain-of-thought, and benchmarks.How is the list organized?It's split into 'Awesome Papers' (instruction tuning, hallucination, in-context learning, chain-of-thought, visual reasoning, foundation models, evaluation, RLHF) and 'Awesome Datasets' (pre-training, instruction tuning, in-context learning, chain-of-thought, RLHF, and evaluation benchmarks), each as a table with title, venue, date, code, and demo links.
It's split into 'Awesome Papers' (instruction tuning, hallucination, in-context learning, chain-of-thought, visual reasoning, foundation models, evaluation, RLHF) and 'Awesome Datasets' (pre-training, instruction tuning, in-context learning, chain-of-thought, RLHF, and evaluation benchmarks), each as a table with title, venue, date, code, and demo links.Do I need anything to browse this list?No, it's just a Markdown document you read on GitHub; no installation or accounts are needed.
No, it's just a Markdown document you read on GitHub; no installation or accounts are needed.Are entries vetted and still updated?Entries are listed chronologically with venue and date, mixing peer-reviewed papers, arXiv preprints, and company blog posts, so rigor varies by source; the list is actively updated, with entries dated into 2026.
Entries are listed chronologically with venue and date, mixing peer-reviewed papers, arXiv preprints, and company blog posts, so rigor varies by source; the list is actively updated, with entries dated into 2026.Who is this list for?Researchers and engineers working on or studying multimodal AI who want a fast way to track the newest MLLM papers, datasets, and benchmarks.
Researchers and engineers working on or studying multimodal AI who want a fast way to track the newest MLLM papers, datasets, and benchmarks.Where should a beginner start?Start with the three highlighted surveys at the top (MME-Survey, Unified Multimodal Understanding and Generation survey, and the NSR 2024 survey on MLLMs) for a broad overview before diving into specific paper categories.
Start with the three highlighted surveys at the top (MME-Survey, Unified Multimodal Understanding and Generation survey, and the NSR 2024 survey on MLLMs) for a broad overview before diving into specific paper categories.What is the VITA series mentioned at the top?VITA is an open-source interactive omni-modal LLM series (VITA, VITA-1.5, VITA-Audio) aimed at real-time vision and speech interaction comparable to GPT-4o, maintained with its own WeChat community.
VITA is an open-source interactive omni-modal LLM series (VITA, VITA-1.5, VITA-Audio) aimed at real-time vision and speech interaction comparable to GPT-4o, maintained with its own WeChat community.What is MME and why does it matter here?MME is a widely cited comprehensive evaluation benchmark for MLLMs (NeurIPS 2025 DB Highlight), and the list tracks its related Video-MME and Video-MME-v2 benchmarks for video understanding evaluation.
MME is a widely cited comprehensive evaluation benchmark for MLLMs (NeurIPS 2025 DB Highlight), and the list tracks its related Video-MME and Video-MME-v2 benchmarks for video understanding evaluation.Is there a community around this list?Yes, the README links to dedicated WeChat groups for MLLM surveys and for the VITA model series.
Yes, the README links to dedicated WeChat groups for MLLM surveys and for the VITA model series.What are similar projects?LLMsPracticalGuide, LLMSurvey, S-Space, AI-Papers-of-the-Week, Deep-Learning-Papers-Reading-Roadmap, GNNPapers, Awesome-LLM, awesome-rsi
LLMsPracticalGuide, LLMSurvey, S-Space, AI-Papers-of-the-Week, Deep-Learning-Papers-Reading-Roadmap, GNNPapers, Awesome-LLM, awesome-rsiWhich topics is it in?in-context-learning, chain-of-thought, multimodal-ai, instruction-tuning, papers, nlp, large-language-models, academic-research, computer-vision, benchmark, awesome-list, list
in-context-learning, chain-of-thought, multimodal-ai, instruction-tuning, papers, nlp, large-language-models, academic-research, computer-vision, benchmark, awesome-list, list