Qwen3-VL README

The project's README from github.com/QwenLM/Qwen3-VL. As one markdown file: README.md.

Qwen3-VL

💜 Qwen Chat   |   🤗 Hugging Face   |   🤖 ModelScope   |   📑 Blog   |   📚 Cookbooks   |   📑 Paper  
🖥️ Demo   |   💬 WeChat (微信)   |   🫨 Discord   |   📑 API   |   🖥️ PAI-DSW

Introduction

Meet Qwen3-VL — the most powerful vision-language model in the Qwen series to date.

This generation delivers comprehensive upgrades across the board: superior text understanding & generation, deeper visual perception & reasoning, extended context length, enhanced spatial and video dynamics comprehension, and stronger agent interaction capabilities.

Available in Dense and MoE architectures that scale from edge to cloud, with Instruct and reasoning‑enhanced Thinking editions for flexible, on‑demand deployment.

Key Enhancements:

Model Architecture Updates:

  1. Interleaved-MRoPE: Full‑frequency allocation over time, width, and height via robust positional embeddings, enhancing long‑horizon video reasoning.

  2. DeepStack: Fuses multi‑level ViT features to capture fine‑grained details and sharpen image–text alignment.

  3. Text–Timestamp Alignment: Moves beyond T‑RoPE to precise, timestamp‑grounded event localization for stronger video temporal modeling.

News

Performance

Visual Tasks

Text-Centric Tasks

Cookbooks

We are preparing cookbooks for many capabilities, including recognition, localization, document parsing, video understanding, key information extraction, and more. Welcome to learn more!

Cookbook Description Open
Omni Recognition Not only identify animals, plants, people, and scenic spots but also recognize various objects such as cars and merchandise. Colab
Powerful Document Parsing Capabilities The parsing of documents has reached a higher level, including not only text but also layout position information and our Qwen HTML format. Colab
Precise Object Grounding Across Formats Using relative position coordinates, it supports both boxes and points, allowing for diverse combinations of positioning and labeling tasks. Colab
General OCR and Key Information Extraction Stronger text recognition capabilities in natural scenes and multiple languages, supporting diverse key information extraction needs. Colab
Video Understanding Better video OCR, long video understanding, and video grounding. Colab
Mobile Agent Locate and think for mobile phone control. Colab
Computer-Use Agent Locate and think for controlling computers and Web. Colab
3D Grounding Provide accurate 3D bounding boxes for both indoor and outdoor objects. Colab
Thinking with Images Utilize image_zoom_in_tool and search_tool to facilitate the model’s precise comprehension of fine-grained visual details within images. Colab
MultiModal Coding Generate accurate code based on rigorous comprehension of multimodal information. Colab
Long Document Understanding Achieve rigorous semantic comprehension of ultra-long documents. Colab
Spatial Understanding See, understand and reason about the spatial information Colab

Quickstart

Below, we provide simple examples to show how to use Qwen3-VL with 🤖 ModelScope and 🤗 Transformers.

# The Qwen3-VL model requires transformers >= 4.57.0
pip install "transformers>=4.57.0"

🤖 ModelScope

We strongly advise users especially those in mainland China to use ModelScope. snapshot_download can help you solve issues concerning downloading checkpoints.

Using 🤗 Transformers to Chat

Here we show a code snippet to show you how to use the chat model with transformers:

from transformers import AutoModelForImageTextToText, AutoProcessor

# default: Load the model on the available device(s)
model = AutoModelForImageTextToText.from_pretrained(
    "Qwen/Qwen3-VL-235B-A22B-Instruct", dtype="auto", device_map="auto"
)

# We recommend enabling flash_attention_2 for better acceleration and memory saving, especially in multi-image and video scenarios.
# model = AutoModelForImageTextToText.from_pretrained(
#     "Qwen/Qwen3-VL-235B-A22B-Instruct",
#     dtype=torch.bfloat16,
#     attn_implementation="flash_attention_2",
#     device_map="auto",
# )

processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-235B-A22B-Instruct")

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

# Preparation for inference
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt"
)
inputs = inputs.to(model.device)

# Inference: Generation of the output
generated_ids = model.generate(**inputs, max_new_tokens=128)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)
Multi image inference 
# Messages containing multiple images and a text query
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "file:///path/to/image1.jpg"},
            {"type": "image", "image": "file:///path/to/image2.jpg"},
            {"type": "text", "text": "Identify the similarities between these images."},
        ],
    }
]

# Preparation for inference
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt"
)
inputs = inputs.to(model.device)

# Inference: Generation of the output
generated_ids = model.generate(**inputs, max_new_tokens=128)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)
Video inference 
# Messages containing a video url(or a local path) and a text query
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "video",
                "video": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen2-VL/space_woaudio.mp4",
            },
            {"type": "text", "text": "Describe this video."},
        ],
    }
]

# Preparation for inference
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt"
)
inputs = inputs.to(model.device)

# Inference: Generation of the output
generated_ids = model.generate(**inputs, max_new_tokens=128)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)
Batch inference 
# for batch generation, padding_side should be set to left!
processor.tokenizer.padding_side = 'left'

# Sample messages for batch inference
messages1 = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "file:///path/to/image1.jpg"},
            {"type": "image", "image": "file:///path/to/image2.jpg"},
            {"type": "text", "text": "What are the common elements in these pictures?"},
        ],
    }
]
messages2 = [
    {"role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}]},
    {"role": "user", "content": [{"type": "text", "text": "Who are you?"}]},
]
# Combine messages for batch processing
messages = [messages1, messages2]

# Preparation for inference
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
    padding=True # padding should be set for batch generation!
)
inputs = inputs.to(model.device)

# Inference: Generation of the output
generated_ids = model.generate(**inputs, max_new_tokens=128)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output

Qwen3-VL (QwenLM/Qwen3-VL)

Searching Database…

20.1K ·GitHub
Qwen3-VL
What is this?Qwen3-VL is a family of multimodal vision-language models built by the Qwen team at Alibaba Cloud, combining strong text understanding with deep visual perception, reasoning, and agent capabilities.
Is this one model or a whole lineup of sizes?It is a lineup spanning Dense and MoE architectures from edge-sized to cloud-scale, each offered in a direct-response Instruct edition and a reasoning-enhanced Thinking edition.
How do I start using it?You load a model with Hugging Face Transformers (AutoProcessor/AutoModel) and chat with it using provided code snippets, or download the checkpoints through ModelScope, which the team recommends especially for users in mainland China.
Are all the model sizes equally easy to run?No — the series ranges from a 2B dense model up to a 235B-A22B MoE flagship, so compute needs differ drastically; quantized AWQ/GPTQ builds exist for some larger checkpoints but still require substantial GPU memory.
Do I need serious GPU hardware to run the bigger models?Yes for the larger checkpoints — the project ships quantized versions (AWQ, GPTQ) to make the heaviest models more feasible, but even those still need multi-GPU or high-VRAM setups for the 70B+ and MoE variants.
What is the difference between the Instruct and Thinking editions?Instruct editions give direct, immediate responses, while Thinking editions add an explicit reasoning process for harder visual and logical tasks, similar to the earlier QvQ-72B-Preview research model focused on visual reasoning.
What does the Visual Agent capability actually let the model do?It lets the model operate PC and mobile GUIs directly: recognizing on-screen elements, understanding their functions, invoking tools, and completing multi-step tasks on its own.
How much context and video length can it handle?Qwen3-VL has a native 256K context window expandable to 1M tokens, letting it process entire books or hours-long video with full recall and second-level event indexing.
How good is its OCR compared to earlier Qwen models?It supports 32 languages, up from 10 previously, and is more robust to low light, blur, tilt, rare or ancient characters, and complex long-document layouts.
What are similar projects?ZDTaichu5.0-9B, MiniCPM, OraRL, Kimi-K3, Valen, Qwen3.8-Flash-Next, airllm, MiniCPM-V
Which topics is it in?video-understanding, alibaba, vision-language-model, edge-ai, reasoning, moe, ocr, document-processing, qwen, jupyter notebook, quantization, multimodal
Creator
QwenLM
Size
134.8 MB
Stars
20.1K
Updated
CommitJan 30
Age
2y
Commits · 90 Days