What is this?
A family of pocket-sized multimodal AI models from OpenBMB (MiniCPM-V and MiniCPM-o) built to run on local devices, even phones — understanding images, video and text, and in the o-series also speech.
A family of pocket-sized multimodal AI models from OpenBMB (MiniCPM-V and MiniCPM-o) built to run on local devices, even phones — understanding images, video and text, and in the o-series also speech.Is it an app, a model, or a service?Open-weight models plus Python inference code rather than a consumer app: you run them via Ollama, llama.cpp, vLLM or Transformers, or call the hosted API. An open-sourced iOS app and a realtime web demo also exist.
Open-weight models plus Python inference code rather than a consumer app: you run them via Ollama, llama.cpp, vLLM or Transformers, or call the hosted API. An open-sourced iOS app and a realtime web demo also exist.How do I get it running?Quickest: pull MiniCPM-V 4.6 from Ollama's official model library. Otherwise load it with Transformers, run GGUF builds in llama.cpp, or use the API; the Cookbook and Docs Site cover common scenarios.
Quickest: pull MiniCPM-V 4.6 from Ollama's official model library. Otherwise load it with Transformers, run GGUF builds in llama.cpp, or use the API; the Cookbook and Docs Site cover common scenarios.Are there any setup gotchas I should know about?Video decoding via torchcodec conflicts with some CUDA versions (torch>=2.11 bundles CUDA 13.1); swap in PyAV or pin your CUDA version. The realtime web demo can also lag on poor networks.
Video decoding via torchcodec conflicts with some CUDA versions (torch>=2.11 bundles CUDA 13.1); swap in PyAV or pin your CUDA version. The realtime web demo can also lag on poor networks.Which devices and systems does it actually run on?All three mainstream mobile OSes — iOS, Android and HarmonyOS — with the edge adaptation code open-sourced, plus an open-sourced iOS app for iPhone and iPad. On desktops it runs through Ollama or llama.cpp.
All three mainstream mobile OSes — iOS, Android and HarmonyOS — with the edge adaptation code open-sourced, plus an open-sourced iOS app for iPhone and iPad. On desktops it runs through Ollama or llama.cpp.If it runs on my device, does my data stay there?Yes — deployment is on-device, so the images, video and audio you feed it stay on your phone or computer. A hosted API service exists as the alternative if you don't want to run anything locally.
Yes — deployment is on-device, so the images, video and audio you feed it stay on your phone or computer. A hosted API service exists as the alternative if you don't want to run anything locally.Can my laptop or phone handle this, or do I need a big GPU?Not a big one for MiniCPM-V 4.6: at 1.3B parameters it targets phones and edge devices, and fine-tuning works on consumer-grade GPUs. The 9B MiniCPM-o 4.5 is heavier, and the realtime demo needs a Mac or a GPU.
Not a big one for MiniCPM-V 4.6: at 1.3B parameters it targets phones and edge devices, and fine-tuning works on consumer-grade GPUs. The 9B MiniCPM-o 4.5 is heavier, and the realtime demo needs a Mac or a GPU.How is MiniCPM-o different from MiniCPM-V?MiniCPM-V handles efficient image, video and text understanding. MiniCPM-o goes further: real-time streaming of video and audio in, text and speech out, with full-duplex live conversation — it can see, listen and speak at the same time.
MiniCPM-V handles efficient image, video and text understanding. MiniCPM-o goes further: real-time streaming of video and audio in, text and speech out, with full-duplex live conversation — it can see, listen and speak at the same time.How good can a 1.3B model actually be?MiniCPM-V 4.6 scores 13 on the Artificial Analysis Intelligence Index vs Qwen3.5-0.8B's 10, with far fewer tokens per answer, and it surpasses the larger Ministral 3 3B. MiniCPM-o 4.5 approaches Gemini 2.5 Flash in vision and speech.
MiniCPM-V 4.6 scores 13 on the Artificial Analysis Intelligence Index vs Qwen3.5-0.8B's 10, with far fewer tokens per answer, and it surpasses the larger Ministral 3 3B. MiniCPM-o 4.5 approaches Gemini 2.5 Flash in vision and speech.What does full-duplex mean in practice?Output streams (speech and text) and real-time input streams (video and audio) do not block each other, so the model can watch, listen and talk simultaneously and act proactively, for example by reminding you of something unprompted.
Output streams (speech and text) and real-time input streams (video and audio) do not block each other, so the model can watch, listen and talk simultaneously and act proactively, for example by reminding you of something unprompted.Can I fine-tune it on my own data?Yes — it plugs into fine-tuning ecosystems like SWIFT and LLaMA-Factory, and Align-Anything supports DPO and SFT on both vision and audio. Quantized variants ship in GGUF, BNB, AWQ and GPTQ formats.
Yes — it plugs into fine-tuning ecosystems like SWIFT and LLaMA-Factory, and Align-Anything supports DPO and SFT on both vision and audio. Quantized variants ship in GGUF, BNB, AWQ and GPTQ formats.Which topics is it in?harmonyos, video-understanding, llama-cpp, on-device, edge-ai, vision-language-model, gguf, transformers, vllm, text-to-speech, quantization, fine-tuning
harmonyos, video-understanding, llama-cpp, on-device, edge-ai, vision-language-model, gguf, transformers, vllm, text-to-speech, quantization, fine-tuning