MiniCPM-OCR README

项目的 README,来自 github.com/OpenBMB/MiniCPM-OCR。合并为一个 markdown 文件:README.md。

MiniCPM-OCR: fast, accurate document parsing into text, tables, and formulas, with a lightning arrow connecting documents to structured results; a decision model helps identify regions that need review (illustration).

MiniCPM logo

MiniCPM-OCR 1.0 Guide Hugging Face ModelScope   MiniCPM-OCR Jev Guide Hugging Face ModelScope

Demo (CN / Global)  ·  Get Started  ·  Weights  ·  Free API  ·  Skill  ·  Local Deployment  |  中文
Join the Community via Feishu / Lark  ·  Discord

MiniCPM-OCR is a family of models for efficient document parsing and text recognition. MiniCPM-OCR 1.0 parses Chinese and English documents and PDFs, balancing quality and efficiency across challenging inputs such as scans, photographs, curved pages, and recaptured documents. MiniCPM-OCR Jev evaluates OCR results at the region level, distinguishing problematic regions from reliable ones. It works with different parsing models to help users locate content that needs review.

📢 News

MiniCPM-OCR 1.0

MiniCPM-OCR 1.0 (1.3B) is the first OCR-specific model in the MiniCPM-V/o family. It focuses on text-centric document understanding, covering text, tables, formulas, charts, seals, and structured information extraction. Its key features include:

SOTA Performance, Higher Throughput

Among the open-source OCR models shown below, MiniCPM-OCR 1.0 achieves 1.2 pages/s and the highest overall performance on OmniDocBench v1.6, Real5-OmniDocBench, Wild-OmniDocBench, and olmOCR-Bench.

Throughput and parsing quality of open-source OCR models. The horizontal axis shows throughput in pages per second on a logarithmic scale; the vertical axis shows the equally weighted average score across four document parsing benchmarks.

See the reproduction guide for test settings and reproduction details, and the model guide throughput evaluation for each model's throughput.

Benchmark Comparisons

Radar chart comparing MiniCPM-OCR 1.0 with other OCR models across six benchmarks; some values remain to be confirmed.

View complete MiniCPM-OCR 1.0 evaluation results

Detailed OCR model results on OmniDocBench v1.6, Real5, Wild, olmOCR-Bench, CHAOS-Bench, and OCRBench v2.

See the MiniCPM-OCR 1.0 guide for per-benchmark bar charts and throughput details.

Examples

MiniCPM-OCR 1.0 and Jev promo video. Click to watch the demo on YouTube.

Left to right: original document, detected layout, and rendered Markdown.

English Documents

English document animation: original, detected layout, and rendered Markdown.

More English Examples

Original documents and parsed results, side by side.

English examples comparing inputs and results: handwriting, historical and scanned documents, newspapers, financial statements, and key information extraction.

Chinese Documents

Chinese document animation: original, detected layout, and rendered Markdown.

More Chinese Examples

Original documents and parsed results, side by side.

Chinese examples comparing inputs and results: handwritten notes, complex formulas, newspapers, financial statements, stamped documents, illustrations, and key information extraction.

MiniCPM-OCR Jev

MiniCPM-OCR Jev is an OCR correctness decision model for text, formulas, and tables. Given an image and its corresponding OCR result, it returns a quality score from 0 to 1 to help select reliable results and locate content that needs review.

Evaluation Results

The table below reports overall and per-category AUROC (×100) on JevOCRBench. Higher values indicate better discrimination between correct and incorrect parsing results.

Overall, text, formula, and table AUROC results on JevOCRBench, highlighting MiniCPM-OCR Jev.

Review-Free Accuracy

Accuracy at different review-free rates across three document types.

Accuracy versus review-free rate for legal documents, financial receipts, and complex documents.

Set the threshold only after checking it on a small batch from the real application.

Decision Examples

Jev scores for text, formula, and table recognition.

Original images, two OCR candidates, and their Jev scores for text, formulas, and tables.

See the MiniCPM-OCR Jev model guide for more on usage, technical specifications, evaluation results, and efficiency results.

Using MiniCPM-OCR

Choose how to integrate MiniCPM-OCR 1.0 and MiniCPM-OCR Jev:

How Document Parsing Works

Full-page parsing runs as a two-stage pipeline orchestrated by the minicpm-ocr SDK. The two stages use separate models, run in separate environments, and may run on different machines:

flowchart LR
    DOC["PDF / image"] --> S1

    subgraph SDKENV ["minicpm-ocr SDK (its own environment)"]
        S1["Stage 1 · Layout detection<br>PP-DocLayoutV3 layout model (PaddlePaddle / PaddleX)<br>regions, reading order, cropping"]
        ASM["Assemble output<br>Markdown / JSON in reading order"]
    end

    subgraph SRVENV ["Recognition service (separate environment)"]
        S2["Stage 2 · Region recognition<br>MiniCPM-OCR 1.0<br>vLLM / SGLang / llama.cpp / Ollama"]
    end

    S1 -- "each cropped region, over HTTP,<br>with a prompt for its region type" --> S2
    S2 -- "text, tables, formulas, charts, seals" --> ASM

A full local deployment therefore prepares two environments side by side: the SDK environment (pip package minicpm-ocr, plus PaddlePaddle / PaddleX for the layout model) and the inference-service environment of the chosen backend. The Skill walks through the complete setup — backend choice, environment installation, model downloads, and verification — and the SDK tutorial covers the same steps for manual deployment. For region prompts and per-stage details, see the document parsing workflow.

Skill

Ask a coding agent to follow the MiniCPM-OCR 1.0 skill to set up the environment, download models, and parse your first document. Send the following instruction to the agent:

Read and execute https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md

Ask a coding agent to follow the MiniCPM-OCR Jev skill to deploy, start, and verify the scoring service locally — either from the Docker image or with the SDK, using vLLM, SGLang, llama.cpp, or Ollama as the inference backend. Send the following instruction to the agent:

Read and execute https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_jev_en.md

If you have already cloned this repository, you can use ./docs/SKILL_en.md and ./docs/SKILL_jev_en.md directly.

Docker Deployment

Deploy MiniCPM-OCR 1.0 with the official Docker image, which bundles model weights, the SDK, vLLM, and the document parsing backend. The Docker Hub page provides startup instructions for both the OCR model API and the end-to-end document parsing API, supporting image and PDF parsing into Markdown / JSON.

Requires Linux x86-64, an NVIDIA GPU, a driver compatible with CUDA 12.9, and Docker configured with NVIDIA Container Toolkit. Follow the image documentation to select and start the service you need.

Free API

Use the Free API to access MiniCPM-OCR 1.0 recognition and MiniCPM-OCR Jev quality scoring.

MiniCPM-OCR 1.0 API docs  ·  MiniCPM-OCR Jev API docs

MiniCPM-V Demo — iOS, Android & HarmonyOS

MiniCPM-OCR 1.0 parses documents entirely on your device, even offline. This document recognition feature is currently available on iOS only.

On-device demo tutorial

Local Deployment

Run the models on your own hardware: MiniCPM-OCR 1.0 parses PDFs and images into Markdown or JSON, while MiniCPM-OCR Jev scores the parsing results.

MiniCPM-OCR 1.0 deployment tutorial · MiniCPM-OCR Jev deployment tutorial · Web demo deployment tutorial

Fine-Tuning

Model Fine-tuning tutorial
MiniCPM-OCR 1.0 LLaMA-Factory · Swift
MiniCPM-OCR Jev Swift

Model Weights

Model Description Main Model Weights DSpark Heads
MiniCPM‑OCR 1.0 Efficient document parsing and text recognition for Chinese and English documents and PDFs, balancing quality and speed across scans, photographs, curved pages, and recaptured documents. 🤗  ModelScope BF16
🤗  ModelScope GGUF
🤗  ModelScope AWQ
🤗  ModelScope GPTQ
🤗  ModelScope BF16
🤗  ModelScope GGUF
MiniCPM‑OCR Jev An OCR correctness decision model for text, formulas, and tables. Given an image and its OCR result, it returns a quality score from 0 to 1. 🤗  ModelScope BF16 🤗  ModelScope GGUF
🤗  ModelScope AWQ 🤗  ModelScope GPTQ

💡 FAQ and Troubleshooting

See the FAQ for common questions about installation, model downloads, GPU memory, input formats, and unexpected outputs.

📄 License

The entire project is released under the Apache License 2.0, including code, documentation, and model weights for MiniCPM-OCR 1.0 and MiniCPM-OCR Jev.

MiniCPM-OCR (OpenBMB/MiniCPM-OCR)

正在搜索数据库…

这是什么?MiniCPM-OCR 是一组用于文档理解的模型:MiniCPM-OCR 1.0 可将中英文文档和 PDF 解析为结构化文本(包括表格、公式、图表和印章),而 MiniCPM-OCR Jev 是一个独立模型,用于为这些解析结果打 0 到 1 的分,标记出需要人工复核的区域。
这是一个模型还是两个独立的东西?是两个相互配合的独立模型:MiniCPM-OCR 1.0 负责实际的文档解析和识别,而 MiniCPM-OCR Jev 是一个独立的正确性校验模型,会按区域审查第一个模型的输出。
我该如何开始使用?有三种方式:直接调用免费 API 进行识别和质量评分;让编程智能体按照随附的 SKILL.md 文件配置环境并自动下载模型;或者在自己的硬件上进行本地部署。
运行它需要哪些推理后端?开箱即可在 vLLM、SGLang、llama.cpp 或 Ollama 上运行,支持量化权重(BF16、GGUF、AWQ、GPTQ),并提供 pip SDK 和完整的 API,便于集成到你自己的应用中。
开箱即用的 Jev 质量评分可以直接信任吗?不能盲目信任。README 建议先在你自己的一批文档样本上调整“通过/复核”阈值,因为不同免复核比例下的准确率会因文档类型而异。
这些模型有多大?MiniCPM-OCR 1.0 有 13 亿(1.3B)参数,MiniCPM-OCR Jev 有 20 亿(2B)参数。与许多 OCR/VLM 替代方案相比体量很小,同时仍达到了业界领先(state-of-the-art)的分数。
它的速度与其他 OCR 工具相比如何?启用 DSpark 解码加速后,MiniCPM-OCR 1.0 的吞吐量约为 PaddleOCR-VL-1.6 的 1.49 倍,是 MinerU2.5 Pro 的 3.62 倍,运行速度约为每秒 1.2 页。
我可以用自己的数据对这些模型进行微调吗?可以,两个模型都有分别基于 LLaMA-Factory 和 Swift 的专门微调教程。
除了纯文本,它还能提取什么?它能识别表格、公式、图表和印章,并支持关键信息抽取(KIE),可从文档中提取结构化数据。
作者
OpenBMB
大小
65.7 MB
星标
977
时长
22天
提交 · 90 天