lift README

The project's README from github.com/datalab-to/lift. As one markdown file: README.md.

Datalab Logo

Datalab

State of the Art models for Document Intelligence

Code License Model License Discord

Homepage Docs Public Playground

lift

lift extracts structured JSON from PDFs and images by passing a schema. It's a 9B vision model that returns a JSON object matching your schema, with schema-constrained decoding guaranteeing valid output.

lift extracting schema-aligned JSON from an invoice Extraction accuracy benchmark

Try lift on Datalab

Our managed platform runs improved extraction with higher accuracy than the open weights, plus per-field verification, citations, and confidence scores.

If you have high volume workloads, we offer a batch processing service that has processed 1B+ pages per week.

Get started with $20 in free credits per month — sign up - takes under 30 seconds - or try lift in our public playground.

Commercial self-hosting requires a license — see Commercial usage. For on-prem licensing, contact us.

Features

Quickstart

The easiest way to start is with the CLI tools:

pip install lift-pdf

# With vLLM (recommended, lightweight install)
lift_vllm
lift_extract input.pdf ./output --schema schema.json

# With HuggingFace (requires torch)
pip install lift-pdf[hf]
lift_extract input.pdf ./output --schema schema.json --method hf

Benchmarks

Evaluated on a 225-document extraction benchmark (6–64 pages per document, ~11,000 scored fields) with adversarial cases planted throughout: cross-page values, exhaustive lists, fields that must be left null, near-miss distractors, multi-source aggregation. Scoring is deterministic exact-match against ground truth (numeric tolerance, normalized strings).

All models receive the same rendered page images, and extract each document in a single pass.

Model Size Field accuracy Full-document accuracy Median latency* Features
Datalab API — 95.9% 44.4% 30.8s Citations + Verification
Gemini Flash 3.5 — 91.3% 40.0% 28.1s
lift 9B 90.2% 20.9% 9.5s
Azure Content Understanding — 83.4% 22.2% 73.7s Citations
NuExtract3 4B 81.5% 8.4% 8.3s
Qwen3.5-9B 9B 76.32% 24.0% 16.8s

* Per document, 8 concurrent requests. Local models (lift, Qwen3.5-9B, NuExtract3) served with vLLM on a single GPU; Gemini, Datalab, and Azure via API. Latency varies with hardware and load - treat as relative, not absolute.

Latency benchmark

Hosted models with verification, citations, and confidence scores are available via the Datalab API - test in the playground.

Installation

Package

# Base install (for vLLM backend)
pip install lift-pdf

# With HuggingFace backend (includes torch, transformers)
pip install lift-pdf[hf]

# With the Schema Studio app
pip install lift-pdf[app]

# With all extras
pip install lift-pdf[all]

If you're using the HuggingFace method, we also recommend installing flash attention for better performance.

From Source

git clone https://github.com/datalab-to/lift.git
cd lift
uv sync
source .venv/bin/activate

Usage

Schemas

A schema is standard JSON Schema. Keep it simple — string, number, integer, boolean, arrays of those, arrays of objects, and nested objects are all supported. Avoid enum, anyOf/oneOf, $ref, and additionalProperties; the schema-constrained decoder skips schemas it can't compile, which weakens the output guarantee.

{
  "type": "object",
  "properties": {
    "invoice_number": {"type": "string", "description": "Invoice identifier"},
    "total": {"type": "number", "description": "Total amount due"},
    "line_items": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "description": {"type": "string"},
          "amount": {"type": "number"}
        }
      }
    }
  },
  "required": ["invoice_number", "total"]
}

Write a description for any field whose name isn't self-explanatory. Mark a field required only when it must appear; fields genuinely absent from a document come back null.

CLI

Process single files or entire directories:

# Single file, with the vLLM server (see below for how to launch it)
lift_extract input.pdf ./output --schema schema.json

# Inline JSON schema
lift_extract scans/ ./output --schema '{"type": "object", "properties": {...}}'

# A schema saved by name in the schemas/ directory, limited to some pages
lift_extract input.pdf ./output --schema invoice --page-range 0-5,8

# Process a whole directory with the local HuggingFace model
lift_extract ./documents ./output --schema schema.json --method hf

CLI Options:

Output Structure:

For each processed file, lift_extract writes to the output directory:

Python

from lift import extract

# schema: a dict, a path to a .json file, an inline JSON string, or a library name
result = extract("document.pdf", "schema.json")
if result.extraction is not None:
    data = result.extraction  # dict matching the schema

Pass a reused model - from lift.model import InferenceManager; model=InferenceManager(method="hf") - to load weights in-process and reuse them across calls, and page_range="0-5" to limit PDF pages. Set VLLM_API_BASE to target a remote server.

Schema Studio

Launch the interactive app to build, save, and test extraction schemas against your documents (requires pip install lift-pdf[app]):

lift_app

vLLM Server

For production deployments or batch processing, launch the vLLM server:

lift_vllm                # defaults to H100 settings
lift_vllm --gpu a100-80  # tune batch settings for your GPU

This launches a Docker container with optimized inference settings, automatically scaling batch size to your GPU's VRAM. Supported GPUs: h100, a100-80, a100/a100-40, l40s, a10, l4, 4090, 3090, t4.

You can also start your own vLLM server with the datalab-to/lift model.

Configuration

Settings can be configured via environment variables or a local.env file:

# Model settings
MODEL_CHECKPOINT=datalab-to/lift
MAX_OUTPUT_TOKENS=12384
TORCH_DEVICE=cuda:0     # pin the HF backend to a device

# vLLM settings
VLLM_API_BASE=http://localhost:8000/v1
VLLM_MODEL_NAME=lift
VLLM_GPUS=0

Commercial usage

This code is Apache 2.0, and our model weights use a modified OpenRAIL-M license (free for research, personal use, and startups under $5M funding/revenue, cannot be used competitively with our API). To remove the OpenRAIL license requirements, or for broader commercial licensing, visit our pricing page here.

Credits

Thank you to the following open source projects:

lift (datalab-to/lift)

Searching Database…

1.2K ·GitHub
Lift
Что это?lift — открытая визуально-языковая модель 9B и набор инструментов, извлекающие структурированные JSON-данные из PDF и изображений по заданной вами схеме, с ограничивающим схемой декодированием, гарантирующим корректный вывод.
Это модель, CLI-инструмент или Python-библиотека?И то, и другое, и третье: визуально-языковая модель 9B, которую можно запускать локально или через vLLM, с CLI для отдельных файлов или каталогов, Python-библиотекой (InferenceManager) для использования в своём процессе и приложением на Streamlit под названием Schema Studio для построения схем.
Как это запустить?Установите пакет через pip (pip install lift-pdf или lift-pdf[app] для Schema Studio) или соберите из исходников, затем запускайте CLI с --schema, указывающим на JSON-файл схемы, и выберите --method hf или vllm для инференса.
Насколько точное извлечение?На бенчмарке авторов из 225 документов lift (9B) показывает точность полей 90,2%, но лишь 20,9% точности по документу целиком (все поля верны) при медианной задержке 9,5 с — быстрее, но менее стабильно идеально, чем хостинговый API Datalab.
Каковы условия лицензии?Код — под Apache 2.0, но веса модели используют изменённую лицензию OpenRAIL-M: бесплатно для исследований, личного использования и стартапов с финансированием/выручкой до 5 млн долларов, но нельзя использовать конкурентно по отношению к собственному API Datalab, а более широкое коммерческое использование на своём хостинге требует отдельной лицензии.
Что такое Schema Studio?Это приложение на Streamlit в комплекте с lift для интерактивного построения, сохранения и тестирования JSON-схем извлечения на собственных документах перед запуском в большом масштабе.
Какая альтернатива самостоятельному хостингу lift?Datalab предлагает управляемый API с более высокой точностью, чем открытые веса, с проверкой по каждому полю, цитатами и оценками уверенности, плюс 20 долларов бесплатных кредитов ежемесячно и публичную песочницу, чтобы попробовать без установки чего-либо.
Какие JSON-схемы поддерживаются?Поддерживается стандартный JSON Schema со строками, числами, булевыми значениями, массивами и вложенными объектами, но enum, anyOf/oneOf, $ref и additionalProperties лучше избегать, так как декодер с ограничением по схеме пропускает схемы, которые не может скомпилировать.
What are similar projects?surya, ZDTaichu5.0-9B, Unlimited-OCR, marker, olmocr, StartLux-Decision, chandra, EvoVLM
Which topics is it in?document-intelligence, vision-language, json-schema, schema, data-extraction, vlm, llm-inference, vllm, ocr, document-processing, evaluation, multimodal
Creator
datalab-to
Size
1.4 MB
Stars
1.2K
Updated
CommitOct 05Releasev0.1.2Oct 05
Age
4mo
Commits · 90 Days