ncnn README

The project's README from github.com/Tencent/ncnn. As one markdown file: README.md.

ncnn

ncnn

License Download Total Count codecov

ncnn is a high-performance neural network inference framework optimized for mobile, embedded, and desktop deployment. It has no third-party runtime dependencies, runs across CPU and Vulkan GPU backends, and provides tools such as pnnx for converting PyTorch and ONNX models to ncnn. Developers can deploy deep learning models efficiently on phones, PCs, browsers, and edge devices. ncnn is currently being used in many Tencent applications, such as QQ, Qzone, WeChat, Pitu, and so on.

ncnn 是一个面向移动端、嵌入式和桌面端部署优化的高性能神经网络推理框架。 ncnn 无第三方运行时依赖，支持 CPU 和 Vulkan GPU 后端，并提供 pnnx 等工具将 PyTorch 和 ONNX 模型转换为 ncnn 模型。 基于 ncnn，开发者可以将深度学习模型高效部署到手机、PC、浏览器和边缘设备上。 ncnn 目前已在腾讯多款应用中使用，如：QQ，Qzone，微信，天天 P 图等。

Quick Start

The recommended beginner path is PyTorch -> pnnx -> ncnn.

Install pnnx in a PyTorch environment

pip3 install pnnx

Export a PyTorch model to ncnn

import torch
import torch.nn as nn
import pnnx

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d(3, 8, 1)
        self.relu = nn.ReLU()
        self.fc = nn.Linear(8, 4)

    def forward(self, x):
        x = self.conv(x)
        x = self.relu(x)
        x = x.mean((2, 3))
        return self.fc(x)

model = Model().eval()

x = torch.rand(1, 3, 224, 224)
pnnx.export(model, "model.pt", (x,))

This generates model.ncnn.param and model.ncnn.bin.

Run with ncnn C++ API

#include "net.h"

ncnn::Net net;
net.load_param("model.ncnn.param");
net.load_model("model.ncnn.bin");

ncnn::Mat in(224, 224, 3);

auto ex = net.create_extractor();
ex.input("in0", in);

ncnn::Mat out;
ex.extract("out0", out);

Or use Python

import numpy as np
import ncnn

net = ncnn.Net()
net.load_param("model.ncnn.param")
net.load_model("model.ncnn.bin")

x = np.zeros((3, 224, 224), np.float32)
mat = ncnn.Mat(x)

ex = net.create_extractor()
ex.input("in0", mat)

ret, out = ex.extract("out0")
print(np.array(out).shape)

See pnnx, use ncnn with PyTorch or ONNX, Python API, and examples for complete workflows.

Community

技术交流 QQ 群
637093648 (超多大佬)
答案：卷卷卷卷卷（已满） 		Telegram Group

https://t.me/ncnnyes

 Discord Channel

https://discord.gg/YRsxgmF
Pocky QQ 群（MLIR YES!）
677104663 (超多大佬)
答案：multi-level intermediate representation
他们都不知道 pnnx 有多好用群
818998520 (新群！)

Download & Build status

https://github.com/Tencent/ncnn/releases/latest

how to build ncnn library on Linux / Windows / macOS / Raspberry Pi3, Pi4 / POWER / Android / NVIDIA Jetson / iOS / WebAssembly / AllWinner D1 / Loongson 2K1000
Source

Android

Android shared

HarmonyOS

HarmonyOS shared

iOS

iOS-Simulator

macOS

Mac-Catalyst

watchOS

watchOS-Simulator

tvOS

tvOS-Simulator

visionOS

visionOS-Simulator

Apple xcframework

Ubuntu 22.04

Ubuntu 24.04

windows
VS2015

VS2017

VS2019

VS2022

WebAssembly

Linux (arm)

Linux (aarch64)

Linux (mips)

Linux (mips64)

Linux (ppc64)

Linux (riscv64)

Linux (loongarch64)

Build

Use the prebuilt packages above when possible. To build from source, see the full how to build ncnn library guide for Linux, Windows, macOS, Android, iOS, WebAssembly, HarmonyOS, Raspberry Pi, Jetson, and embedded targets.

Common Linux build:

git clone --recursive https://github.com/Tencent/ncnn.git
cd ncnn
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DNCNN_VULKAN=ON -DNCNN_BUILD_EXAMPLES=ON ..
cmake --build . -j$(nproc)

Model Conversion

Source model Recommended path Docs
PyTorch pnnx.export(model, "model.pt", (input_tensor,)) or pnnx model.pt inputshape=[...] pnnx, PyTorch / ONNX guide
ONNX pnnx model.onnx pnnx, onnx tools
ncnn model optimization ncnnoptimize model.param model.bin new.param new.bin flag quantization, model file spec
Legacy Caffe / MXNet / Darknet Use compatibility converters when maintaining older models caffe, mxnet, darknet, AlexNet legacy tutorial

Use Netron to inspect .param, .onnx, and .pnnx.param graphs.

Features

Model and Workload Coverage

ncnn is still strong for classic and mobile CNN workloads, but current usage is broader than CNN-only deployment.

For operator-level detail, see supported PyTorch operator status, supported ONNX operator status, and operation param weight table.

Project Examples

Area Project
Image generation zimage-ncnn-vulkan - Z-Image generation with ncnn and Vulkan
LLM / embedding / vision-language ncnn_llm - LLM, embedding, and vision-language examples with ncnn
Android classification ncnn-android-squeezenet
Android style transfer ncnn-android-styletransfer
Android detection ncnn-android-mobilenetssd, ncnn-android-yolov5, ncnn-android-yolov7, ncnn-android-scrfd
Face detection mtcnn_ncnn
Qt / Android integration qt_android_ncnn_lib_encrypt_example
Colorization ncnn-colorization-siggraph17
Fortran binding ncnn-fortran
Speech recognition sherpa - real-time speech recognition on embedded and mobile devices

Documentation And FAQ

Topic Links
Build how to build
PyTorch / ONNX conversion use ncnn with PyTorch or ONNX, pnnx, PyTorch converter notes
API and examples C++ examples, Python API, low-level operation API
Model format param and model file spec, operation param weight table
Extension custom layer guide, plugin tools
FAQ deepwiki, throw error, wrong result, Vulkan
Legacy beginner material use ncnn with AlexNet, AlexNet Chinese tutorial

License

BSD 3 Clause

ncnn (Tencent/ncnn)

Searching Database…

24K ·GitHub
Ncnn
What is this?ncnn is a high-performance neural network inference framework built by Tencent and optimized for mobile, embedded, and desktop deployment. It is used in Tencent apps like QQ, Qzone, and WeChat to run deep learning models on-device.
Is this a training framework or just for running models?It's purely an inference engine, not a training framework. You train your model elsewhere (PyTorch, etc.) and convert it to ncnn's format to run it efficiently on-device.
How do I get a model running with ncnn?The recommended path is PyTorch -> pnnx -> ncnn: install pnnx in your PyTorch environment, export the model to generate model.ncnn.param and model.ncnn.bin, then load and run it with the ncnn C++ API or Python bindings.
Do I need BLAS or NNPACK installed first?No, ncnn has no third-party runtime dependencies and doesn't require BLAS or NNPACK; it's a pure C++ implementation with a C API and Python bindings.
Any pitfalls when building from source?Building from source requires platform-specific setup documented separately for each target (Linux, Windows, macOS, Android, iOS, WebAssembly, HarmonyOS, Raspberry Pi, Jetson, and more), so prebuilt packages are recommended when possible.
Which platforms does ncnn actually run on?It supports Linux, Windows, macOS, Android, iOS (and Apple Catalyst/watchOS/tvOS/visionOS), WebAssembly, HarmonyOS, plus embedded targets like Raspberry Pi, NVIDIA Jetson, RISC-V, and AllWinner D1.
Does it use the GPU or only the CPU?ncnn runs on CPU with optimizations like ARM NEON and multi-core scheduling, and also supports Vulkan for GPU acceleration on supported platforms.
What model formats can ncnn convert from?Through pnnx it converts PyTorch and ONNX models directly, and it also has legacy converters for Caffe, MXNet, and Darknet models.
What kinds of models does ncnn support out of the box?Classification backbones (VGG, ResNet, MobileNet, ShuffleNet), detection/face models (SSD, YOLOv2-v8, YOLOX, RetinaFace, MTCNN), and segmentation/pose/OCR models (UNet, YOLACT, SimplePose, PP-OCR), plus int8 quantized and fp16 inference.
Why would I use ncnn instead of a general framework like PyTorch Mobile?ncnn has zero third-party runtime dependencies, a small memory footprint via its explicit blob/workspace allocator design, and is specifically optimized for mobile/embedded CPUs and Vulkan GPUs, which keeps deployments lightweight and fast on phones and edge devices.
Can I inspect a converted ncnn model visually?Yes, you can use Netron to inspect .param, .onnx, and .pnnx.param graph files.
What are similar projects?MNN, YOLOX, transformers.js, onnxruntime, deeplearning4j, yolov7, openvino, DeepLearning
Which topics is it in?darknet, riscv, ncnn, mxnet, simd, keras, edge-ai, object-detection, onnx, tensorflow, vulkan, raspberry-pi
Creator
Tencent
Size
36.9 MB
Stars
24K
Updated
CommitOct 17Release20260526May 26
Age
9y
Commits · 90 Days