ncnn README

The project's README from github.com/Tencent/ncnn. As one markdown file: README.md.

ncnn

ncnn

License Download Total Count codecov

ncnn is a high-performance neural network inference framework optimized for mobile, embedded, and desktop deployment. It has no third-party runtime dependencies, runs across CPU and Vulkan GPU backends, and provides tools such as pnnx for converting PyTorch and ONNX models to ncnn. Developers can deploy deep learning models efficiently on phones, PCs, browsers, and edge devices. ncnn is currently being used in many Tencent applications, such as QQ, Qzone, WeChat, Pitu, and so on.

ncnn 是一个面向移动端、嵌入式和桌面端部署优化的高性能神经网络推理框架。 ncnn 无第三方运行时依赖，支持 CPU 和 Vulkan GPU 后端，并提供 pnnx 等工具将 PyTorch 和 ONNX 模型转换为 ncnn 模型。 基于 ncnn，开发者可以将深度学习模型高效部署到手机、PC、浏览器和边缘设备上。 ncnn 目前已在腾讯多款应用中使用，如：QQ，Qzone，微信，天天 P 图等。

Quick Start

The recommended beginner path is PyTorch -> pnnx -> ncnn.

Install pnnx in a PyTorch environment

pip3 install pnnx

Export a PyTorch model to ncnn

import torch
import torch.nn as nn
import pnnx

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d(3, 8, 1)
        self.relu = nn.ReLU()
        self.fc = nn.Linear(8, 4)

    def forward(self, x):
        x = self.conv(x)
        x = self.relu(x)
        x = x.mean((2, 3))
        return self.fc(x)

model = Model().eval()

x = torch.rand(1, 3, 224, 224)
pnnx.export(model, "model.pt", (x,))

This generates model.ncnn.param and model.ncnn.bin.

Run with ncnn C++ API

#include "net.h"

ncnn::Net net;
net.load_param("model.ncnn.param");
net.load_model("model.ncnn.bin");

ncnn::Mat in(224, 224, 3);

auto ex = net.create_extractor();
ex.input("in0", in);

ncnn::Mat out;
ex.extract("out0", out);

Or use Python

import numpy as np
import ncnn

net = ncnn.Net()
net.load_param("model.ncnn.param")
net.load_model("model.ncnn.bin")

x = np.zeros((3, 224, 224), np.float32)
mat = ncnn.Mat(x)

ex = net.create_extractor()
ex.input("in0", mat)

ret, out = ex.extract("out0")
print(np.array(out).shape)

See pnnx, use ncnn with PyTorch or ONNX, Python API, and examples for complete workflows.

Community

技术交流 QQ 群
637093648 (超多大佬)
答案：卷卷卷卷卷（已满） 		Telegram Group

https://t.me/ncnnyes

 Discord Channel

https://discord.gg/YRsxgmF
Pocky QQ 群（MLIR YES!）
677104663 (超多大佬)
答案：multi-level intermediate representation
他们都不知道 pnnx 有多好用群
818998520 (新群！)

Download & Build status

https://github.com/Tencent/ncnn/releases/latest

how to build ncnn library on Linux / Windows / macOS / Raspberry Pi3, Pi4 / POWER / Android / NVIDIA Jetson / iOS / WebAssembly / AllWinner D1 / Loongson 2K1000
Source

Android

Android shared

HarmonyOS

HarmonyOS shared

iOS

iOS-Simulator

macOS

Mac-Catalyst

watchOS

watchOS-Simulator

tvOS

tvOS-Simulator

visionOS

visionOS-Simulator

Apple xcframework

Ubuntu 22.04

Ubuntu 24.04

windows
VS2015

VS2017

VS2019

VS2022

WebAssembly

Linux (arm)

Linux (aarch64)

Linux (mips)

Linux (mips64)

Linux (ppc64)

Linux (riscv64)

Linux (loongarch64)

Build

Use the prebuilt packages above when possible. To build from source, see the full how to build ncnn library guide for Linux, Windows, macOS, Android, iOS, WebAssembly, HarmonyOS, Raspberry Pi, Jetson, and embedded targets.

Common Linux build:

git clone --recursive https://github.com/Tencent/ncnn.git
cd ncnn
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DNCNN_VULKAN=ON -DNCNN_BUILD_EXAMPLES=ON ..
cmake --build . -j$(nproc)

Model Conversion

Source model Recommended path Docs
PyTorch pnnx.export(model, "model.pt", (input_tensor,)) or pnnx model.pt inputshape=[...] pnnx, PyTorch / ONNX guide
ONNX pnnx model.onnx pnnx, onnx tools
ncnn model optimization ncnnoptimize model.param model.bin new.param new.bin flag quantization, model file spec
Legacy Caffe / MXNet / Darknet Use compatibility converters when maintaining older models caffe, mxnet, darknet, AlexNet legacy tutorial

Use Netron to inspect .param, .onnx, and .pnnx.param graphs.

Features

Model and Workload Coverage

ncnn is still strong for classic and mobile CNN workloads, but current usage is broader than CNN-only deployment.

For operator-level detail, see supported PyTorch operator status, supported ONNX operator status, and operation param weight table.

Project Examples

Area Project
Image generation zimage-ncnn-vulkan - Z-Image generation with ncnn and Vulkan
LLM / embedding / vision-language ncnn_llm - LLM, embedding, and vision-language examples with ncnn
Android classification ncnn-android-squeezenet
Android style transfer ncnn-android-styletransfer
Android detection ncnn-android-mobilenetssd, ncnn-android-yolov5, ncnn-android-yolov7, ncnn-android-scrfd
Face detection mtcnn_ncnn
Qt / Android integration qt_android_ncnn_lib_encrypt_example
Colorization ncnn-colorization-siggraph17
Fortran binding ncnn-fortran
Speech recognition sherpa - real-time speech recognition on embedded and mobile devices

Documentation And FAQ

Topic Links
Build how to build
PyTorch / ONNX conversion use ncnn with PyTorch or ONNX, pnnx, PyTorch converter notes
API and examples C++ examples, Python API, low-level operation API
Model format param and model file spec, operation param weight table
Extension custom layer guide, plugin tools
FAQ deepwiki, throw error, wrong result, Vulkan
Legacy beginner material use ncnn with AlexNet, AlexNet Chinese tutorial

License

BSD 3 Clause

ncnn (Tencent/ncnn)

Searching Database…

24K ·GitHub
Ncnn
这是什么？ncnn 是由 Tencent 开发的高性能神经网络推理框架，针对移动端、嵌入式和桌面端部署进行了优化。它被用于 QQ、Qzone 和 WeChat 等腾讯应用中，在设备端运行深度学习模型。
这是训练框架，还是只用来运行模型？它纯粹是推理引擎，不是训练框架。你可以在别处（比如 PyTorch）训练模型，再把它转换成 ncnn 的格式，从而在设备端高效运行。
我该怎么用 ncnn 跑起模型？推荐的路径是 PyTorch -> pnnx -> ncnn：先在 PyTorch 环境中安装 pnnx，导出模型以生成 model.ncnn.param 和 model.ncnn.bin，然后使用 ncnn 的 C++ API 或 Python 绑定加载并运行。
我需要先安装 BLAS 或 NNPACK 吗？不需要。ncnn 没有第三方运行时依赖，也不需要 BLAS 或 NNPACK；它是纯 C++ 实现，提供 C API 和 Python 绑定。
从源码构建时有什么需要注意的坑吗？从源码构建需要针对每个目标平台（Linux、Windows、macOS、Android、iOS、WebAssembly、HarmonyOS、Raspberry Pi、Jetson 等）分别做平台相关的配置，各平台的说明单独成文。因此能用预编译包时，建议优先使用预编译包。
ncnn 实际能在哪些平台上运行？它支持 Linux、Windows、macOS、Android、iOS（以及 Apple Catalyst/watchOS/tvOS/visionOS）、WebAssembly 和 HarmonyOS，还支持 Raspberry Pi、NVIDIA Jetson、RISC-V、AllWinner D1 等嵌入式平台。
它是用 GPU 运行，还是只用 CPU？ncnn 在 CPU 上运行，并使用 ARM NEON、多核调度等优化手段；在受支持的平台上，它还支持通过 Vulkan 进行 GPU 加速。
ncnn 能从哪些模型格式转换？通过 pnnx 可以直接转换 PyTorch 和 ONNX 模型；此外还提供针对 Caffe、MXNet 和 Darknet 模型的旧版转换器。
ncnn 开箱即用支持哪些类型的模型？支持的模型包括：分类骨干网络（VGG、ResNet、MobileNet、ShuffleNet）；检测和人脸模型（SSD、YOLOv2-v8、YOLOX、RetinaFace、MTCNN）；分割、姿态和 OCR 模型（UNet、YOLACT、SimplePose、PP-OCR）。此外还支持 int8 量化模型和 fp16 推理。
为什么不直接用 PyTorch Mobile 这类通用框架，而选择 ncnn？ncnn 没有第三方运行时依赖，通过显式的 blob/workspace 分配器设计，内存占用较小；并且专门针对移动端和嵌入式 CPU 以及 Vulkan GPU 做了优化，使部署在手机和边缘设备上更加轻量、快速。
能直观地查看转换后的 ncnn 模型吗？可以，你可以用 Netron 查看 .param、.onnx 和 .pnnx.param 等图结构文件。
What are similar projects?MNN, YOLOX, transformers.js, onnxruntime, deeplearning4j, yolov7, openvino, DeepLearning
Which topics is it in?darknet, riscv, ncnn, mxnet, simd, keras, edge-ai, object-detection, onnx, tensorflow, vulkan, raspberry-pi
Creator
Tencent
Size
36.8 MB
Stars
24K
Updated
CommitOct 14Release20260526May 26
Age
9y
Commits · 90 Days