What is this?
PaddleNLP is a large language model (LLM) and small language model (SLM) development toolkit built on the PaddlePaddle deep learning framework, offering efficient training, lossless compression, and high-performance inference for large models.
PaddleNLP is a large language model (LLM) and small language model (SLM) development toolkit built on the PaddlePaddle deep learning framework, offering efficient training, lossless compression, and high-performance inference for large models.Is this a standalone app, or a library I import into my training code?It's a Python toolkit/framework you install and import into your own scripts to train, fine-tune, compress, and serve LLMs — not a standalone application.
It's a Python toolkit/framework you install and import into your own scripts to train, fine-tune, compress, and serve LLMs — not a standalone application.How do I start training or fine-tuning a model with it?The README points to dedicated Installation and Quick Start sections covering environment setup for your target hardware, after which you configure the Trainer with your chosen distributed strategy (data/tensor/pipeline parallel) to begin pretraining or fine-tuning.
The README points to dedicated Installation and Quick Start sections covering environment setup for your target hardware, after which you configure the Trainer with your chosen distributed strategy (data/tensor/pipeline parallel) to begin pretraining or fine-tuning.Which hardware does it support beyond NVIDIA GPUs?Besides NVIDIA GPUs, PaddleNLP supports Kunlun XPU, Ascend NPU, Enflame GCU, and Hygon DCU accelerators, letting you switch hardware backends through the same套件 interface.
Besides NVIDIA GPUs, PaddleNLP supports Kunlun XPU, Ascend NPU, Enflame GCU, and Hygon DCU accelerators, letting you switch hardware backends through the same套件 interface.Is this tied to the PaddlePaddle framework instead of PyTorch?Yes — it's built specifically on the PaddlePaddle deep learning framework rather than PyTorch, so you work within Paddle's ecosystem, though it supports porting popular model families like LLaMA, Qwen, and DeepSeek.
Yes — it's built specifically on the PaddlePaddle deep learning framework rather than PyTorch, so you work within Paddle's ecosystem, though it supports porting popular model families like LLaMA, Qwen, and DeepSeek.Is this meant for hobbyist experiments or production-scale deployment?It's aimed at production/industrial-scale LLM applications, combining ease of use with heavily optimized performance for real deployment scenarios.
It's aimed at production/industrial-scale LLM applications, combining ease of use with heavily optimized performance for real deployment scenarios.Which model families are supported out of the box?PaddleNLP ships support for LLaMA 1/2/3, Baichuan, Bloom, ChatGLM, DeepSeek V2/V3/R1, Gemma, Mistral/Mixtral, OPT, Qwen (1.5/2/2.5/3), and its own PP-UIE models, among others.
PaddleNLP ships support for LLaMA 1/2/3, Baichuan, Bloom, ChatGLM, DeepSeek V2/V3/R1, Gemma, Mistral/Mixtral, OPT, Qwen (1.5/2/2.5/3), and its own PP-UIE models, among others.What is PP-UIE?PP-UIE is PaddleNLP's self-developed universal information extraction model, supporting 8K-token-long documents, strong zero/few-shot extraction, and a reported 1.8x training efficiency gain over LLaMA-Factory.
PP-UIE is PaddleNLP's self-developed universal information extraction model, supporting 8K-token-long documents, strong zero/few-shot extraction, and a reported 1.8x training efficiency gain over LLaMA-Factory.What inference optimizations does it offer?It includes FP8/INT8/4-bit quantization, MTP speculative decoding, and fused operators — DeepSeek V3/R1 FP8 inference reportedly exceeds 1000 tokens/s and 4-bit inference exceeds 2100 tokens/s on a single machine.
It includes FP8/INT8/4-bit quantization, MTP speculative decoding, and fused operators — DeepSeek V3/R1 FP8 inference reportedly exceeds 1000 tokens/s and 4-bit inference exceeds 2100 tokens/s on a single machine.What is Unified Checkpoint?Unified Checkpoint is PaddleNLP's storage system that enables elastic resume across changing machine resources, with asynchronous saving cutting checkpoint storage time and space by up to 95% and 78.5% respectively.
Unified Checkpoint is PaddleNLP's storage system that enables elastic resume across changing machine resources, with asynchronous saving cutting checkpoint storage time and space by up to 95% and 78.5% respectively.What are similar projects?airllm, ms-swift, jeeves, DMI, Hy-MT2, TensorRT-LLM, ktransformers, halo
airllm, ms-swift, jeeves, DMI, Hy-MT2, TensorRT-LLM, ktransformers, haloWhich topics is it in?document-intelligence, neural-search, embedding, bert, question-answering, sentiment-analysis, distributed-training, speculative-decoding, compression, search-engine, moe, llama
document-intelligence, neural-search, embedding, bert, question-answering, sentiment-analysis, distributed-training, speculative-decoding, compression, search-engine, moe, llama