segmentation_models.pytorch README

The project's README from github.com/qubvel-org/segmentation_models.pytorch. As one markdown file: README.md.

logo Python library with Neural Networks for Image Semantic Segmentation based on PyTorch.

GitHub Workflow Status (branch) Codecov Read the Docs
PyPI PyTorch - Version Python - Version
Generic badge PyPI - Downloads

The main features of the library are:

🤝 Sponsor: withoutBG

Sponsored by withoutBG

withoutBG is a high-quality background removal tool. They built their open-source image matting and refiner models using smp.Unet and are proudly sponsoring this project.

📚 Project Documentation 📚

Visit Read The Docs Project Page or read the following README to know more about Segmentation Models Pytorch (SMP for short) library

📋 Table of content

  1. Quick start
  2. Examples
  3. Models and encoders
  4. Models API
    1. Input channels
    2. Auxiliary classification output
    3. Depth
  5. Installation
  6. Competitions won with the library
  7. Contributing
  8. Citing
  9. License

⏳ Quick start

1. Create your first Segmentation model with SMP

The segmentation model is just a PyTorch torch.nn.Module, which can be created as easy as:

import segmentation_models_pytorch as smp

model = smp.Unet(
    encoder_name="resnet34",        # choose encoder, e.g. mobilenet_v2 or efficientnet-b7
    encoder_weights="imagenet",     # use `imagenet` pre-trained weights for encoder initialization
    in_channels=1,                  # model input channels (1 for gray-scale images, 3 for RGB, etc.)
    classes=3,                      # model output channels (number of classes in your dataset)
)

2. Configure data preprocessing

All encoders have pretrained weights. Preparing your data the same way as during weights pre-training may give you better results (higher metric score and faster convergence). It is not necessary in case you train the whole model, not only the decoder.

from segmentation_models_pytorch.encoders import get_preprocessing_fn

preprocess_input = get_preprocessing_fn('resnet18', pretrained='imagenet')

Congratulations! You are done! Now you can train your model with your favorite framework!

💡 Examples

Name Link Colab
Train pets binary segmentation on OxfordPets Notebook Open In Colab
Train cars binary segmentation on CamVid Notebook Open In Colab
Train multiclass segmentation on CamVid Notebook Open In Colab
Train clothes binary segmentation by @ternaus Repo
Load and inference pretrained Segformer Notebook Open In Colab
Load and inference pretrained DPT Notebook Open In Colab
Load and inference pretrained UPerNet Notebook Open In Colab
Save and load models locally / to HuggingFace Hub Notebook Open In Colab
Export trained model to ONNX Notebook Open In Colab

📦 Models and encoders

Architectures

Architecture Paper Documentation Checkpoints
Unet paper docs
Unet++ paper docs
MAnet paper docs
Linknet paper docs
FPN paper docs
PSPNet paper docs
PAN paper docs
DeepLabV3 paper docs
DeepLabV3+ paper docs
UPerNet paper docs checkpoints
Segformer paper docs checkpoints
DPT paper docs checkpoints

Encoders

The library provides a wide range of pretrained encoders (also known as backbones) for segmentation models. Instead of using features from the final layer of a classification model, we extract intermediate features and feed them into the decoder for segmentation tasks.

All encoders come with pretrained weights, which help achieve faster and more stable convergence when training segmentation models.

Given the extensive selection of supported encoders, you can choose the best one for your specific use case, for example:

By selecting the right encoder, you can balance efficiency, performance, and model complexity to suit your project needs.

All encoders and corresponding pretrained weight are listed in the documentation:

🔁 Models API

Input channels

The input channels parameter allows you to create a model that can process a tensor with an arbitrary number of channels. If you use pretrained weights from ImageNet, the weights of the first convolution will be reused:

model = smp.FPN('resnet34', in_channels=1)
mask = model(torch.ones([1, 1, 64, 64]))

Auxiliary classification output

All models support aux_params parameters, which is default set to None. If aux_params = None then classification auxiliary output is not created, else model produce not only mask, but also label output with shape NC. Classification head consists of GlobalPooling->Dropout(optional)->Linear->Activation(optional) layers, which can be configured by aux_params as follows:

aux_params=dict(
    pooling='avg',             # one of 'avg', 'max'
    dropout=0.5,               # dropout ratio, default is None
    activation='sigmoid',      # activation function, default is None
    classes=4,                 # define number of output labels
)
model = smp.Unet('resnet34', classes=4, aux_params=aux_params)
mask, label = model(x)

Depth

Depth parameter specify a number of downsampling operations in encoder, so you can make your model lighter if specify smaller depth.

model = smp.Unet('resnet34', encoder_depth=4)

🛠 Installation

PyPI version:

$ pip install segmentation-models-pytorch

The latest version from GitHub:

$ pip install git+https://github.com/qubvel/segmentation_models.pytorch

🏆 Competitions won with the library

Segmentation Models package is widely used in image segmentation competitions. Here you can find competitions, names of the winners and links to their solutions.

🛠 Projects built with SMP

🤝 Contributing

  1. Install SMP in dev mode
make install_dev  # Create .venv, install SMP in dev mode
  1. Run tests and code checks
make test          # Run tests suite with pytest
make fixup         # Ruff for formatting and lint checks
  1. Update a table (in case you added an encoder)
make table        # Generates a table with encoders and print to stdout

📝 Citing 

@misc{Iakubovskii:2019,
  Author = {Pavel Iakubovskii},
  Title = {Segmentation Models Pytorch},
  Year = {2019},
  Publisher = {GitHub},
  Journal = {GitHub repository},
  Howpublished = {\url{https://github.com/qubvel/segmentation_models.pytorch}}
}

🛡️ License

The project is primarily distributed under MIT License, while some files are subject to other licenses. Please refer to LICENSES and license statements in each file for careful check, especially for commercial use.

segmentation_models.pytorch (qubvel-org/segmentation_models.pytorch)

Searching Database…

11.8K ·GitHub
Segmentation_models.pytorch
What is this?A Python library for building image semantic segmentation models in PyTorch, with over 500 pretrained convolutional and transformer-based backbones.
Is this a framework or a library I just import?It is a library: you import it in your own PyTorch training code and get a model object (torch.nn.Module), not a standalone app or training framework.
How do I create a segmentation model with it?Install it, then create a model in two lines by picking an architecture (e.g. Unet, DeepLabV3+, Segformer) and an encoder/backbone, then plug it into your own PyTorch training loop.
Do I need PyTorch already set up to use this?Yes, it's built on top of PyTorch, and many of its 500+ encoders are supported via the timm library.
Does it include training code or just model building blocks?It only provides model architectures, encoders, losses and metrics as building blocks; you still write your own training loop in your framework of choice.
Is the whole repo MIT licensed?Mostly MIT, but some files are under other licenses, so check carefully especially for commercial use.
Which model architectures are supported?12 architectures including Unet, Unet++, MAnet, Linknet, FPN, PSPNet, PAN, DeepLabV3, DeepLabV3+, UPerNet, Segformer, and DPT.
Can I change the number of input channels, like for grayscale or multispectral images?Yes, the in_channels parameter lets a model accept any number of channels, and pretrained ImageNet weights are automatically adapted (summed for 1-channel input or tiled/scaled for other counts).
Can models also output a classification label alongside the segmentation mask?Yes, setting aux_params creates an auxiliary classification head (GlobalPooling -> Dropout -> Linear -> Activation) that outputs a label in addition to the mask.
Can I export a trained model to ONNX?Yes, the library is designed to be ONNX export and torch script/trace/compile friendly, and there's an example notebook for exporting to ONNX.
What are similar projects?VisionHOPE, detectron2, deep-learning-for-image-processing, HivisionIDPhotos, segment-anything, Pytorch-UNet, CVPR2026-Papers-with-Code, pytorch-image-models
Which topics is it in?unet, imagenet, semantic-segmentation, image-segmentation, segmentation, onnx, transformers, neural-network, image-processing, huggingface, computer-vision, pytorch
Creator
qubvel-org
Size
16.3 MB
Stars
11.8K
Updated
CommitOct 14Releasev0.5.0Apr 17
Age
7y
Commits · 90 Days