mmdetection README

The project's README from github.com/open-mmlab/mmdetection. As one markdown file: README.md.

 
OpenMMLab website HOT      OpenMMLab platform TRY IT OUT
 

PyPI docs badge codecov license open issues issue resolution Open in OpenXLab

📘Documentation | 🛠️Installation | 👀Model Zoo | 🆕Update News | 🚀Ongoing Projects | 🤔Reporting Issues

English | 简体中文

Introduction

MMDetection is an open source object detection toolbox based on PyTorch. It is a part of the OpenMMLab project.

The main branch works with PyTorch 1.8+.

Major features
  • Modular Design

    We decompose the detection framework into different components and one can easily construct a customized object detection framework by combining different modules.

  • Support of multiple tasks out of box

    The toolbox directly supports multiple detection tasks such as object detection, instance segmentation, panoptic segmentation, and semi-supervised object detection.

  • High efficiency

    All basic bbox and mask operations run on GPUs. The training speed is faster than or comparable to other codebases, including Detectron2, maskrcnn-benchmark and SimpleDet.

  • State of the art

    The toolbox stems from the codebase developed by the MMDet team, who won COCO Detection Challenge in 2018, and we keep pushing it forward. The newly released RTMDet also obtains new state-of-the-art results on real-time instance segmentation and rotated object detection tasks and the best parameter-accuracy trade-off on object detection.

Apart from MMDetection, we also released MMEngine for model training and MMCV for computer vision research, which are heavily depended on by this toolbox.

What's New

💎 We have released the pre-trained weights for MM-Grounding-DINO Swin-B and Swin-L, welcome to try and give feedback.

Highlight

v3.3.0 was released in 5/1/2024:

MM-Grounding-DINO: An Open and Comprehensive Pipeline for Unified Object Grounding and Detection

Grounding DINO is a grounding pre-training model that unifies 2d open vocabulary object detection and phrase grounding, with wide applications. However, its training part has not been open sourced. Therefore, we propose MM-Grounding-DINO, which not only serves as an open source replication version of Grounding DINO, but also achieves significant performance improvement based on reconstructed data types, exploring different dataset combinations and initialization strategies. Moreover, we conduct evaluations from multiple dimensions, including OOD, REC, Phrase Grounding, OVD, and Fine-tune, to fully excavate the advantages and disadvantages of Grounding pre-training, hoping to provide inspiration for future work.

code: mm_grounding_dino/README.md

We are excited to announce our latest work on real-time object recognition tasks, RTMDet, a family of fully convolutional single-stage detectors. RTMDet not only achieves the best parameter-accuracy trade-off on object detection from tiny to extra-large model sizes but also obtains new state-of-the-art performance on instance segmentation and rotated object detection tasks. Details can be found in the technical report. Pre-trained models are here.

PWC PWC PWC

Task Dataset AP FPS(TRT FP16 BS1 3090)
Object Detection COCO 52.8 322
Instance Segmentation COCO 44.6 188
Rotated Object Detection DOTA 78.9(single-scale)/81.3(multi-scale) 121

Installation

Please refer to Installation for installation instructions.

Getting Started

Please see Overview for the general introduction of MMDetection.

For detailed user guides and advanced guides, please refer to our documentation:

We also provide object detection colab tutorial Open in Colab and instance segmentation colab tutorial Open in Colab.

To migrate from MMDetection 2.x, please refer to migration.

Overview of Benchmark and Model Zoo

Results and models are available in the model zoo.

Architectures
Object Detection Instance Segmentation Panoptic Segmentation Other
  • Contrastive Learning
  • Distillation
  • Semi-Supervised Object Detection
  • Components
    Backbones Necks Loss Common

    mmdetection (open-mmlab/mmdetection)

    Searching Database…

    What is this?MMDetection is an open source object detection toolbox built on PyTorch, part of the OpenMMLab project, that supports object detection, instance segmentation, panoptic segmentation, and semi-supervised detection.
    Is this a framework I build on, or a ready-to-run app?It is a modular research framework/library: it decomposes the detection pipeline into components (backbones, necks, heads) so you can combine modules to build custom detectors.
    How do I get it running?Follow the official Installation guide; after installing, use the User Guides on training/testing, running inference with existing models, and preparing datasets.
    What do I need installed before this works?It requires PyTorch 1.8+ and relies on two sibling OpenMMLab libraries, MMEngine (training) and MMCV (core CV ops), which it depends on heavily.
    Is this still the actively maintained version, or should I look at something newer?The project is actively updated with new releases (e.g., v3.3.0 and MM-Grounding-DINO in 2024), but users migrating from the older 2.x series need to follow a separate migration guide since APIs changed.
    How big is the model zoo?Huge — it covers dozens of architectures including Faster R-CNN, Mask R-CNN, RetinaNet, DETR variants, Swin Transformer, YOLOX, RTMDet, Grounding DINO, and many more across detection, instance and panoptic segmentation.
    What is RTMDet and why is it highlighted?RTMDet is a family of real-time, fully convolutional single-stage detectors released by the project that achieves state-of-the-art parameter-accuracy trade-offs on object detection and also leads in instance and rotated object detection, e.g. 52.8 AP at 322 FPS on COCO.
    What is MM-Grounding-DINO?It's the team's open-source reproduction and improvement of Grounding DINO, a model unifying open-vocabulary object detection and phrase grounding, with released Swin-B and Swin-L pretrained weights and evaluations across OOD, REC, phrase grounding, OVD, and fine-tuning settings.
    How does MMDetection compare in training speed to other toolboxes?The authors state its training speed is faster than or comparable to other codebases like Detectron2, maskrcnn-benchmark, and SimpleDet.
    Creator
    open-mmlab
    Size
    63.3 MB
    Stars
    33K
    Age
    8y
    Commits · 90 Days