Topics / #interpretability

Interpretability projects on GitHub

Opening the black box of models through attribution and visualization.

As of , the most-starred Interpretability projects on GitHub are shap (25.8K stars), awesome-production-machine-learning (21K stars) and pykan (16.3K stars). GitHub.Locker tracks 11 Interpretability repositories with 112.7K stars combined. The fastest-growing on October 16 was OBLITERATUS, with +15 stars in a day.

11Projects
112.7KStars in total
+36Stars in a day
Browse in the catalog →github.com/topics/interpretability ↗

Hot right now

Most stars gained in a day · Oct 16

+15

OBLITERATUS

elder-plinius/OBLITERATUS · ★ 8.8K

OBLITERATUS is an open-source toolkit for 'abliteration' — it probes a language model's hidden states to locate the internal directions responsible for refusing prompts, then surgically removes them so the model stops refusing while keeping its core language abilities.

+11

ai-torture-chamber

terrafying/ai-torture-chamber · ★ 965

A research project that steers small open-weight language models into strong negative or positive emotional states and measures what they say and do. A live version runs at wirehead.agency.

+3

awesome-production-machine-learning

EthicalML/awesome-production-machine-learning · ★ 21K

A curated, community-maintained directory of open source libraries for production machine learning: tools to deploy, monitor, version, scale, and secure ML systems, grouped into 25+ annotated sections.

+2

shap

shap/shap · ★ 25.8K

SHAP is a Python library that explains the output of any machine learning model using Shapley values from game theory — it tells you how much each feature contributed to a given prediction.

The all-time top

11 projects · by stars

  1. #1 · shap/shap

    shap

    SHAP is a Python library that explains the output of any machine learning model using Shapley values from game theory — it tells you how much each feature contributed to a given prediction.

    25.8K
  2. #2 · EthicalML/awesome-production-machine-learning

    awesome-production-machine-learning

    A curated, community-maintained directory of open source libraries for production machine learning: tools to deploy, monitor, version, scale, and secure ML systems, grouped into 25+ annotated sections.

    21K
  3. #3 · KindXiaoming/pykan

    pykan

    A Python implementation of Kolmogorov-Arnold Networks (KANs), a promising alternative to MLPs, released with the papers 'KAN: Kolmogorov-Arnold Networks' and 'KAN 2.0'.

    16.3K
  1. 4

    pytorch-grad-cam

    13Kjacobgil/pytorch-grad-cam · Python

    A Python package that implements many state-of-the-art Explainable AI (pixel attribution/Class Activation Map) methods for computer vision models, letting you visualize which parts of an image a model focuses on.

  2. 5

    lime

    12.2Kmarcotcr/lime · JavaScript

    LIME (Local Interpretable Model-agnostic Explanations) is a Python package that explains individual predictions of any machine learning classifier, whether it works on text, tabular data, or images.

  3. 6

    nlp_course

    10.7Kyandexdataschool/nlp_course · Jupyter Notebook

    This is the Yandex School of Data Analysis (YSDA) Natural Language Processing course: a full set of lecture slides, seminar notebooks, and homework assignments organized by week, covering NLP from word embeddings through modern LLM topics.

  4. 7

    OBLITERATUS

    8.8Kelder-plinius/OBLITERATUS · Python

    OBLITERATUS is an open-source toolkit for 'abliteration' — it probes a language model's hidden states to locate the internal directions responsible for refusing prompts, then surgically removes them so the model stops refusing while keeping its core language abilities.

  5. 8

    bdh

    3.6Kpathwaycom/bdh · Python

    BDH (Dragon Hatchling) is a biologically inspired large language model architecture that connects deep learning with neuroscience, aiming to explain how reasoning and generalization emerge in AI systems at the level of individual neuron interactions.

  6. 9

    ai-torture-chamber

    965terrafying/ai-torture-chamber · Python

    A research project that steers small open-weight language models into strong negative or positive emotional states and measures what they say and do. A live version runs at wirehead.agency.

  7. 10

    S-Space

    174MirroS-Lab/S-Space · Python

    S-Space is a research project that finds a three-dimensional linear subspace inside multimodal model activations which encodes object positions along horizontal, vertical, and distance axes, and shows it can be read out and even causally edited to change the model's spatial judgments.

  8. 11

    DMI

    65ProjectDMX/DMI · Python

    DMI is a decoupled, asynchronous observability stack that gives real-time access to a running LLM's internal state — residual streams, attention patterns, MLP outputs, KV-cache slices, logits — during inference or training, with minimal overhead.

Often tagged together