ndif-team/nnsight

Capture every activation and gradient of any PyTorch model — forward and backward — with automatic graph visualization, rich metadata, and live interventions. Works on any architecture, including dynamic and recurrent ones.

Model interpretability and understanding for PyTorch

Class activation maps for your PyTorch models (CAM, Grad-CAM, Grad-CAM++, Smooth Grad-CAM++, Score-CAM, SS-CAM, IS-CAM, XGrad-CAM, Layer-CAM, Finer-CAM, LeGrad, RefineCAM)

A library for mechanistic interpretability of GPT-style language models

open source interpretability platform 🧠

👋 Xplique is a Neural Networks Explainability Toolbox

View model summaries in PyTorch!

A JAX research toolkit for building, editing, and visualizing neural networks.