tidymodels/probably

Post-hoc calibration methods and metrics for classification

Python implementation of binary and multi-class Venn-ABERS calibration

Fast implementation of Venn-ABERS probabilistic predictors

relplot: Utilities for measuring calibration and plotting reliability diagrams

R package to compute and plot predictions, slopes, marginal means, and comparisons (contrasts, risk ratios, odds, etc.) for over 100 classes of statistical and ML models. Conduct linear and non-linear hypothesis tests, or equivalence tests. Calculate uncertainty estimates using the delta method, bootstrapping, or simulation-based inference

MCGrad is a scalable and easy-to-use tool for multicalibration. It ensures your ML model predictions are well-calibrated not just globally (across all data), but also across virtually any segment defined by your features

CalArena is a large-scale benchmark for post-hoc calibration of classification models.

Lightweight library for probabilistic forecast evaluation and optimization.

👻 Utilities for analyzing Bayesian models and posterior distributions