base on ⚡ TabPFN: Foundation Model for Tabular Data ⚡ # TabPFN [](https://badge.fury.io/py/tabpfn) [](https://pepy.tech/project/tabpfn) [](https://discord.com/channels/1285598202732482621/) [](https://priorlabs.ai/docs) [](https://tinyurl.com/tabpfn-colab-local) [](https://pypi.org/project/tabpfn/) <img src="https://github.com/PriorLabs/tabpfn-extensions/blob/main/tabpfn_summary.webp" width="80%" alt="TabPFN Summary"> ⚠️ **Major Update: Version 2.0:** Complete codebase overhaul with new architecture and features. Previous version available at [v1.0.0](../../tree/v1.0.0) and `pip install tabpfn==0.1.11`. 📚 For detailed usage examples and best practices, check out [Interactive Colab Tutorial](https://tinyurl.com/tabpfn-colab-local) ## 🏁 Quick Start TabPFN is a foundation model for tabular data that outperforms traditional methods while being dramatically faster. This repository contains the core PyTorch implementation with CUDA optimization. > ⚡ **GPU Recommended**: > For optimal performance, use a GPU (even older ones with ~8GB VRAM work well; 16GB needed for some large datasets). > On CPU, only small datasets (≲1000 samples) are feasible. > No GPU? Use our free hosted inference via [TabPFN Client](https://github.com/PriorLabs/tabpfn-client). ### Installation Official installation (pip) ```bash pip install tabpfn ``` OR installation from source ```bash pip install "tabpfn @ git+https://github.com/PriorLabs/TabPFN.git" ``` OR local development installation ```bash git clone https://github.com/PriorLabs/TabPFN.git pip install -e "TabPFN[dev]" ``` ### Basic Usage #### Classification ```python from sklearn.datasets import load_breast_cancer from sklearn.metrics import accuracy_score, roc_auc_score from sklearn.model_selection import train_test_split from tabpfn import TabPFNClassifier # Load data X, y = load_breast_cancer(return_X_y=True) X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42) # Initialize a classifier clf = TabPFNClassifier() clf.fit(X_train, y_train) # Predict probabilities prediction_probabilities = clf.predict_proba(X_test) print("ROC AUC:", roc_auc_score(y_test, prediction_probabilities[:, 1])) # Predict labels predictions = clf.predict(X_test) print("Accuracy", accuracy_score(y_test, predictions)) ``` #### Regression ```python from sklearn.datasets import fetch_openml from sklearn.metrics import mean_squared_error, r2_score from sklearn.model_selection import train_test_split # Assuming there is a TabPFNRegressor (if not, a different regressor should be used) from tabpfn import TabPFNRegressor # Load Boston Housing data df = fetch_openml(data_id=531, as_frame=True) # Boston Housing dataset X = df.data y = df.target.astype(float) # Ensure target is float for regression # Train-test split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42) # Initialize the regressor regressor = TabPFNRegressor() regressor.fit(X_train, y_train) # Predict on the test set predictions = regressor.predict(X_test) # Evaluate the model mse = mean_squared_error(y_test, predictions) r2 = r2_score(y_test, predictions) print("Mean Squared Error (MSE):", mse) print("R² Score:", r2) ``` ### Best Results For optimal performance, use the `AutoTabPFNClassifier` or `AutoTabPFNRegressor` for post-hoc ensembling. These can be found in the [TabPFN Extensions](https://github.com/PriorLabs/tabpfn-extensions) repository. Post-hoc ensembling combines multiple TabPFN models into an ensemble. **Steps for Best Results:** 1. Install the extensions: ```bash git clone https://github.com/priorlabs/tabpfn-extensions.git pip install -e tabpfn-extensions ``` 2. ```python from tabpfn_extensions.post_hoc_ensembles.sklearn_interface import AutoTabPFNClassifier clf = AutoTabPFNClassifier(max_time=120, device="cuda") # 120 seconds tuning time clf.fit(X_train, y_train) predictions = clf.predict(X_test) ``` ## 🌐 TabPFN Ecosystem Choose the right TabPFN implementation for your needs: - **[TabPFN Client](https://github.com/priorlabs/tabpfn-client)** Simple API client for using TabPFN via cloud-based inference. - **[TabPFN Extensions](https://github.com/priorlabs/tabpfn-extensions)** A powerful companion repository packed with advanced utilities, integrations, and features - great place to contribute: - 🔍 **`interpretability`**: Gain insights with SHAP-based explanations, feature importance, and selection tools. - 🕵️‍♂️ **`unsupervised`**: Tools for outlier detection and synthetic tabular data generation. - 🧬 **`embeddings`**: Extract and use TabPFN’s internal learned embeddings for downstream tasks or analysis. - 🧠 **`many_class`**: Handle multi-class classification problems that exceed TabPFN's built-in class limit. - 🌲 **`rf_pfn`**: Combine TabPFN with traditional models like Random Forests for hybrid approaches. - ⚙️ **`hpo`**: Automated hyperparameter optimization tailored to TabPFN. - 🔁 **`post_hoc_ensembles`**: Boost performance by ensembling multiple TabPFN models post-training. ✨ To install: ```bash git clone https://github.com/priorlabs/tabpfn-extensions.git pip install -e tabpfn-extensions ``` - **[TabPFN (this repo)](https://github.com/priorlabs/tabpfn)** Core implementation for fast and local inference with PyTorch and CUDA support. - **[TabPFN UX](https://ux.priorlabs.ai)** No-code graphical interface to explore TabPFN capabilities—ideal for business users and prototyping. ## 📜 License Prior Labs License (Apache 2.0 with additional attribution requirement): [here](https://priorlabs.ai/tabpfn-license/) ## 🤝 Join Our Community We're building the future of tabular machine learning and would love your involvement: 1. **Connect & Learn**: - Join our [Discord Community](https://discord.gg/VJRuU3bSxt) - Read our [Documentation](https://priorlabs.ai/docs) - Check out [GitHub Issues](https://github.com/priorlabs/tabpfn/issues) 2. **Contribute**: - Report bugs or request features - Submit pull requests - Share your research and use cases 3. **Stay Updated**: Star the repo and join Discord for the latest updates ## 📚 Citation You can read our paper explaining TabPFN [here](https://doi.org/10.1038/s41586-024-08328-6). ```bibtex @article{hollmann2025tabpfn, title={Accurate predictions on small data with a tabular foundation model}, author={Hollmann, Noah and M{\"u}ller, Samuel and Purucker, Lennart and Krishnakumar, Arjun and K{\"o}rfer, Max and Hoo, Shi Bin and Schirrmeister, Robin Tibor and Hutter, Frank}, journal={Nature}, year={2025}, month={01}, day={09}, doi={10.1038/s41586-024-08328-6}, publisher={Springer Nature}, url={https://www.nature.com/articles/s41586-024-08328-6}, } @inproceedings{hollmann2023tabpfn, title={TabPFN: A transformer that solves small tabular classification problems in a second}, author={Hollmann, Noah and M{\"u}ller, Samuel and Eggensperger, Katharina and Hutter, Frank}, booktitle={International Conference on Learning Representations 2023}, year={2023} } ``` ## ❓ FAQ ### **Usage & Compatibility** **Q: What dataset sizes work best with TabPFN?** A: TabPFN is optimized for **datasets up to 10,000 rows**. For larger datasets, consider using **Random Forest preprocessing** or other extensions. See our [Colab notebook](https://colab.research.google.com/drive/154SoIzNW1LHBWyrxNwmBqtFAr1uZRZ6a#scrollTo=OwaXfEIWlhC8) for strategies. **Q: Why can't I use TabPFN with Python 3.8?** A: TabPFN v2 requires **Python 3.9+** due to newer language features. Compatible versions: **3.9, 3.10, 3.11, 3.12, 3.13**. ### **Installation & Setup** **Q: How do I use TabPFN without an internet connection?** TabPFN automatically downloads model weights when first used. For offline usage: **Using the Provided Download Script** If you have the TabPFN repository, you can use the included script to download all models (including ensemble variants): ```bash # After installing TabPFN python scripts/download_all_models.py ``` This script will download the main classifier and regressor models, as well as all ensemble variant models to your system's default cache directory. **Manual Download** 1. Download the model files manually from HuggingFace: - Classifier: [tabpfn-v2-classifier.ckpt](https://huggingface.co/Prior-Labs/TabPFN-v2-clf/resolve/main/tabpfn-v2-classifier.ckpt) - Regressor: [tabpfn-v2-regressor.ckpt](https://huggingface.co/Prior-Labs/TabPFN-v2-reg/resolve/main/tabpfn-v2-regressor.ckpt) 2. Place the file in one of these locations: - Specify directly: `TabPFNClassifier(model_path="/path/to/model.ckpt")` - Set environment variable: `os.environ["TABPFN_MODEL_CACHE_DIR"] = "/path/to/dir"` - Default OS cache directory: - Windows: `%APPDATA%\tabpfn\` - macOS: `~/Library/Caches/tabpfn/` - Linux: `~/.cache/tabpfn/` **Q: I'm getting a `pickle` error when loading the model. What should I do?** A: Try the following: - Download the newest version of tabpfn `pip install tabpfn --upgrade` - Ensure model files downloaded correctly (re-download if needed) ### **Performance & Limitations** **Q: Can TabPFN handle missing values?** A: **Yes!** **Q: How can I improve TabPFN’s performance?** A: Best practices: - Use **AutoTabPFNClassifier** from [TabPFN Extensions](https://github.com/priorlabs/tabpfn-extensions) for post-hoc ensembling - Feature engineering: Add domain-specific features to improve model performance Not effective: - Adapt feature scaling - Convert categorical features to numerical values (e.g., one-hot encoding) ## 🛠️ Development 1. Setup environment: ```bash python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate git clone https://github.com/PriorLabs/TabPFN.git cd tabpfn pip install -e ".[dev]" pre-commit install ``` 2. Before committing: ```bash pre-commit run --all-files ``` 3. Run tests: ```bash pytest tests/ ``` --- Built with ❤️ by [Prior Labs](https://priorlabs.ai) - Copyright (c) 2025 Prior Labs GmbH ", Assign "at most 3 tags" to the expected json: {"id":"12693","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"