base on Simulation framework for accelerating research in Private Federated Learning # `pfl`: Python framework for Private Federated Learning simulations [](https://github.com/apple/pfl-research/blob/main/LICENSE) [](https://dl.circleci.com/status-badge/redirect/gh/apple/pfl-research/tree/main) [](https://github.com/apple/pfl-research/blob/main/pyproject.toml#L18) **Documentation website:** https://apple.github.io/pfl-research `pfl` is a Python framework developed at Apple to empower researchers to run efficient simulations with privacy-preserving federated learning (FL) and disseminate the results of their research in FL. We are a team comprising engineering and research expertise, and we encourage researchers to publish their papers, with this code, with confidence. The framework is `not` intended to be used for third-party FL deployments but the results of the simulations can be tremendously useful in actual FL deployments. We hope that `pfl` will promote open research in FL and its effective dissemination. ``pfl`` provides several useful features, including the following: * Get started quickly trying out PFL for your use case with your existing model and data. * Iterate quickly with fast simulations utilizing multiple levels of distributed training (multiple processes, GPUs and machines). * Flexibility and expressiveness - when a researcher has a PFL idea to try, ``pfl`` has flexible APIs to express these ideas. * Scalable simulations for large experiments with state-of-the-art algorithms and models. * Support both PyTorch and TensorFlow. * Unified benchmarks for datasets that have been vetted for both PyTorch and TensorFlow. * Support other models in addition to neural networks, e.g. GBDTs. Switching between types of models is seamless. * Tight integration with privacy features, including common mechanisms for local and central differential privacy. Results from benchmarks are maintained in [this Weights & Biases report](https://api.wandb.ai/links/pfl/5scd5f66). ## Installation Installation instructions can be found [here](http://apple.github.io/pfl-research/installation.html). `pfl` is available on PyPI and a full installation be done with pip: ``` pip install 'pfl[tf,pytorch,trees]' ``` ## Getting started - tutorial notebooks To try out `pfl` immediately without installation, we provide several colab notebooks for learning the different components in `pfl` hands-on. * [](https://colab.research.google.com/github/apple/pfl-research/blob/develop/tutorials/Introduction%20to%20Federated%20Learning%20with%20CIFAR10%20and%20TensorFlow.ipynb) Introduction to Federated Learning with CIFAR10 and TensorFlow. * [](https://colab.research.google.com/github/apple/pfl-research/blob/develop/tutorials/Introduction%20to%20PFL%20research%20with%20FLAIR.ipynb) Introduction to PFL research with FLAIR and PyTorch. * [](https://colab.research.google.com/github/apple/pfl-research/blob/develop/tutorials/Introduction%20to%20Differential%20Privacy%20with%20Federated%20Learning.ipynb) Introduction to Differential Privacy (DP) with Federated Learning. * [](https://colab.research.google.com/github/apple/pfl-research/blob/develop/tutorials/Creating%20Federated%20Dataset%20for%20PFL%20Experiment.ipynb) Creating Federated Dataset for PFL Experiment. We also support MLX! * ([Jupyter notebook](https://github.com/apple/pfl-research/blob/develop/tutorials/Introduction%20to%20Federated%20Learning%20with%20CIFAR10%20and%20MLX.ipynb)) Introduction to Federated Learning with CIFAR10 and MLX. But you have to run this notebook locally on your Apple silicon, see all Jupyter notebooks available [here](https://github.com/apple/pfl-research/tree/develop/tutorials). ## Getting started - benchmarks `pfl` aims to streamline the benchmarking process of testing hypotheses in the Federated Learning paradigm. The official benchmarks are available in the [benchmarks](./benchmarks) directory, using a variety of realistic dataset-model combinations with and without differential privacy (yes, we do also have CIFAR10). **Copying these examples is a great starting point for doing your own research.** [See the quickstart](./benchmarks#quickstart) on how to start converging a model on the simplest benchmark (CIFAR10) in just a few minutes. ## Contributing Researchers are invited to contribute to the framework. Please, see [here](http://apple.github.io/pfl-research/support/contributing.html) for more details. ## Citing pfl-research ``` @article{granqvist2024pfl, title={pfl-research: simulation framework for accelerating research in Private Federated Learning}, author={Granqvist, Filip and Song, Congzheng and Cahill, {\'A}ine and van Dalen, Rogier and Pelikan, Martin and Chan, Yi Sheng and Feng, Xiaojun and Krishnaswami, Natarajan and Jina, Vojta and Chitnis, Mona}, journal={arXiv preprint arXiv:2404.06430}, year={2024}, } ```", Assign "at most 3 tags" to the expected json: {"id":"8308","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"