base on [NeurIPS 2024] Official code for PuLID: Pure and Lightning ID Customization via Contrastive Alignment # PuLID (NeurIPS 2024) ### :open_book: PuLID: Pure and Lightning ID Customization via Contrastive Alignment > [](https://arxiv.org/abs/2404.16022) [](https://huggingface.co/spaces/yanze/PuLID) [](https://huggingface.co/spaces/yanze/PuLID-FLUX) [](https://replicate.com/zsxkib/pulid) [](https://replicate.com/zsxkib/flux-pulid)<br> > Zinan Guo*, Yanze Wu*✝, Zhuowei Chen, Lang Chen, Peng Zhang, Qian He <br> > (*Equal Contribution, ✝Corresponding Author) <br> > ByteDance Inc <br> ### :triangular_flag_on_post: Updates * **2024.10.31**: 🔥 We are happy to release our latest [models](https://huggingface.co/guozinan/PuLID), **PuLID-v1.1** and **PuLID-FLUX-v0.9.1**. See more in [Model Zoo](#european_castle-model-zoo) and [pulid v1.1 model](docs/pulid_v1.1.md). We also update a new revision for the [arXiv paper](https://arxiv.org/abs/2404.16022), which includes more results, details, and analysis, please check it out. * **2024.09.26**: 🎉 PuLID accepted by NeurIPS 2024 * **2024.09.12**: We're thrilled to announce the release of the **PuLID-FLUX-v0.9.0 model**. Enjoy exploring its capabilities! 😊 [Learn more about this model](docs/pulid_for_flux.md) * **2024.05.23**: share the [preview of our upcoming v1.1 model](docs/v1.1_preview.md), please stay tuned * **2024.05.01**: release v1 codes&models, also the [🤗HuggingFace Demo](https://huggingface.co/spaces/yanze/PuLID) * **2024.04.25**: release arXiv paper. ## PuLID for FLUX Please check the doc and demo of PuLID-FLUX [here](docs/pulid_for_flux.md). ### updates - [x] Local gradio demo is ready now - [x] Online HuggingFace demo is ready now [](https://huggingface.co/spaces/yanze/PuLID-FLUX) - [x] We have optimized the codes to support consumer-grade GPUS, and now **PuLID-FLUX can run on a 16GB graphic card**. Check the details [here](https://github.com/ToTheBeginning/PuLID/blob/main/docs/pulid_for_flux.md#local-gradio-demo) - [x] (Community Implementation) Online Replicate demo is ready now [](https://replicate.com/zsxkib/flux-pulid) - [x] Local gradio demo supports 12GB graphic card now - [x] v0.9.1 is ready now Below results are generated with PuLID-FLUX.  ## Examples Images generated with our PuLID  Applications https://github.com/ToTheBeginning/PuLID/assets/11482921/9bdd0c8a-99e8-4eab-ab9e-39bf796cc6b8 ## :european_castle: Model Zoo | Version | Base Model | Description | |:--------------------------------------------------------------------------------------------------:|:----------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:| | [PuLID-v1](https://huggingface.co/guozinan/PuLID/blob/main/pulid_v1.bin) | SDXL | Paper model. | | [PuLID-v1.1](https://huggingface.co/guozinan/PuLID/blob/main/pulid_v1.1.safetensors) | SDXL | Compared to PuLID-v1, better compatibility, editability, facial naturalness, and similarity. | | [PuLID-FLUX-v0.9.0](https://huggingface.co/guozinan/PuLID/blob/main/pulid_flux_v0.9.0.safetensors) | FLUX | Our first version for PuLID-FLUX, better prompt-following and image quality (since FLUX is more powerful than SDXL). But ID fidelity is not high enough for some male inputs | | [PuLID-FLUX-v0.9.1](https://huggingface.co/guozinan/PuLID/blob/main/pulid_flux_v0.9.1.safetensors) | FLUX | Compared to PuLID-FLUX-v0.9.0, better ID fidelity. From the quantitative metric of ID similarity, the improvement is about 5 percentage points. Meanwhile, the editability remains similar as before. | ## :wrench: Dependencies and Installation - Python >= 3.9 (Recommend to use [Anaconda](https://www.anaconda.com/download/#linux) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html)) - [PyTorch >= 2.0](https://pytorch.org/) if you don't need flux-dev-fp8, otherwise [PyTorch >= 2.4.1](https://pytorch.org/) ```bash # clone PuLID repo git clone https://github.com/ToTheBeginning/PuLID.git cd PuLID # create conda env conda create --name pulid python=3.10 # activate env conda activate pulid # Install dependent packages # 1. if you don't need flux-fp8, e.g., you are using xl or flux-bf16, install the following requirements.txt pip install -r requirements.txt # 2. if you need flux-fp8 (to put flux on consumer-grade gpu), install the following requirements_fp8.txt pip install -r requirements_fp8.txt ``` ## :zap: Quick Inference ### Local Gradio Demo ```bash # for v1 version python app.py # for v1.1 version python app_v1.1.py --base BASE_MODEL Usage: -base: can be RunDiffusion/Juggernaut-XL-v9 or Lykon/dreamshaper-xl-lightning ``` ### Online HuggingFace Demo Thanks for the GPU grant from HuggingFace team, you can try PuLID HF demo in - [https://huggingface.co/spaces/yanze/PuLID](https://huggingface.co/spaces/yanze/PuLID) for SDXL - [https://huggingface.co/spaces/yanze/PuLID-FLUX](https://huggingface.co/spaces/yanze/PuLID-FLUX) for FLUX ## :paperclip: Related Resources Following are some third-party implementations of PuLID we have found in the Internet. We appreciate the efforts of the respective developers for making PuLID accessible to a wider audience. If there are any PuLID based resources and applications that we have not mentioned here, please let us know, and we will include them in this list. #### Online Demo - **Colab**: https://github.com/camenduru/PuLID-jupyter provided by [camenduru](https://github.com/camenduru) - **Replicate (PuLID)**: https://replicate.com/zsxkib/pulid provided by [zsxkib](https://github.com/zsxkib) - **Replicate (PuLID-FLUX)**: https://replicate.com/zsxkib/flux-pulid provided by [zsxkib](https://github.com/zsxkib) #### ComfyUI - https://github.com/cubiq/PuLID_ComfyUI provided by [cubiq](https://github.com/cubiq), native ComfyUI implementation - https://github.com/ZHO-ZHO-ZHO/ComfyUI-PuLID-ZHO provided by [ZHO](https://github.com/ZHO-ZHO-ZHO), diffusers-based implementation #### WebUI - [SD.Next](https://github.com/vladmandic/automatic/blob/master/CHANGELOG.md#update-for-2024-11-21) Implementation provided by [vladmandic](https://github.com/vladmandic) - https://github.com/Mikubill/sd-webui-controlnet/pull/2838 provided by [huchenlei](https://github.com/huchenlei) #### Other Applications - PuLID-FLUX multi-person generation with [Regional-Prompting-FLUX](https://github.com/instantX-research/Regional-Prompting-FLUX), provided by [Anthony](https://github.com/antonioo-c) ## Disclaimer This project strives to impact the domain of AI-driven image generation positively. Users are granted the freedom to create images using this tool, but they are expected to comply with local laws and utilize it responsibly. The developers do not assume any responsibility for potential misuse by users. ## Citation If PuLID is helpful, please help to ⭐ the repo. If you find this project useful for your research, please consider citing our paper: ```bibtex @InProceedings{guo2024pulid, title={PuLID: Pure and Lightning ID Customization via Contrastive Alignment}, author={Guo, Zinan and Wu, Yanze and Chen, Zhuowei and Chen, Lang and Zhang, Peng and He, Qian}, booktitle={Advances in Neural Information Processing Systems}, year={2024} } ``` ## :e-mail: Contact If you have any comments or questions, please [open a new issue](https://github.com/ToTheBeginning/PuLID/issues/new/choose) or feel free to contact [Yanze Wu](https://tothebeginning.github.io/) and [Zinan Guo](mailto:[email protected]).", Assign "at most 3 tags" to the expected json: {"id":"9869","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"