base on Streamlined interface for generating images with AI in Krita. Inpaint and outpaint with optional text prompt, no tweaking required. <h1><img width="64px" src="ai_diffusion/icons/logo-128.png"> Generative AI <i>for Krita</i></h1> ✨[Features](#features) | ⭳ [Download](https://github.com/Acly/krita-ai-diffusion/releases/latest) | 🛠️[Installation](https://www.interstice.cloud/plugin) | 🎞️ [Video](https://youtu.be/Ly6USRwTHe0) | 🖼️[Gallery](#gallery) | 📖[Wiki](https://github.com/Acly/krita-ai-diffusion/wiki) | 💬[Discussion](https://github.com/Acly/krita-ai-diffusion/discussions) | 🗣️[Discord](https://discord.gg/pWyzHfHHhU) This is a plugin to use generative AI in image painting and editing workflows from within Krita. For a more visual introduction, see [**www.interstice.cloud**](https://www.interstice.cloud) The main goals of this project are: * **Precision and Control.** Creating entire images from text can be unpredictable. To get the result you envision, you can restrict generation to selections, refine existing content with a variable degree of strength, focus text on image regions, and guide generation with reference images, sketches, line art, depth maps, and more. * **Workflow Integration.** Most image generation tools focus heavily on AI parameters. This project aims to be an unobtrusive tool that integrates and synergizes with image editing workflows in Krita. Draw, paint, edit and generate seamlessly without worrying about resolution and technical details. * **Local, Open, Free.** We are committed to open source models. Customize presets, bring your own models, and run everything local on your hardware. Cloud generation is also available to get started quickly without heavy investment. [](https://youtu.be/Ly6USRwTHe0 "Watch video demo") ## <a name="features"></a> Features * **Inpainting**: Use selections for generative fill, expand, to add or remove objects * **Live Painting**: Let AI interpret your canvas in real time for immediate feedback. [Watch Video](https://youtu.be/AF2VyqSApjA?si=Ve5uQJWcNOATtABU) * **Upscaling**: Upscale and enrich images to 4k, 8k and beyond without running out of memory. * **Stable Diffusion**: Supports Stable Diffusion 1.5, and XL. Partial support for SD3. * **ControlNet**: Scribble, Line art, Canny edge, Pose, Depth, Normals, Segmentation, +more * **IP-Adapter**: Reference images, Style and composition transfer, Face swap * **Regions**: Assign individual text descriptions to image areas defined by layers. * **Job Queue**: Queue and cancel generation jobs while working on your image. * **History**: Preview results and browse previous generations and prompts at any time. * **Strong Defaults**: Versatile default style presets allow for a streamlined UI. * **Customization**: Create your own presets - custom checkpoints, LoRA, samplers and more. ## <a name="installation"></a> Getting Started See the [Plugin Installation Guide](https://www.interstice.cloud/plugin) for instructions. A concise (more technical) version is below: ### Requirements * Windows, Linux, MacOS * _On Linux/Mac:_ Python + venv must be installed * recommended version: 3.11 or 3.10 * usually available via package manager, eg. `apt install python3-venv` #### Hardware support To run locally a powerful graphics card with at least 6 GB VRAM (NVIDIA) is recommended. Otherwise generating images will take very long or may fail due to insufficient memory! <table> <tr><td>NVIDIA GPU</td><td>supported via CUDA</td></tr> <tr><td>AMD GPU</td><td>limited support, DirectML on Windows, ROCm on Linux (custom install)</td></tr> <tr><td>Apple Silicon</td><td>community support, MPS on macOS</td></tr> <tr><td>CPU</td><td>supported, but very slow</td></tr> </table> ### Installation 1. If you haven't yet, go and install [Krita](https://krita.org/)! _Required version: 5.2.0 or newer_ 1. [Download the plugin](https://github.com/Acly/krita-ai-diffusion/releases/latest). 2. Start Krita and install the plugin via Tools ▸ Scripts ▸ Import Python Plugin from File... * Point it to the ZIP archive you downloaded in the previous step. * ⚠ _This will delete any previous install of the plugin._ If you are updating from 1.14 or older please read [updating to a new version](https://github.com/Acly/krita-ai-diffusion/wiki/Common-Issues#how-do-i-update-to-a-new-version-of-the-plugin). * Check [Krita's official documentation](https://docs.krita.org/en/user_manual/python_scripting/install_custom_python_plugin.html) for more options. 3. Restart Krita and create a new document or open an existing image. 4. To show the plugin docker: Settings ‣ Dockers ‣ AI Image Generation. 5. In the plugin docker, click "Configure" to start local server installation or connect. > [!NOTE] > If you encounter problems please check the [FAQ / list of common issues](https://github.com/Acly/krita-ai-diffusion/wiki/Common-Issues) for solutions. > > Reach out via [discussions](https://github.com/Acly/krita-ai-diffusion/discussions), our [Discord](https://discord.gg/pWyzHfHHhU), or report [an issue here](https://github.com/Acly/krita-ai-diffusion/issues). Please note that official Krita channels are **not** the right place to seek help with > issues related to this extension! ### _Optional:_ Custom ComfyUI Server The plugin uses [ComfyUI](https://github.com/comfyanonymous/ComfyUI) as backend. As an alternative to the automatic installation, you can install it manually or use an existing installation. If the server is already running locally before starting Krita, the plugin will automatically try to connect. Using a remote server is also possible this way. Please check the list of [required extensions and models](https://github.com/Acly/krita-ai-diffusion/wiki/ComfyUI-Setup) to make sure your installation is compatible. ### _Optional:_ Object selection tools (Segmentation) If you're looking for a way to easily select objects in the image, there is a [separate plugin](https://github.com/Acly/krita-ai-tools) which adds AI segmentation tools. ## Contributing Contributions are very welcome! Check the [contributing guide](CONTRIBUTING.md) to get started. ## <a name="gallery"></a> Gallery _Live painting with regions (Click for video)_ [](https://youtu.be/PPxOE9YH57E "Watch video demo") _Inpainting on a photo using a realistic model_ <img src="media/screenshot-2.png"> _Reworking and adding content to an AI generated image_ <img src="media/screenshot-1.png"> _Adding detail and iteratively refining small parts of the image_ <img src="media/screenshot-3.png"> _Modifying the pose vector layer to control character stances (Click for video)_ [](https://youtu.be/-QDPEcVmdLI "Watch video demo") _Control layers: Scribble, Line art, Depth map, Pose_     ## Technology * Image generation: [Stable Diffusion](https://github.com/Stability-AI/generative-models) * Diffusion backend: [ComfyUI](https://github.com/comfyanonymous/ComfyUI) * Inpainting: [ControlNet](https://github.com/lllyasviel/ControlNet), [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter)", Assign "at most 3 tags" to the expected json: {"id":"5092","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"