Trendshift - Ask AI

base on ComfyUI related stuff and things # Overview A variety of ComfyUI related workflows and other stuff. You'll need different models and custom nodes for each different workflow. As this page has multiple headings you'll need to scroll down to see more. # Resources You'll need models and other resources for ComfyUI. Check the table below for links to everything from ControlNet models to Upscalers Item | Description | Link | --- | --- | --- | ComfyUI | The main thing you'll need! | https://github.com/comfyanonymous/ComfyUI See https://youtu.be/2r3uM_b3zA8 for an install guide ComfyUI Manager | Install any missing nodes using this | https://github.com/ltdrdata/ComfyUI-Manager Stability AI | Models & VAEs | https://huggingface.co/stabilityai Text-to-Image models | Text-2-image models | https://huggingface.co/models?pipeline_tag=text-to-image&sort=trending SSD-1B | Text2-image model | https://huggingface.co/segmind/SSD-1B ControlNet Models | ControlNet Models | https://huggingface.co/lllyasviel/sd_control_collection/tree/main https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main QR Code Monster Control Net | ControlNet Model | https://huggingface.co/monster-labs/control_v1p_sd15_qrcode_monster IP Adapter | Github Repo | https://github.com/tencent-ailab/IP-Adapter IP Adapter models | Models | https://huggingface.co/h94/IP-Adapter T2I Adapter | Github Repo | https://github.com/TencentARC/T2I-Adapter Control LoRA | Control Models | https://huggingface.co/stabilityai/control-lora AnimateDiff | Original repo, many links and more info | https://github.com/guoyww/AnimateDiff Latent Consistency Models | Models | https://huggingface.co/latent-consistency Upscale Wiki | Many models & info | https://upscale.wiki/wiki/Main_Page Artist Style Studies | SDXL Prompt output examples for inspiration | https://sdxl.parrotzone.art/ # List of workflows available In ComfyUI the image **IS** the workflow. Simply drag or load a workflow image into ComfyUI! See the "troubleshooting" section if your local install is giving errors :) Workflow | Description | Version | --- | --- | --- | <img src="workflows/SDXL/SDXL_Depth_Badger.png" width="256px"></img> | Basic SDXL ControlNet workflow. Introductory SDXL Canny & Depth ControlNet example. See https://youtu.be/reqamcrPYiM for more information. | SDXL <img src="workflows/SD15/nr_sd15_QR_Monster.png" width="256px"></img>| Basic QR Code Monster SD 1.5 ControlNet - make spiral art! See also - https://youtu.be/D4oJz0w36ps | SD 1.5 <img src="workflows/SD15/nr_sd15_QR_Monster_AnimateDiff_LatentUpscale.png " width="256px"></img>| QR Code Monster SD 1.5 ControlNet - make animated spiral art! See also: https://youtu.be/D4oJz0w36ps | SD 1.5 <img src="workflows/SD15/AnimateDIff_FreeU.png" width="256px"></img> | Updated QR Code Monster SD 1.5 ControlNet with AnimateDiff and FreeU Works best with the v1 QR Code Monster - https://huggingface.co/monster-labs/control_v1p_sd15_qrcode_monster | SD 1.5 <img src="workflows/SD15/AnimateDiff_MotionLoRA.png" width="256px"></img> | AnimateDiff with Montion LoRA example. Pan up, down, left right, etc. | SD 1.5 <img src="workflows/SD15/Instant_LoRA_1.png" width="256px"></img>|Instant LoRA 1 Inspired by <a href="https://civitai.com/articles/2345/aloeveras-instant-lora-no-training-15-sdxl">AloeVera</a> (almost identical). Really simple, no training, "LoRA" like functionality. SD 1.5. IP Adapter models: **1.** https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter-plus_sd15.bin -> ```custom_nodes/IPAdapter-ComfyUI/models```. **2.** https://huggingface.co/h94/IP-Adapter/blob/main/models/image_encoder/model.safetensors -> ```models/clip_vision```. NB (2024). As IPAdapter-ComfyUI from 2023 is now deprecated, you should replace it with a currently supported version before use **Video guide** - https://youtu.be/HtmIC6fqsMQ | SD 1.5 <img src="workflows/SD15/Instant_LoRA_2.png" width="256px"></img>|Instant LoRA 2 As above, but with ControlNet to guide the shape | SD 1.5 <img src="workflows/SD15/Instant_LoRA_3.png" width="256px"></img>|Instant LoRA 3 As above, but with QR Code Monster ControlNet too :) | SD 1.5 <img src="workflows/SD15/Instant_LoRA_4.png" width="256px"></img>|Instant LoRA 4 As above, but with basic upscaling | SD 1.5 <img src="workflows/SD15/Instant_LoRA_5.png" width="256px"></img>|Instant LoRA 5 As above, but with more upscaling to 16k+ | SD 1.5 <img src="workflows/SD15/Instant_LoRA_6.png" width="256px"></img>|Instant LoRA 6 As above, but different upscaling to 16k+ | SD 1.5 <img src="workflows/SD15/PromptTravel_AnimateDiff_IPAdapter.png" width="256px"></img>|Morphing AI videos of any length using AnimateDiff. SD 1.5. Includes IPAdapter & Upscaling. IP Adapter models: **1.** https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter-plus_sd15.bin -> ```custom_nodes/IPAdapter-ComfyUI/models```. **2.** https://huggingface.co/h94/IP-Adapter/blob/main/models/image_encoder/model.safetensors -> ```models/clip_vision```. **Video guide** - https://youtu.be/6A3a0QNPhIs | SD 1.5 <img src="workflows/SD15/PromptTravel_AnimateDiff.png" width="256px"></img>|Morphing AI videos of any length using AnimateDiff. SD 1.5. Includes Upscaling. Like above, but without IPAdapter controls. | SD 1.5 <img src="workflows/SDXL/SDXL_Instant_LoRA_1.png" width="256px"></img>|SDXL "Instant LoRA" - basic. Really simple, no training, "LoRA" like functionality. Uses SDXL IP Adapter - https://huggingface.co/h94/IP-Adapter Video - https://youtu.be/dGL02W4QatI | SDXL <img src="workflows/SDXL/SDXL_Instant_LoRA_2.png" width="256px"></img>|SDXL "Instant LoRA" - with CLIP Vision Uses SDXL IP Adapter - https://huggingface.co/h94/IP-Adapter Also use "Revisions" CLIP vision - https://huggingface.co/stabilityai/control-lora | SDXL <img src="workflows/SDXL/SDXL_Instant_LoRA_3.png" width="256px"></img>|SDXL "Instant LoRA" - with CLIP Vision & ControlNet Uses SDXL IP Adapter - https://huggingface.co/h94/IP-Adapter Also use "Revisions" CLIP vision - https://huggingface.co/stabilityai/control-lora | SDXL <img src="workflows/SD15/AnimateDiff_QRCode_Video.png" width="256px"></img>|AnimateDiff + QRCode (Vid2Vid) Use any high-contrast input video to create guided animations! Spirals away... | SD 1.5 <img src="workflows/SD15/Reposer2.png" width="256px"></img> <img src="workflows/SD15/Reposer_Plus_bypass.png" width="256px"></img></img>|SD 1.5 Reposer (2 versions) - single face image to any pose. Get consistent faces! **No** "roop" or similar face-swapping nodes required = easy install! SD 1.5 ControlNet models: https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main IP Adapter models: **1.** Face = https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter-plus-face_sd15.bin **2.** Vision = https://huggingface.co/h94/IP-Adapter/blob/main/models/image_encoder/model.safetensors *NOTE* Reposer2.png now uses the _even more_ updated version of IPAdapter Reposer Plus Bypass Edition is deprecated, but still available for download if you want to update any nodes at home. **Original Reposer Basic Video guide** - https://youtu.be/SacK9tMVNUA **Original Reposer Plus Video guide** - https://youtu.be/ZcCfwTkYSz8 | SD 1.5 <img src="workflows/SD15/Video_Restyler.png" width="256px">|SD 1.5 Video Styler! Combining IPAdapter with Video-to-video for strange styles and weird animations Uses https://github.com/cubiq/ComfyUI_IPAdapter_plus The pre-trained models are available on [huggingface](https://huggingface.co/h94/IP-Adapter), download and place them in the ```ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/models``` directory. For SD1.5 you need: * ip-adapter_sd15.bin * ip-adapter_sd15_light.bin * ip-adapter-plus_sd15.bin * ip-adapter-plus-face_sd15.bin Additionally you need the image encoder to be placed in the ```ComfyUI/models/clip_vision/``` directory. They are the same models used by the _other_ IPAdapter custom nodes ;) - symlinks are your friend! **Video guide** - https://youtu.be/kJp8JzA2aVU | SD 1.5 <img src="workflows/SDXL/SDXL_Reposer_Basic.png" width="256px">|SDXL version of Reposer using the SDXL "IPAdapter Plus Face" model Pick a face then add a body in any pose - no training! Works with photorealistic faces, anime faces, cartoon faces, etc | SDXL <img src="workflows/SDXL/SSD1B-SDXL-8GB.png" width="256px">|SSD-1B Workflow - SDXL for 8GB VRAM cards! Model - https://huggingface.co/segmind/SSD-1B Video - https://youtu.be/F-bKndyQ7L8|SSD-1B <img src="workflows/SD15/LCM_LoRA_Compare.png" width="256px">|LCM LoRA vs Normal|1.5, SDXL, SSD-1B <img src="workflows/SD15/SD15_IPAdapterMask_Upscale.png" width="256px">|IPAdapter Attention Masking Example **Video** https://youtu.be/riLmjBlywcg|1.5 <img src="workflows/SD15/SD15_LCM_IPAdapter_Facefix.png" width="256px">|IPAdapter Attention Masking Example with extra toppings (LCM, Facefix) **Video** https://youtu.be/riLmjBlywcg|1.5 <img src="workflows/SDCore/SVD_Basic_Upscale.png" width="256px">|Stable Video Diffusion example with a simple upscale and frame interpolation|SVD <img src="workflows/SDCore/SDXL_Turbo_Basic.png" width="256px">|SDXL Turbo - 1 step diffusion!|SDXL Turbo, SD2 Turbo <img src="workflows/SD15/ComfyMagicAnimate.png" width="256px">|A very basic attempt at a "Comfy MagicAnimate". Needs more work :) Links: Magic Animate - https://github.com/magic-research/magic-animate Magic Animate (Windows) - https://github.com/sdbds/magic-animate-for-windows DreamingTulpa - https://twitter.com/dreamingtulpa/status/1730876691755450572 CocktailPeanut - https://twitter.com/cocktailpeanut/status/1732052909720797524 Google Colab - https://github.com/camenduru/MagicAnimate-colab Huggingface Space - https://huggingface.co/spaces/zcxu-eric/magicanimate Vid2DensePose - https://github.com/Flode-Labs/vid2densepose Model Downloads for the MagicAnimate Gradio App: `mkdir -p magic-animate/pretrained_models` `cd magic-animate/pretrained_models` `git lfs clone https://huggingface.co/runwayml/stable-diffusion-v1-5 -b fp16` `git lfs clone https://huggingface.co/stabilityai/sd-vae-ft-mse` `git lfs clone https://huggingface.co/zcxu-eric/MagicAnimate` **Video** - https://youtu.be/td27SyA9M80| SD 1.5 <img src="workflows/SD15/BatchImageAnimate.png" width="256px">|Steerable Motion - Image Batch with AnimateDiff Video guide - https://youtu.be/bH-56e3cR2g| SD 1.5 <img src="workflows/SD15/Renoiser.png" width="256px">|Unsampler - Turn images into noise and back again, as modified by your prompts! Video guide - https://youtu.be/qW1I7in1WL0| SD 1.5, SDXL <img src="workflows/SDXL/Refacer.png" width="256px">|Refacer - Change the style of any face! Video guide - https://youtu.be/r7Iz8Ps7R2s| SDXL # Troubleshooting When troubleshooting (working to fix issues) - such as with your local custom node installs, it's best to do _all_ of these steps until resolution. * Need even more help or updated workflows? Drop me a DM on https://www.patreon.com/NerdyRodent and help support the channel! :) * Make sure you've installed the drivers for your graphics card * In ComfyUI the image IS the workflow. * These workflows require ComfyUI to run, so you'll need to install that first. See https://youtu.be/2r3uM_b3zA8 for an install guide * Install **ComfyUI Manager** next - https://github.com/ltdrdata/ComfyUI-Manager * Use **ComfyUI Manager** to install missing custom nodes by clicking "Install Missing Custom Nodes" * If ComfyUI Manager can't find a node automatically, use the _search_ feature * __Be sure to keep ComfyUI updated regularly__ - including all custom nodes. Old versions may result in errors appearing. This is the most common issue, so update now! * These are just workflows - no custom nodes here, so no code to go wrong :) * Need a model or checkpoint? See the resources section! * By default, models are saved in subdirectories under ``ComfyUI/models``, though some custom nodes have their own models directory. * _Don't mix SDXL and SD1.5 models_ (unless stated, such as SDXL needing the SD 1.5 vision model) - chances are you'll get an error! * Don't try to use SDXL models in workflows not designed for SDXL - chances are they won't work! * Ensure your model files aren't corrupt - try a fresh download if a particular model gives errors * Some workflows are _large_. Zoom out to see more of the canvas. * Custom node still red after installing it? Remember to restart ComfyUI! * Custom node still giving an error? Check the GitHub page for that custom node - maybe someone else has a similar issue open? * Not sure where the GitHub page is for a custom node? You can click on it via ComfyUI Manager * Check the output when ComfyUI starts up as issues can show up there * Try updating custom nodes manually (`git pull`) * Over time, some custom nodes implement breaking changes so nodes may need to be replaced. Often this can be done with "fix node". Typically the custom node GitHub page has additional information. * Sometimes custom nodes just break! Check the github page for the custom node causing any issues for more information and to raise issues * Sometimes custom nodes change functionality, so check for updates. Changes include: * Segement anything - mask output inverted & now returns multiple images * Dynamic Thresholding - output different * The original IPAdapter ('IPAdapter-ComfyUI') is deprecated and has been moved to the legacy channel. * March 2024 - the "new" IP Adapter node (IP Adapter Plus) implemented breaking changes which require the node the be re-created. The usual "fix node" won't work. * The Microsoft Windows portable version of ComfyUI apparently has issues with various custom nodes, whereas normal installs are OK. Unknown error? Try a normal install! * Need more help? See this [Playlist with loads of ComfyUI guides](https://www.youtube.com/playlist?list=PLjC8P1vEncQDahWnl_WKYsjF_tmIDXWEa) # Updating / Installing Custom Nodes * Use ComfyUI Manager to install and update custom nodes with ease! * Click "Install Missing Custom Nodes" to install any red nodes * Use the "search" feature to find any nodes * __Be sure to keep ComfyUI updated regularly__ - including all custom nodes. Old versions may result in errors appearing. <img src="ComfyUI_Manager.png" width="720px"></img> # Custom Node List * Screenshots of my installed custom nodes for reference. Not all nodes are used in workflows. * Install custom nodes using ComfyUI manager * See the Troubleshooting section if you have errors with your local ComfyUI install <img src="ComfyUI_Custom_1.png" width="720px"> <img src="ComfyUI_Custom_2.png" width="720px"> <img src="ComfyUI_Custom_3.png" width="720px"> ", Assign "at most 3 tags" to the expected json: {"id":"6212","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"

AI prompts

AI prompts