base on Latent Consistency Model for AUTOMATIC1111 Stable Diffusion WebUI # Latent Consistency Model for Stable Diffusion WebUI <!-- omit from toc -->  This extension aims to integrate [Latent Consistency Model (LCM)](https://latent-consistency-models.github.io/) into [AUTOMATIC1111 Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui). Note that LCMs are a completely different class of models than Stable Diffusion, and the only available checkpoint currently is [LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7). **This is a very barebone implementation written in an hour, so any PRs are welcome.** ## Table of Contents <!-- omit from toc --> - [Installation:](#installation) - [Img2Img / Vid2Vid](#img2img--vid2vid) - [Known Issues](#known-issues) - [1. Can't Find 'LCM' tab or `ImportError: cannot import name 'xxx' from 'diffusers.xxx'`](#1-cant-find-lcm-tab-or-importerror-cannot-import-name-xxx-from-diffusersxxx) - [2. `ValueError: Non-consecutive added token '<|startoftext|>' found. Should have index 49408 but has index 49406 in saved vocabulary.`](#2-valueerror-non-consecutive-added-token-startoftext-found-should-have-index-49408-but-has-index-49406-in-saved-vocabulary) - [3. `torch.cuda.OutOfMemoryError...`](#3-torchcudaoutofmemoryerror) ## Installation: Simply clone this repo to your `extensions/` directory: ``` git clone https://github.com/0xbitches/sd-webui-lcm ``` Or go to "Extensions->Install from URL". Then reload your WebUI. Generated images will be saved to `outputs/txt2img-images/LCM`. You can use PNG Info to examine generation data. ## Img2Img / Vid2Vid  Img2Img/Vid2Vid with LCM is now supported in A1111. Simply update your extension and you should see the extra tabs. Note that for these features, output height and width will be the same as input, and currently not changeable. Generated videos will be saved to `outputs/LCM-vid2vid`. ## Known Issues #### 1. Can't Find 'LCM' tab or `ImportError: cannot import name 'xxx' from 'diffusers.xxx'` This is usually due to the installed version of `diffusers` in your `venv` is not up-to-date. To check the version you have, go to `stable-diffusion-webui/venv/Scripts` and run the activate script (`Activate.ps1` on Windows). Then, run `pip show diffusers` to see the version installed. To upgrade `diffusers` to the latest version, run `pip3 install --upgrade diffusers`. If the problem still persists, then you likely have an extension that force installs an old version of `diffusers`. [EasyPhoto](https://github.com/aigc-apps/sd-webui-EasyPhot), for example, force installs `diffusers=0.18.2`. #### 2. `ValueError: Non-consecutive added token '<|startoftext|>' found. Should have index 49408 but has index 49406 in saved vocabulary.` To resolve this, locate your huggingface hub cache directory. It will be something like `~/.cache/huggingface/hub/path_to_lcm_dreamshaper_v7/tokenizer/`. On Windows, it will roughly be `C:\Users\YourUserName\.cache\huggingface\hub\models--SimianLuo--LCM_Dreamshaper_v7\snapshots\c7f9b672c65a664af57d1de926819fd79cb26eb8\tokenizer\`. Find the file `added_tokens.json` and change the contents to: ``` { "<|endoftext|>": 49409, "<|startoftext|>": 49408 } ``` or simply remove it. #### 3. `torch.cuda.OutOfMemoryError...` This is because Automatic1111 loads an SD checkpoint on top of LCM. Try Settings -> Actions -> Unload SD checkpoint to free VRAM ", Assign "at most 3 tags" to the expected json: {"id":"4089","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"