base on Versatile audio super resolution (any -> 48kHz) with AudioSR. # AudioSR: Versatile Audio Super-resolution at Scale [](https://arxiv.org/abs/2309.07314) [](https://audioldm.github.io/audiosr) [](https://replicate.com/nateraw/audio-super-resolution) Pass your audio in, AudioSR will make it high fidelity! Work on all types of audio (e.g., music, speech, dog, raining, ...) & all sampling rates. Share your thoughts/samples/issues in our discord channel: https://discord.gg/HWeBsJryaf  ## Change Log - 2025-06-28: Add [LSD calculation pitfall demonstration](example/lsd_calculation_pitfall/README.md) showing the importance of energy scaling for fair Log Spectral Distance evaluation. - 2024-12-31: The training code of AudioSR can be found [here](https://drive.google.com/file/d/1BaZuHbk1AfURX7SvkaD5_ZWLwun-wdpW/view?usp=drive_link) (For reference only. The code is not carefully organized.). - 2024-12-16: Add [Important things to know to make AudioSR work](example/how_to_make_audiosr_work.md).  - 2023-09-24: Add replicate demo (@nateraw); Fix error on windows, librosa warning etc (@ORI-Muchim). - 2023-09-16: Fix DC shift issue. Fix duration padding bug. Update default DDIM steps to 50. ## Gradio Demo To run the Gradio demo locally: 1. Install dependencies: `pip install -r requirements.txt` 2. Run the app: `python app.py` 3. Open the URL displayed to view the demo ## Commandline Usage ## Installation ```shell # Optional conda create -n audiosr python=3.9; conda activate audiosr # Install AudioLDM pip3 install audiosr==0.0.7 # or # pip3 install git+https://github.com/haoheliu/versatile_audio_super_resolution.git ``` ## Usage Process a list of files. The result will be saved at ./output by default. ```shell audiosr -il batch.lst ``` Process a single audio file. ```shell audiosr -i example/music.wav ``` Full usage instruction ```shell > audiosr -h > usage: audiosr [-h] -i INPUT_AUDIO_FILE [-il INPUT_FILE_LIST] [-s SAVE_PATH] [--model_name {basic,speech}] [-d DEVICE] [--ddim_steps DDIM_STEPS] [-gs GUIDANCE_SCALE] [--seed SEED] optional arguments: -h, --help show this help message and exit -i INPUT_AUDIO_FILE, --input_audio_file INPUT_AUDIO_FILE Input audio file for audio super resolution -il INPUT_FILE_LIST, --input_file_list INPUT_FILE_LIST A file that contains all audio files that need to perform audio super resolution -s SAVE_PATH, --save_path SAVE_PATH The path to save model output --model_name {basic,speech} The checkpoint you gonna use -d DEVICE, --device DEVICE The device for computation. If not specified, the script will automatically choose the device based on your environment. --ddim_steps DDIM_STEPS The sampling step for DDIM -gs GUIDANCE_SCALE, --guidance_scale GUIDANCE_SCALE Guidance scale (Large => better quality and relavancy to text; Small => better diversity) --seed SEED Change this value (any integer number) will lead to a different generation result. --suffix SUFFIX Suffix for the output file ``` ## TODO [](https://www.buymeacoffee.com/haoheliuP) - [ ] Add gradio demo. - [ ] Optimize the inference speed. ## Cite our work If you find this repo useful, please consider citing: ```bibtex @inproceedings{liu2024audiosr, title={{AudioSR}: Versatile audio super-resolution at scale}, author={Liu, Haohe and Chen, Ke and Tian, Qiao and Wang, Wenwu and Plumbley, Mark D}, booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing}, pages={1076--1080}, year={2024}, organization={IEEE} } ``` # Understanding the Impact of Cutoff Patterns on AudioSR Performance **AudioSR** is a powerful tool for audio super-resolution. However, its performance can be significantly influenced by the characteristics of the input data, especially the cutoff pattern. ## 🚩 When AudioSR May Fail 1. **Input Audio with Unfamiliar Cutoff Patterns** If the input audio file contains a cutoff pattern that is **significantly different** from those used in training, AudioSR may fail to perform effectively. 2. **Input Audio with Severe Distortions** Strong distortions such as excessive noise or reverb can degrade the performance of AudioSR. ## ❓ Why Do Cutoff Patterns Have Such a Huge Impact on AudioSR? During training, our data was simulated using **low-pass filtering**. The model was not trained to handle other causes of high-frequency loss, such as MP3 compression. As a result, AudioSR struggles when encountering unfamiliar cutoff patterns. For example, MP3 compression can introduce a cutoff pattern that looks like this:  ### Why This Matters As you can see, there are **spectrogram holes** near the cutoff range, which differ significantly from the patterns seen during training. When you apply AudioSR to such data, the output may look like this:  The higher frequencies are not adequately inpainted due to the unfamiliar cutoff pattern. ### A Simple Solution: Low-Pass Filtering To mitigate this issue, you can perform a **low-pass filtering** on the audio before feeding it into AudioSR. After low-pass filtering, the audio would resemble a standard low-pass cutoff pattern, like this:  When processed by AudioSR, the output will then be as expected, with improved high-frequency inpainting:  --- By understanding the limitations and addressing them with preprocessing, you can maximize the performance of AudioSR! ", Assign "at most 3 tags" to the expected json: {"id":"1951","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"