base on [CVPR 2024 Oral] Official repository of FMA-Net <div align="center"> <h2>FMA-Net (CVPR 2024 Oral)</h2> <div> <a href='https://sites.google.com/view/geunhyukyouk/' target='_blank'>Geunhyuk Youk</a>¹&nbsp; <a href='https://sites.google.com/view/ozbro/' target='_blank'>Jihyong Oh</a>^{† 2}&nbsp; <a href='https://www.viclab.kaist.ac.kr/' target='_blank'>Munchurl Kim</a>^{† 1} </div> <div> ^†Co-corresponding authors</span> </div> <div> ¹Korea Advanced Institute of Science and Technology, South Korea </div> <div> ²Chung-Ang University, South Korea </div> <div> <h4 align="center"> <a href="https://kaist-viclab.github.io/fmanet-site/" target='_blank'> <img src="https://img.shields.io/badge/🐳-Project%20Page-blue"> </a> <a href="https://arxiv.org/abs/2401.03707" target='_blank'> <img src="https://img.shields.io/badge/arXiv-2401.03707-b31b1b.svg"> </a> <a href="https://www.youtube.com/watch?v=kO7KavOH6vw" target='_blank'> <img src="https://img.shields.io/badge/Demo%20Video-%23FF0000.svg?logo=YouTube&logoColor=white"> </a> <a href="https://www.youtube.com/watch?v=G6qqJXztJDM" target='_blank'> <img src="https://img.shields.io/badge/Presentation-%23FF0000.svg?logo=YouTube&logoColor=white"> </a> <img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/KAIST-VICLab/FMA-Net"> </h4> </div> --- <div align="center"> <h4> This repository is the official PyTorch implementation of "FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring". </h4> </div> </div> ## 📧 News - **Apr 19, 2024:** Codes of FMA-Net (including the training, testing code, and pretrained model) are released :fire: - **Apr 05, 2024:** FMA-Net is selected for an ORAL presentation at CVPR 2024 (0.78% of 11,532 valid submissions) - **Feb 27, 2024:** FMA-Net accepted to CVPR 2024 :tada: - **Jan 14, 2024:** This repository is created ## 📝 TODO - [x] Release FMA-Net code - [x] Release pretrained FMA-Net model - [x] Add data preprocessing scripts <!-- **Reference**: --> ## Reference If you find FMA-Net useful, please consider citing: ```BibTeX @InProceedings{Youk_2024_CVPR, author = {Youk, Geunhyuk and Oh, Jihyong and Kim, Munchurl}, title = {FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2024}, pages = {44-55} } ``` ## Contents - [Requirements](#requirements) - [Data Preprocessing](#data-preprocessing) - [Pretrained Model](#pretrained-model) - [Training](#training) - [Testing](#testing) - [Results](#results) - [License](#license) - [Acknowledgement](#acknowledgement) ## Requirements > - Python 3.9, PyTorch >= 1.9.1 > - Platforms: Ubuntu 22.04, cuda 11.8 ## Data Preprocessing > - Download [REDS](https://seungjunnah.github.io/Datasets/reds.html) dataset > - Generate REDS4: run ./preprocessing/generate_reds4.py > - Generate RAFT pseudo-GT optical flow: run ./preprocessing/generate_flow.py (or download the optical flow from [here](https://www.dropbox.com/scl/fo/qgzadp9cqnmzyvghjk4v6/AN-b711qSN5RvakS9VaIpUc?rlkey=6di5bb9um962l8uko1hpiyx1h&st=tzd13ym2&dl=0)) ## Pretrained Model Pre-trained model can be downloaded from [here](https://www.dropbox.com/scl/fo/4392nxna1wptrw06ktv6r/AIyy20JrXK_9CMcXHUQY7Ko?rlkey=n4hhgl7p2c63y3l6lkpqlthi0&st=mnmmvm9y&dl=0). * *FMA-Net_REDS.zip*: trained on REDS dataset. ## Training ```bash # download code git clone https://github.com/KAIST-VICLab/FMA-Net cd FMA-Net # train FMA-Net on REDS dataset python main.py --train --config_path experiment.cfg ``` ## Testing ```bash # test FMA-Net on REDS dataset python main.py --test --config_path experiment.cfg # test on your own datasets python main.py --test_custom --config_path experiment.cfg ``` ## Results Please visit our [project page](https://kaist-viclab.github.io/fmanet-site/) and [demo video](https://www.youtube.com/watch?v=kO7KavOH6vw) for diverse visual results. ## License The source codes including the checkpoint can be freely used for research and education only. Any commercial use should get formal permission from the principal investigator (Prof. Munchurl Kim, [email protected]). ## Acknowledgement This work was supported by the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT): No. 2021-0-00087, Development of high-quality conversion technology for SD/HD low-quality media and No. RS2022-00144444, Deep Learning Based Visual Representational Learning and Rendering of Static and Dynamic Scenes. ", Assign "at most 3 tags" to the expected json: {"id":"6981","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"