NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.

Overview

Real-ESRGAN ncnn Vulkan

CI License: MIT Open issue Closed issue

This project is the ncnn implementation of Real-ESRGAN. Real-ESRGAN ncnn Vulkan heavily borrows from realsr-ncnn-vulkan. Many thanks to nihui, ncnn and realsr-ncnn-vulkan 😁

Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration. We also optimize it for anime images.

Contents


If Real-ESRGAN is helpful in your photos/projects, please help to this repo or recommend it to your friends. Thanks 😊
Other recommended projects:
▶️ Real-ESRGAN: A practical algorithm for general image restoration
▶️ GFPGAN: A practical algorithm for real-world face restoration
▶️ BasicSR: An open-source image and video restoration toolbox
▶️ facexlib: A collection that provides useful face-relation functions.
▶️ HandyView: A PyQt5-based image viewer that is handy for view and comparison.

📖 Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

[Paper]   [Project Page]   [Demo]
Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan
Tencent ARC Lab; Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences

TODO List

  • Support further cheap arbitrary resize (e.g., bicubic, bilinear) for the model outputs
  • Bug: Some PCs will output black images
  • Add the guidance for ncnn model conversion
  • Support face restoration - GFPGAN

💻 Usages

Example Command

realesrgan-ncnn-vulkan.exe -i input.jpg -o output.png -n realesrgan-x4plus-anime

Full Usages

Usage: realesrgan-ncnn-vulkan.exe -i infile -o outfile [options]...

  -h                   show this help
  -v                   verbose output
  -i input-path        input image path (jpg/png/webp) or directory
  -o output-path       output image path (jpg/png/webp) or directory
  -s scale             upscale ratio (4, default=4)
  -t tile-size         tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu
  -m model-path        folder path to pre-trained models(default=models)
  -n model-name        model name (default=realesrgan-x4plus, can be realesrgan-x4plus | realesrgan-x4plus-anime | realesrnet-x4plus)
  -g gpu-id            gpu device to use (default=0) can be 0,1,2 for multi-gpu
  -j load:proc:save    thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu
  -x                   enable tta mode
  -f format            output image format (jpg/png/webp, default=ext/png)
  • input-path and output-path accept either file path or directory path
  • scale = scale level, 4 = upscale 4x
  • tile-size = tile size, use smaller value to reduce GPU memory usage, default selects automatically
  • load:proc:save = thread count for the three stages (image decoding + model upscaling + image encoding), using larger values may increase GPU usage and consume more GPU memory. You can tune this configuration with "4:4:4" for many small-size images, and "2:2:2" for large-size images. The default setting usually works fine for most situations. If you find that your GPU is hungry, try increasing thread count to achieve faster processing.
  • format = the format of the image to be output, png is better supported, however webp generally yields smaller file sizes, both are losslessly encoded

If you encounter crash or error, try to upgrade your GPU driver

🌏 Other Open-Source Code Used

📜 BibTeX

@InProceedings{wang2021realesrgan,
    author    = {Xintao Wang and Liangbin Xie and Chao Dong and Ying Shan},
    title     = {Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data},
    booktitle = {International Conference on Computer Vision Workshops (ICCVW)},
    date      = {2021}
}

📧 Contact

If you have any question, please email [email protected] or [email protected].

Comments
  • Segfault realesrnet-x4plus

    Segfault realesrnet-x4plus

    %  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg  -o FdLbqKcWIAAaCeY.png -n realesrnet-x4plus
    zsh: segmentation fault  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg -o FdLbqKcWIAAaCeY.png -n 
    
    opened by atomical 0
  • Convert result image to OpenCV `cv::Mat` ERROR

    Convert result image to OpenCV `cv::Mat` ERROR

    I have test ncnn::Mat.to_pixels() with inimage, but outimage report error when converting:

    // (main.cpp 376 line)
    ncnn::Mat out = v.outimage;
    cv::Mat res(out.h, out.w, CV_8UC3);
    out.to_pixels(res.data, ncnn::Mat::PIXEL_BGR2RGB); // ERROR here
    cv::imshow("res", res);
    cv::waitKey();
    cv::destroyAllWindows();
    

    With OpenCV 4.6.0 Windows 10x, always get this:

    Exception has occurred: W32/0xC0000005
    Unhandled exception thrown: read access violation.
    ptr1 was 0x1ACAC042000.
    

    /src/ncnn/src/mat_pixel.cpp 734 line, is the problem with out.data or other ?

    opened by Sun-ZhenXing 0
  • Your Results in New Super-Resolution Benchmarks

    Your Results in New Super-Resolution Benchmarks

    Hello,

    MSU Graphics & Media Lab Video Group has recently launched two new Super-Resolution Benchmarks.

    Your method achieved 1st place in Video Upscalers Benchmark: Quality Enhancement in 'Animation 2x' category and 3rd place in Super-Resolution for Video Compression Benchmark in 'x265 compression' category. We congratulate you on your result and look forward to your future work!

    We would be grateful for your feedback on our work.

    opened by EvgeneyZ 0
  • CMake errors with GLSLANG

    CMake errors with GLSLANG

    I think GLSLANG made something changes in their cmake files. I get next errors from your Cmakelists.txt with CMAKE

    Found glslangValidator: Z:/AI_SDK/CPP_GFPGAN/Real-ESRGAN-ncnn-vulkan/vulkan/Vulkan_SDK/Bin/glslangValidator.exe
    Using glslang install located at Z:/AI_SDK/CPP_GFPGAN/Real-ESRGAN-ncnn-vulkan/vulkan/glslang-master-tot/Build/install/lib/cmake
    CMake Warning at Z:/AI_SDK/CPP_GFPGAN/Real-ESRGAN-ncnn-vulkan/vulkan/glslang-master-tot/Build/install/lib/cmake/OSDependentTargets.cmake:2 (message):
      Using `OSDependentTargets.cmake` is deprecated: use `find_package(glslang)`
      to find glslang CMake targets.
    Call Stack (most recent call first):
      CMakeLists.txt:110 (include)
    
    
    CMake Warning at Z:/AI_SDK/CPP_GFPGAN/Real-ESRGAN-ncnn-vulkan/vulkan/glslang-master-tot/Build/install/lib/cmake/OGLCompilerTargets.cmake:2 (message):
      Using `OGLCompilerTargets.cmake` is deprecated: use `find_package(glslang)`
      to find glslang CMake targets.
    Call Stack (most recent call first):
      CMakeLists.txt:111 (include)
    
    
    CMake Warning at Z:/AI_SDK/CPP_GFPGAN/Real-ESRGAN-ncnn-vulkan/vulkan/glslang-master-tot/Build/install/lib/cmake/HLSLTargets.cmake:2 (message):
      Using `HLSLTargets.cmake` is deprecated: use `find_package(glslang)` to
      find glslang CMake targets.
    Call Stack (most recent call first):
      CMakeLists.txt:114 (include)
    
    
    CMake Warning at Z:/AI_SDK/CPP_GFPGAN/Real-ESRGAN-ncnn-vulkan/vulkan/glslang-master-tot/Build/install/lib/cmake/glslangTargets.cmake:2 (message):
      Using `glslangTargets.cmake` is deprecated: use `find_package(glslang)` to
      find glslang CMake targets.
    Call Stack (most recent call first):
      CMakeLists.txt:116 (include)
    
    
    CMake Warning at Z:/AI_SDK/CPP_GFPGAN/Real-ESRGAN-ncnn-vulkan/vulkan/glslang-master-tot/Build/install/lib/cmake/SPIRVTargets.cmake:2 (message):
      Using `SPIRVTargets.cmake` is deprecated: use `find_package(glslang)` to
      find glslang CMake targets.
    Call Stack (most recent call first):
      CMakeLists.txt:117 (include)
    
    
    CMake Error at CMakeLists.txt:245 (message):
      The submodules were not downloaded! Please update submodules with "git
      submodule update --init --recursive" and try again.
    
    
    Configuring incomplete, errors occurred!
    
    opened by magicse 1
  • Request: offer sample apps for Android and IOS

    Request: offer sample apps for Android and IOS

    As the title says. It seems this repository should make it possible, because of this:

    https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-android https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-ios-on-macos-with-xcode

    If you create a sample project (here or somewhere else and link to it), it could be easier for everyone to use it. Please, if you do it, show also how to update it in case there are new versions of the libraries.

    I would really appreciate it.

    opened by AndroidDeveloperLB 6
Owner
Xintao
Researcher at Tencent ARC Lab, (Applied Research Center)
Xintao
RealSR-NCNN-Android is a simple Android application that based on Realsr-NCNN & Real-ESRGAN.

RealSR-NCNN-Android Real-ESRGAN is a Practical Algorithms for General Image Restoration. RealSR-NCNN-Android is a simple Android application that base

null 200 Sep 29, 2022
This is a sample ncnn android project, it depends on ncnn library and opencv

This is a sample ncnn android project, it depends on ncnn library and opencv

null 231 Sep 30, 2022
ncnn demo of DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction

DocTr-ncnn ncnn demo of DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction model support: 1.Document Segmentation 2

FeiGeChuanShu 23 Sep 3, 2022
NCNN+Int8+YOLOv4 quantitative modeling and real-time inference

NCNN+Int8+YOLOv4 quantitative modeling and real-time inference

pengtougu 19 Sep 13, 2022
Reactive Light Training Module used in fitness for developing agility and reaction speed.

Hello to you , Thanks for taking interest in this project. Use case of this project is to help people that want to improve their agility and reactio

null 4 Sep 16, 2022
The repository contains our dataset and C++ implementation of the CVPR 2022 paper, Geometric Structure Preserving Warp for Natural Image Stitching.

Geometric Structure Preserving Warp for Natural Image Stitching This repository contains our dataset and C++ implementation of the CVPR 2022 paper, Ge

null 15 Sep 15, 2022
GPTPU: General-Purpose Computing on (Edge) Tensor Processing Units

GPTPU: General-Purpose Computing on (Edge) Tensor Processing Units Welcome to the repository of ESCAL @ UCR's GPTPU project! We aim at demonstrating t

Extreme Storage and Computer Architecture Lab 32 Jul 31, 2022
General broad-phase collision detection framework using BVH and BVTT front tracking.

This is the collision detection package by littlemine (Xinlei Wang). Configuration Instructions This project is developed using Visual Studio 2015 and

Xinlei Wang 47 Sep 9, 2022
Deep Learning API and Server in C++11 support for Caffe, Caffe2, PyTorch,TensorRT, Dlib, NCNN, Tensorflow, XGBoost and TSNE

Open Source Deep Learning Server & API DeepDetect (https://www.deepdetect.com/) is a machine learning API and server written in C++11. It makes state

JoliBrain 2.4k Oct 5, 2022
the C++ version of solov2 with ncnn

the C++ version of SOLOV2 with ncnn

DayBreak 68 Sep 24, 2022
Android hand detect and pose estimation by ncnn

ncnn_nanodet_hand 1.hand detect:用nanode-m训练了个hand detect模型, 2.hand pose:用CMU的数据集训练了个ghostnet作为backbone模仿pfld的handpose模型 3.推理:handpose.cpp单独检测pose,nano

null 76 Sep 19, 2022
Portrait segmentation in your web browser with ncnn and webassembly

ncnn-webassembly-portrait-segmentation open https://nihui.github.io/ncnn-webassembly-portrait-segmentation and enjoy build and deploy Install emscript

null 25 Sep 21, 2022
The SCRFD face detection, depends on ncnn library and opencv

The SCRFD face detection, depends on ncnn library and opencv

null 152 Sep 27, 2022
Android face detect and segmentation by ncnn

Android face detect and segmentation by ncnn

null 104 Oct 1, 2022
RapidOCR ncnn 推理

RapidOCRNcnnCpp Project下载 有整合好源码和依赖库的完整工程项目,文件比较大,可到Q群共享内下载,找以Project开头的压缩包文件 如果想自己折腾,则请继续阅读本说明 介绍 RapidOCR ncnn 推理 模型转换路线: paddle-> onnx -> onnx-simp

RapidOCR Team 14 Jun 13, 2022
ncnn of yolov5 v5.0 branch

YOLOv5转NCNN 基于YOLOv5最新v5.0 release,和NCNN官方给出example的差别主要有: 激活函数hardswish变为siLu; 流程和详细记录u版YOLOv5目标检测ncnn实现略微不同 编译运行 动态库用的是官方编译好的ncnn-20210507-ubuntu-16

null 73 Sep 8, 2022
Android hair/human segmentation demo by ncnn

ncnn_Android_human Android hair/human segmentation demo by ncnn PS:performance maybe poor.it's just a demo:) Reference: 1.https://github.com/Tencent/n

null 23 May 23, 2022
Deploy SCRFD, an efficient high accuracy face detection approach, in your web browser with ncnn and webassembly

ncnn-webassembly-scrfd open https://nihui.github.io/ncnn-webassembly-scrfd and enjoy build and deploy Install emscripten

null 41 Sep 19, 2022
the C++ version of Seq2Seq with ncnn

the C++ version of Seq2Seq with ncnn

DayBreak 21 Aug 25, 2022