(Go: >> BACK << -|- >> HOME <<)
SlideShare a Scribd company logo

Single Image Super Resolution Overview

•
•
LEE HOSEONG
LEE HOSEONG

This document provides an overview of single image super resolution using deep learning. It discusses how super resolution can be used to generate a high resolution image from a low resolution input. Deep learning models like SRCNN were early approaches for super resolution but newer models use deeper networks and perceptual losses. Generative adversarial networks have also been applied to improve perceptual quality. Key applications are in satellite imagery, medical imaging, and video enhancement. Metrics like PSNR and SSIM are commonly used but may not correlate with human perception. Overall, deep learning has advanced super resolution techniques but challenges remain in fully evaluating perceptual quality.

1 of 31
Download to read offline
Intelligence Machine Vision Lab Strictly Confidential Single Image Super Resolution Overview SUALAB Ho Seong Lee
2Type A-3 Contents • What is Super Resolution? • Applications of Super Resolution • Deep Learning for Single Image Super Resolution • Some Issues for Super Resolution
3Type A-3 What is Super Resolution? • Super Resolution • Restore High-Resolution(HR) image(or video) from Low-Resolution(LR) image(or video) • According to the number of input LR images, SR can be classified SISR or MISR • Single Image Super Resolution Efficient & Popular Super Resolution
4Type A-3 What is Super Resolution? • Single Image Super Resolution • Restore High-Resolution(HR) image(or video) from Low-Resolution(LR) image(or video) • Ill-Posed Problem.. (Regular Inverse Problem) → We can’t have ground truth from LR image HR 3 HR 2 HR 1 LR
5Type A-3 What is Super Resolution? • Interpolation-based Single Image Super Resolution • In image upscaling task, bicubic or bilinear or Lanczos interpolation is usually used. • Fast, easy.. but low quality.. Super Resolution Deep SR bilinear
6Type A-3 What is Super Resolution? • Single Image Super Resolution algorithms • Interpolation-based method • Reconstruction-based method • (Deep) Learning-based method • Today, I will cover learning-based method
7Type A-3 Applications of Super Resolution • Satellite image processing • Medical image processing • Multimedia Industry and Video Enhancement Reference: “Super Resolution Applications in Modern Digital Image Processing”, 2016 IJCA TV & Monitor HD(1280x720), FHD(1920x180) UHD(3840x2160)
8Type A-3 Deep Learning for Single Image Super Resolution • Learning-based Single Image Super Resolution • For tackling regular inverse problem, almost use this paradigm • [HR(GT) image] + [distortion & down-sampling] → [LR(input) image] • This is limitation of SISR training • Overall restoration quality is dependent on the distortion & down-sampling method Reference: “Deep Learning for Single Image Super-Resolution: A Brief Review”, 2018 IEEE Transactions on Multimedia (TMM)
9Type A-3 Deep Learning for Single Image Super Resolution • Learning-based Single Image Super Resolution • [HR(GT) image] + [distortion & down-sampling] → [LR(input) image] • In CVPR 2017 SR Challenge, many team showed many degradation of quality metric Reference: http://www.vision.ee.ethz.ch/~timofter/publications/NTIRE2017SRchallenge_factsheets.pdf
10Type A-3 Deep Learning for Single Image Super Resolution • First Deep Learning architecture for Single Image Super Resolution • SRCNN(2014) – three-layer CNN, MSE Loss, Early upsampling • Compared to traditional methods, it shows excellent performance. Reference: “Image Super-Resolution Using Deep Convolutional Networks”, 2014 ECCV
11Type A-3 Deep Learning for Single Image Super Resolution • Efficient Single Image Super Resolution • FSRCNN(2016), ESPCN(2016) • Use Late Upsampling with deconvolution or sub-pixel convolutional layer Inefficient in Memory, FLOPS Reference: “Image Super-Resolution Using Deep Convolutional Networks”, 2014 ECCV
12Type A-3 Deep Learning for Single Image Super Resolution • FSRCNN(Fast Super-Resolution Convolutional Neural Network) • Use Deconvolution layer instead of pre-processing(upsampling) • Faster and more accurate than SRCNN Reference: “Accelerating the Super-Resolution Convolutional Neural Network”, 2016 ECCV
13Type A-3 Deep Learning for Single Image Super Resolution • ESPCN(Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network) • Use sub-pixel convolutional layer (pixel shuffler or depth_to_space) • This sub-pixel convolutional layer is used in recent SR models Reference: “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network”, 2016 CVPR Code: https://github.com/leftthomas/ESPCN
14Type A-3 Deep Learning for Single Image Super Resolution • Deeper Networks for Super-Resolution • SRCNN, FSRCNN, ESPCN are shallow network → Why not deep network? • Failed to train deeper models.. → Use shallow network → how to use deeper network? Reference: “Image Super-Resolution Using Deep Convolutional Networks”, 2014 ECCV
15Type A-3 Deep Learning for Single Image Super Resolution • VDSR(Accurate Image Super-Resolution Using Very Deep Convolutional Networks) • VGG based deeper model(20-layer) for Super-Resolution → large receptive field • Residual learning & High learning rate with gradient clipping • MSE Loss, Early upsampling Reference: “Accurate Image Super-Resolution Using Very Deep Convolutional Networks”, 2016 CVPR
16Type A-3 Deep Learning for Single Image Super Resolution • Deeper Networks for Super-Resolution after VDSR • DRCN(Deeply-recursive Convolutional network), 2016 CVPR • SRResNet, 2017 CVPR • DRRN(Deep Recursive Residual Network), 2017 CVPR Reference: “Deep Learning for Single Image Super-Resolution: A Brief Review”, 2018 IEEE Transactions on Multimedia (TMM)
17Type A-3 Deep Learning for Single Image Super Resolution • Deeper Networks for Super-Resolution after VDSR • EDSR, MDSR (Enhanced Deep Residual Network, Multi Scale EDSR), 2017 CVPRW • DenseSR, 2017 CVPR • MemNet, 2017 CVPR Reference: “Deep Learning for Single Image Super-Resolution: A Brief Review”, 2018 IEEE Transactions on Multimedia (TMM)
18Type A-3 Deep Learning for Single Image Super Resolution • Generative Adversarial Network(GAN) for Super-Resolution • SRGAN(Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network) • First GAN-based SR Model, MSE Loss → Blurry Output → GAN loss + Content loss = Perceptual loss Reference: “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”, 2017 CVPR
19Type A-3 Deep Learning for Single Image Super Resolution • Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network • MSE Loss → Blurry Output → GAN loss + Content loss = Perceptual loss • Replace MSE loss to VGG loss (used in style transfer) and add adversarial loss Reference: “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”, 2017 CVPR
20Type A-3 Deep Learning for Single Image Super Resolution • Generative Adversarial Network(GAN) for Super-Resolution • SRGAN, EnhanceNet, SRFeat, ESRGAN Reference: “A Deep Journey into Super-resolution: A survey”, 2019 arXiv
21Type A-3 Deep Learning for Single Image Super Resolution Reference: “A Deep Journey into Super-resolution: A survey”, 2019 arXiv • Deep Learning for Single Image Super Resolution
22Type A-3 Deep Learning for Single Image Super Resolution Reference: “A Deep Journey into Super-resolution: A survey”, 2019 arXiv
23Type A-3 Deep Learning for Single Image Super Resolution • Deep Learning for Single Image Super Resolution
24Type A-3 Some Issues for Super Resolution • Checkerboard artifact • Deconvolution (Transposed convolution) layer can easily have “uneven overlap” • Simple solution: use “resize + conv” or “sub-pixel convolutional layer” Reference: “Deconvolution and Checkerboard Artifacts”, distill blog(https://distill.pub/2016/deconv-checkerboard/)
25Type A-3 Some Issues for Super Resolution • Loss function • Propose a various loss function methods in Image Restoration task • Report the best result when using mixed loss with MS-SSIM loss + 𝒍 𝟏 loss Reference: “Loss Functions for Image Restoration with Neural Networks”, 2016 IEEE TCI
26Type A-3 Some Issues for Super Resolution • Loss function • Recent papers almost use 𝒍 𝟏 loss Reference: “A Deep Journey into Super-resolution: A survey”, 2019 arXiv
27Type A-3 Some Issues for Super Resolution • Metric (Distortion measure) • Almost paper use distortion metric(PSNR and SSIM) as performance metric • But, high PSNR, SSIM do not guarantee good quality that looks good to human.. Reference: “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”, 2017 CVPR
28Type A-3 Some Issues for Super Resolution • Metric (Human opinion score) • But, high PSNR, SSIM do not guarantee good quality that looks good to human.. • In SRGAN Paper, Mean Opinion Score(MOS) Testing is done for quantify perceptual quality • 26 raters, score from 1(bad) to 5(excellent) • The raters were calibrated on the NN(1) and HR(5) version of 20 images from BSD300 dataset
29Type A-3 Some Issues for Super Resolution • Metric Paper (The Perception-Distortion Tradeoff, 2018 CVPR) • Analysis distortion measure(PSNR, SSIM, etc.) vs human opinion score(Perceptual quality) • Good supplementary video: https://www.youtube.com/watch?v=6Yid4dituqo (PR-12) Reference: “The Perception-Distortion Tradeoff”, 2018 CVPR Better quality
30Type A-3 References • “Image Super-Resolution Using Deep Convolutional Networks”, 2014 ECCV • “Super Resolution Applications in Modern Digital Image Processing”, 2016 IJCA • “Loss Functions for Image Restoration with Neural Networks”, 2016 IEEE TCI • “Deconvolution and Checkerboard Artifacts”, distill blog(https://distill.pub/2016/deconv-checkerboard/) • “Accurate Image Super-Resolution Using Very Deep Convolutional Networks”, 2016 CVPR • “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network”, 2016 CVPR • “Accelerating the Super-Resolution Convolutional Neural Network”, 2016 ECCV • “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”, 2017 CVPR • http://www.vision.ee.ethz.ch/~timofter/publications/NTIRE2017SRchallenge_factsheets.pdf • “The Perception-Distortion Tradeoff”, 2018 CVPR • “Deep Learning for Single Image Super-Resolution: A Brief Review”, 2018 IEEE Transactions on Multimedia (TMM) • “A Deep Journey into Super-resolution: A survey”, 2019 arXiv
Thank you

More Related Content

What's hot

Super resolution
Super resolutionSuper resolution
Super resolution
Federico D'Amato
 
Photo-realistic Single Image Super-resolution using a Generative Adversarial ...
Photo-realistic Single Image Super-resolution using a Generative Adversarial ...Photo-realistic Single Image Super-resolution using a Generative Adversarial ...
Photo-realistic Single Image Super-resolution using a Generative Adversarial ...
Hansol Kang
 
Human Pose Estimation by Deep Learning
Human Pose Estimation by Deep LearningHuman Pose Estimation by Deep Learning
Human Pose Estimation by Deep Learning
Wei Yang
 
Image segmentation with deep learning
Image segmentation with deep learningImage segmentation with deep learning
Image segmentation with deep learning
Antonio Rueda-Toicen
 
Convolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNetConvolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNet
SungminYou
 
Semantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
Semantic Segmentation - Fully Convolutional Networks for Semantic SegmentationSemantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
Semantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
岳華 杜
 
Image classification with Deep Neural Networks
Image classification with Deep Neural NetworksImage classification with Deep Neural Networks
Image classification with Deep Neural Networks
Yogendra Tamang
 
image compression ppt
image compression pptimage compression ppt
image compression ppt
Shivangi Saxena
 
U-Net (1).pptx
U-Net (1).pptxU-Net (1).pptx
U-Net (1).pptx
Changjin Lee
 
Understanding Convolutional Neural Networks
Understanding Convolutional Neural NetworksUnderstanding Convolutional Neural Networks
Understanding Convolutional Neural Networks
Jeremy Nixon
 
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Universitat Politècnica de Catalunya
 
CNN Machine learning DeepLearning
CNN Machine learning DeepLearningCNN Machine learning DeepLearning
CNN Machine learning DeepLearning
Abhishek Sharma
 
Introduction to object detection
Introduction to object detectionIntroduction to object detection
Introduction to object detection
Brodmann17
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
 
Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...
Universitat Politècnica de Catalunya
 
Image classification using CNN
Image classification using CNNImage classification using CNN
Image classification using CNN
Noura Hussein
 
[OSGeo-KR Tech Workshop] Deep Learning for Single Image Super-Resolution
[OSGeo-KR Tech Workshop] Deep Learning for Single Image Super-Resolution[OSGeo-KR Tech Workshop] Deep Learning for Single Image Super-Resolution
[OSGeo-KR Tech Workshop] Deep Learning for Single Image Super-Resolution
Taegyun Jeon
 
Object detection with deep learning
Object detection with deep learningObject detection with deep learning
Object detection with deep learning
Sushant Shrivastava
 
Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep Learning
Mohamed Loey
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
Chanuk Lim
 

What's hot (20)

Super resolution
Super resolutionSuper resolution
Super resolution
 
Photo-realistic Single Image Super-resolution using a Generative Adversarial ...
Photo-realistic Single Image Super-resolution using a Generative Adversarial ...Photo-realistic Single Image Super-resolution using a Generative Adversarial ...
Photo-realistic Single Image Super-resolution using a Generative Adversarial ...
 
Human Pose Estimation by Deep Learning
Human Pose Estimation by Deep LearningHuman Pose Estimation by Deep Learning
Human Pose Estimation by Deep Learning
 
Image segmentation with deep learning
Image segmentation with deep learningImage segmentation with deep learning
Image segmentation with deep learning
 
Convolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNetConvolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNet
 
Semantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
Semantic Segmentation - Fully Convolutional Networks for Semantic SegmentationSemantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
Semantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
 
Image classification with Deep Neural Networks
Image classification with Deep Neural NetworksImage classification with Deep Neural Networks
Image classification with Deep Neural Networks
 
image compression ppt
image compression pptimage compression ppt
image compression ppt
 
U-Net (1).pptx
U-Net (1).pptxU-Net (1).pptx
U-Net (1).pptx
 
Understanding Convolutional Neural Networks
Understanding Convolutional Neural NetworksUnderstanding Convolutional Neural Networks
Understanding Convolutional Neural Networks
 
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
 
CNN Machine learning DeepLearning
CNN Machine learning DeepLearningCNN Machine learning DeepLearning
CNN Machine learning DeepLearning
 
Introduction to object detection
Introduction to object detectionIntroduction to object detection
Introduction to object detection
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
 
Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...
 
Image classification using CNN
Image classification using CNNImage classification using CNN
Image classification using CNN
 
[OSGeo-KR Tech Workshop] Deep Learning for Single Image Super-Resolution
[OSGeo-KR Tech Workshop] Deep Learning for Single Image Super-Resolution[OSGeo-KR Tech Workshop] Deep Learning for Single Image Super-Resolution
[OSGeo-KR Tech Workshop] Deep Learning for Single Image Super-Resolution
 
Object detection with deep learning
Object detection with deep learningObject detection with deep learning
Object detection with deep learning
 
Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep Learning
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
 

Similar to Single Image Super Resolution Overview

A Fully Progressive approach to Single image super-resolution
A Fully Progressive approach to Single image super-resolution A Fully Progressive approach to Single image super-resolution
A Fully Progressive approach to Single image super-resolution
Mohammed Ashour
 
Single Image Super Resolution using Fuzzy Deep Convolutional Networks
Single Image Super Resolution using Fuzzy Deep Convolutional NetworksSingle Image Super Resolution using Fuzzy Deep Convolutional Networks
Single Image Super Resolution using Fuzzy Deep Convolutional Networks
Greeshma M.S.R
 
Scaling up Deep Learning Based Super Resolution Algorithms
Scaling up Deep Learning Based Super Resolution AlgorithmsScaling up Deep Learning Based Super Resolution Algorithms
Scaling up Deep Learning Based Super Resolution Algorithms
Xiaoyong Zhu
 
Full resolution image compression with recurrent neural networks
Full resolution image compression with recurrent neural networksFull resolution image compression with recurrent neural networks
Full resolution image compression with recurrent neural networks
Ashis Kumar Chanda
 
Super resolution-review
Super resolution-reviewSuper resolution-review
Super resolution-review
Woojin Jeong
 
Full resolution image compression with recurrent neural networks
Full resolution image compression with recurrent neural networksFull resolution image compression with recurrent neural networks
Full resolution image compression with recurrent neural networks
Ashis Chanda
 
Survey of Super Resolution Task (SISR Only)
Survey of Super Resolution Task (SISR Only)Survey of Super Resolution Task (SISR Only)
Survey of Super Resolution Task (SISR Only)
MYEONGGYU LEE
 
Survey on Single image Super Resolution Techniques
Survey on Single image Super Resolution TechniquesSurvey on Single image Super Resolution Techniques
Survey on Single image Super Resolution Techniques
IOSR Journals
 
Survey on Single image Super Resolution Techniques
Survey on Single image Super Resolution TechniquesSurvey on Single image Super Resolution Techniques
Survey on Single image Super Resolution Techniques
IOSR Journals
 
Seminarpaper
SeminarpaperSeminarpaper
Seminarpaper
PrashantChaudhari75
 
2019 cvpr paper overview by Ho Seong Lee
2019 cvpr paper overview by Ho Seong Lee2019 cvpr paper overview by Ho Seong Lee
2019 cvpr paper overview by Ho Seong Lee
Moazzem Hossain
 
2019 cvpr paper_overview
2019 cvpr paper_overview2019 cvpr paper_overview
2019 cvpr paper_overview
LEE HOSEONG
 
Recent Progress on Object Detection_20170331
Recent Progress on Object Detection_20170331Recent Progress on Object Detection_20170331
Recent Progress on Object Detection_20170331
Jihong Kang
 
Enhance Example-Based Super Resolution to Achieve Fine Magnification of Low ...
Enhance Example-Based Super Resolution to Achieve Fine Magnification of Low ...Enhance Example-Based Super Resolution to Achieve Fine Magnification of Low ...
Enhance Example-Based Super Resolution to Achieve Fine Magnification of Low ...
IJMER
 
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"..."How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
Edge AI and Vision Alliance
 
Single Image Super-Resolution Using Analytical Solution for L2-L2 Algorithm
Single Image Super-Resolution Using Analytical Solution for L2-L2 AlgorithmSingle Image Super-Resolution Using Analytical Solution for L2-L2 Algorithm
Single Image Super-Resolution Using Analytical Solution for L2-L2 Algorithm
ijtsrd
 
SeRanet introduction
SeRanet introductionSeRanet introduction
SeRanet introduction
Kosuke Nakago
 
Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Anal...
Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Anal...Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Anal...
Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Anal...
JaeJun Yoo
 
Image Segmentation Using Deep Learning : A survey
Image Segmentation Using Deep Learning : A surveyImage Segmentation Using Deep Learning : A survey
Image Segmentation Using Deep Learning : A survey
NUPUR YADAV
 
Obscenity Detection in Images
Obscenity Detection in ImagesObscenity Detection in Images
Obscenity Detection in Images
Anil Kumar Gupta
 

Similar to Single Image Super Resolution Overview (20)

A Fully Progressive approach to Single image super-resolution
A Fully Progressive approach to Single image super-resolution A Fully Progressive approach to Single image super-resolution
A Fully Progressive approach to Single image super-resolution
 
Single Image Super Resolution using Fuzzy Deep Convolutional Networks
Single Image Super Resolution using Fuzzy Deep Convolutional NetworksSingle Image Super Resolution using Fuzzy Deep Convolutional Networks
Single Image Super Resolution using Fuzzy Deep Convolutional Networks
 
Scaling up Deep Learning Based Super Resolution Algorithms
Scaling up Deep Learning Based Super Resolution AlgorithmsScaling up Deep Learning Based Super Resolution Algorithms
Scaling up Deep Learning Based Super Resolution Algorithms
 
Full resolution image compression with recurrent neural networks
Full resolution image compression with recurrent neural networksFull resolution image compression with recurrent neural networks
Full resolution image compression with recurrent neural networks
 
Super resolution-review
Super resolution-reviewSuper resolution-review
Super resolution-review
 
Full resolution image compression with recurrent neural networks
Full resolution image compression with recurrent neural networksFull resolution image compression with recurrent neural networks
Full resolution image compression with recurrent neural networks
 
Survey of Super Resolution Task (SISR Only)
Survey of Super Resolution Task (SISR Only)Survey of Super Resolution Task (SISR Only)
Survey of Super Resolution Task (SISR Only)
 
Survey on Single image Super Resolution Techniques
Survey on Single image Super Resolution TechniquesSurvey on Single image Super Resolution Techniques
Survey on Single image Super Resolution Techniques
 
Survey on Single image Super Resolution Techniques
Survey on Single image Super Resolution TechniquesSurvey on Single image Super Resolution Techniques
Survey on Single image Super Resolution Techniques
 
Seminarpaper
SeminarpaperSeminarpaper
Seminarpaper
 
2019 cvpr paper overview by Ho Seong Lee
2019 cvpr paper overview by Ho Seong Lee2019 cvpr paper overview by Ho Seong Lee
2019 cvpr paper overview by Ho Seong Lee
 
2019 cvpr paper_overview
2019 cvpr paper_overview2019 cvpr paper_overview
2019 cvpr paper_overview
 
Recent Progress on Object Detection_20170331
Recent Progress on Object Detection_20170331Recent Progress on Object Detection_20170331
Recent Progress on Object Detection_20170331
 
Enhance Example-Based Super Resolution to Achieve Fine Magnification of Low ...
Enhance Example-Based Super Resolution to Achieve Fine Magnification of Low ...Enhance Example-Based Super Resolution to Achieve Fine Magnification of Low ...
Enhance Example-Based Super Resolution to Achieve Fine Magnification of Low ...
 
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"..."How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
 
Single Image Super-Resolution Using Analytical Solution for L2-L2 Algorithm
Single Image Super-Resolution Using Analytical Solution for L2-L2 AlgorithmSingle Image Super-Resolution Using Analytical Solution for L2-L2 Algorithm
Single Image Super-Resolution Using Analytical Solution for L2-L2 Algorithm
 
SeRanet introduction
SeRanet introductionSeRanet introduction
SeRanet introduction
 
Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Anal...
Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Anal...Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Anal...
Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Anal...
 
Image Segmentation Using Deep Learning : A survey
Image Segmentation Using Deep Learning : A surveyImage Segmentation Using Deep Learning : A survey
Image Segmentation Using Deep Learning : A survey
 
Obscenity Detection in Images
Obscenity Detection in ImagesObscenity Detection in Images
Obscenity Detection in Images
 

More from LEE HOSEONG

Unsupervised anomaly detection using style distillation
Unsupervised anomaly detection using style distillationUnsupervised anomaly detection using style distillation
Unsupervised anomaly detection using style distillation
LEE HOSEONG
 
do adversarially robust image net models transfer better
do adversarially robust image net models transfer betterdo adversarially robust image net models transfer better
do adversarially robust image net models transfer better
LEE HOSEONG
 
CNN Architecture A to Z
CNN Architecture A to ZCNN Architecture A to Z
CNN Architecture A to Z
LEE HOSEONG
 
carrier of_tricks_for_image_classification
carrier of_tricks_for_image_classificationcarrier of_tricks_for_image_classification
carrier of_tricks_for_image_classification
LEE HOSEONG
 
"The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Gen...
"The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Gen..."The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Gen...
"The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Gen...
LEE HOSEONG
 
Mixed Precision Training Review
Mixed Precision Training ReviewMixed Precision Training Review
Mixed Precision Training Review
LEE HOSEONG
 
MVTec AD: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection
MVTec AD: A Comprehensive Real-World Dataset for Unsupervised Anomaly DetectionMVTec AD: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection
MVTec AD: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection
LEE HOSEONG
 
YOLOv4: optimal speed and accuracy of object detection review
YOLOv4: optimal speed and accuracy of object detection reviewYOLOv4: optimal speed and accuracy of object detection review
YOLOv4: optimal speed and accuracy of object detection review
LEE HOSEONG
 
FixMatch:simplifying semi supervised learning with consistency and confidence
FixMatch:simplifying semi supervised learning with consistency and confidenceFixMatch:simplifying semi supervised learning with consistency and confidence
FixMatch:simplifying semi supervised learning with consistency and confidence
LEE HOSEONG
 
"Revisiting self supervised visual representation learning" Paper Review
"Revisiting self supervised visual representation learning" Paper Review"Revisiting self supervised visual representation learning" Paper Review
"Revisiting self supervised visual representation learning" Paper Review
LEE HOSEONG
 
Unsupervised visual representation learning overview: Toward Self-Supervision
Unsupervised visual representation learning overview: Toward Self-SupervisionUnsupervised visual representation learning overview: Toward Self-Supervision
Unsupervised visual representation learning overview: Toward Self-Supervision
LEE HOSEONG
 
Human uncertainty makes classification more robust, ICCV 2019 Review
Human uncertainty makes classification more robust, ICCV 2019 ReviewHuman uncertainty makes classification more robust, ICCV 2019 Review
Human uncertainty makes classification more robust, ICCV 2019 Review
LEE HOSEONG
 
2019 ICLR Best Paper Review
2019 ICLR Best Paper Review2019 ICLR Best Paper Review
2019 ICLR Best Paper Review
LEE HOSEONG
 
"Google Vizier: A Service for Black-Box Optimization" Paper Review
"Google Vizier: A Service for Black-Box Optimization" Paper Review"Google Vizier: A Service for Black-Box Optimization" Paper Review
"Google Vizier: A Service for Black-Box Optimization" Paper Review
LEE HOSEONG
 
"Searching for Activation Functions" Paper Review
"Searching for Activation Functions" Paper Review"Searching for Activation Functions" Paper Review
"Searching for Activation Functions" Paper Review
LEE HOSEONG
 
"Learning transferable architectures for scalable image recognition" Paper Re...
"Learning transferable architectures for scalable image recognition" Paper Re..."Learning transferable architectures for scalable image recognition" Paper Re...
"Learning transferable architectures for scalable image recognition" Paper Re...
LEE HOSEONG
 
"Learning From Noisy Large-Scale Datasets With Minimal Supervision" Paper Review
"Learning From Noisy Large-Scale Datasets With Minimal Supervision" Paper Review"Learning From Noisy Large-Scale Datasets With Minimal Supervision" Paper Review
"Learning From Noisy Large-Scale Datasets With Minimal Supervision" Paper Review
LEE HOSEONG
 
"Dataset and metrics for predicting local visible differences" Paper Review
"Dataset and metrics for predicting local visible differences" Paper Review"Dataset and metrics for predicting local visible differences" Paper Review
"Dataset and metrics for predicting local visible differences" Paper Review
LEE HOSEONG
 
"From image level to pixel-level labeling with convolutional networks" Paper ...
"From image level to pixel-level labeling with convolutional networks" Paper ..."From image level to pixel-level labeling with convolutional networks" Paper ...
"From image level to pixel-level labeling with convolutional networks" Paper ...
LEE HOSEONG
 
"simple does it weakly supervised instance and semantic segmentation" Paper r...
"simple does it weakly supervised instance and semantic segmentation" Paper r..."simple does it weakly supervised instance and semantic segmentation" Paper r...
"simple does it weakly supervised instance and semantic segmentation" Paper r...
LEE HOSEONG
 

More from LEE HOSEONG (20)

Unsupervised anomaly detection using style distillation
Unsupervised anomaly detection using style distillationUnsupervised anomaly detection using style distillation
Unsupervised anomaly detection using style distillation
 
do adversarially robust image net models transfer better
do adversarially robust image net models transfer betterdo adversarially robust image net models transfer better
do adversarially robust image net models transfer better
 
CNN Architecture A to Z
CNN Architecture A to ZCNN Architecture A to Z
CNN Architecture A to Z
 
carrier of_tricks_for_image_classification
carrier of_tricks_for_image_classificationcarrier of_tricks_for_image_classification
carrier of_tricks_for_image_classification
 
"The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Gen...
"The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Gen..."The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Gen...
"The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Gen...
 
Mixed Precision Training Review
Mixed Precision Training ReviewMixed Precision Training Review
Mixed Precision Training Review
 
MVTec AD: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection
MVTec AD: A Comprehensive Real-World Dataset for Unsupervised Anomaly DetectionMVTec AD: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection
MVTec AD: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection
 
YOLOv4: optimal speed and accuracy of object detection review
YOLOv4: optimal speed and accuracy of object detection reviewYOLOv4: optimal speed and accuracy of object detection review
YOLOv4: optimal speed and accuracy of object detection review
 
FixMatch:simplifying semi supervised learning with consistency and confidence
FixMatch:simplifying semi supervised learning with consistency and confidenceFixMatch:simplifying semi supervised learning with consistency and confidence
FixMatch:simplifying semi supervised learning with consistency and confidence
 
"Revisiting self supervised visual representation learning" Paper Review
"Revisiting self supervised visual representation learning" Paper Review"Revisiting self supervised visual representation learning" Paper Review
"Revisiting self supervised visual representation learning" Paper Review
 
Unsupervised visual representation learning overview: Toward Self-Supervision
Unsupervised visual representation learning overview: Toward Self-SupervisionUnsupervised visual representation learning overview: Toward Self-Supervision
Unsupervised visual representation learning overview: Toward Self-Supervision
 
Human uncertainty makes classification more robust, ICCV 2019 Review
Human uncertainty makes classification more robust, ICCV 2019 ReviewHuman uncertainty makes classification more robust, ICCV 2019 Review
Human uncertainty makes classification more robust, ICCV 2019 Review
 
2019 ICLR Best Paper Review
2019 ICLR Best Paper Review2019 ICLR Best Paper Review
2019 ICLR Best Paper Review
 
"Google Vizier: A Service for Black-Box Optimization" Paper Review
"Google Vizier: A Service for Black-Box Optimization" Paper Review"Google Vizier: A Service for Black-Box Optimization" Paper Review
"Google Vizier: A Service for Black-Box Optimization" Paper Review
 
"Searching for Activation Functions" Paper Review
"Searching for Activation Functions" Paper Review"Searching for Activation Functions" Paper Review
"Searching for Activation Functions" Paper Review
 
"Learning transferable architectures for scalable image recognition" Paper Re...
"Learning transferable architectures for scalable image recognition" Paper Re..."Learning transferable architectures for scalable image recognition" Paper Re...
"Learning transferable architectures for scalable image recognition" Paper Re...
 
"Learning From Noisy Large-Scale Datasets With Minimal Supervision" Paper Review
"Learning From Noisy Large-Scale Datasets With Minimal Supervision" Paper Review"Learning From Noisy Large-Scale Datasets With Minimal Supervision" Paper Review
"Learning From Noisy Large-Scale Datasets With Minimal Supervision" Paper Review
 
"Dataset and metrics for predicting local visible differences" Paper Review
"Dataset and metrics for predicting local visible differences" Paper Review"Dataset and metrics for predicting local visible differences" Paper Review
"Dataset and metrics for predicting local visible differences" Paper Review
 
"From image level to pixel-level labeling with convolutional networks" Paper ...
"From image level to pixel-level labeling with convolutional networks" Paper ..."From image level to pixel-level labeling with convolutional networks" Paper ...
"From image level to pixel-level labeling with convolutional networks" Paper ...
 
"simple does it weakly supervised instance and semantic segmentation" Paper r...
"simple does it weakly supervised instance and semantic segmentation" Paper r..."simple does it weakly supervised instance and semantic segmentation" Paper r...
"simple does it weakly supervised instance and semantic segmentation" Paper r...
 

Recently uploaded

Interaction Latency: Square's User-Centric Mobile Performance Metric
Interaction Latency: Square's User-Centric Mobile Performance MetricInteraction Latency: Square's User-Centric Mobile Performance Metric
Interaction Latency: Square's User-Centric Mobile Performance Metric
ScyllaDB
 
Calgary MuleSoft Meetup APM and IDP .pptx
Calgary MuleSoft Meetup APM and IDP .pptxCalgary MuleSoft Meetup APM and IDP .pptx
Calgary MuleSoft Meetup APM and IDP .pptx
ishalveerrandhawa1
 
UiPath Community Day Kraków: Devs4Devs Conference
UiPath Community Day Kraków: Devs4Devs ConferenceUiPath Community Day Kraków: Devs4Devs Conference
UiPath Community Day Kraków: Devs4Devs Conference
UiPathCommunity
 
Running a Go App in Kubernetes: CPU Impacts
Running a Go App in Kubernetes: CPU ImpactsRunning a Go App in Kubernetes: CPU Impacts
Running a Go App in Kubernetes: CPU Impacts
ScyllaDB
 
Cookies program to display the information though cookie creation
Cookies program to display the information though cookie creationCookies program to display the information though cookie creation
Cookies program to display the information though cookie creation
shanthidl1
 
Pigging Solutions Sustainability brochure.pdf
Pigging Solutions Sustainability brochure.pdfPigging Solutions Sustainability brochure.pdf
Pigging Solutions Sustainability brochure.pdf
Pigging Solutions
 
What Not to Document and Why_ (North Bay Python 2024)
What Not to Document and Why_ (North Bay Python 2024)What Not to Document and Why_ (North Bay Python 2024)
What Not to Document and Why_ (North Bay Python 2024)
Margaret Fero
 
Knowledge and Prompt Engineering Part 2 Focus on Prompt Design Approaches
Knowledge and Prompt Engineering Part 2 Focus on Prompt Design ApproachesKnowledge and Prompt Engineering Part 2 Focus on Prompt Design Approaches
Knowledge and Prompt Engineering Part 2 Focus on Prompt Design Approaches
Earley Information Science
 
Navigating Post-Quantum Blockchain: Resilient Cryptography in Quantum Threats
Navigating Post-Quantum Blockchain: Resilient Cryptography in Quantum ThreatsNavigating Post-Quantum Blockchain: Resilient Cryptography in Quantum Threats
Navigating Post-Quantum Blockchain: Resilient Cryptography in Quantum Threats
anupriti
 
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsScaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Mydbops
 
5G bootcamp Sep 2020 (NPI initiative).pptx
5G bootcamp Sep 2020 (NPI initiative).pptx5G bootcamp Sep 2020 (NPI initiative).pptx
5G bootcamp Sep 2020 (NPI initiative).pptx
SATYENDRA100
 
Observability For You and Me with OpenTelemetry
Observability For You and Me with OpenTelemetryObservability For You and Me with OpenTelemetry
Observability For You and Me with OpenTelemetry
Eric D. Schabell
 
MYIR Product Brochure - A Global Provider of Embedded SOMs & Solutions
MYIR Product Brochure - A Global Provider of Embedded SOMs & SolutionsMYIR Product Brochure - A Global Provider of Embedded SOMs & Solutions
MYIR Product Brochure - A Global Provider of Embedded SOMs & Solutions
Linda Zhang
 
Data Protection in a Connected World: Sovereignty and Cyber Security
Data Protection in a Connected World: Sovereignty and Cyber SecurityData Protection in a Connected World: Sovereignty and Cyber Security
Data Protection in a Connected World: Sovereignty and Cyber Security
anupriti
 
Transcript: Details of description part II: Describing images in practice - T...
Transcript: Details of description part II: Describing images in practice - T...Transcript: Details of description part II: Describing images in practice - T...
Transcript: Details of description part II: Describing images in practice - T...
BookNet Canada
 
Coordinate Systems in FME 101 - Webinar Slides
Coordinate Systems in FME 101 - Webinar SlidesCoordinate Systems in FME 101 - Webinar Slides
Coordinate Systems in FME 101 - Webinar Slides
Safe Software
 
Implementations of Fused Deposition Modeling in real world
Implementations of Fused Deposition Modeling in real worldImplementations of Fused Deposition Modeling in real world
Implementations of Fused Deposition Modeling in real world
Emerging Tech
 
@Call @Girls Pune 0000000000 Riya Khan Beautiful Girl any Time
@Call @Girls Pune 0000000000 Riya Khan Beautiful Girl any Time@Call @Girls Pune 0000000000 Riya Khan Beautiful Girl any Time
@Call @Girls Pune 0000000000 Riya Khan Beautiful Girl any Time
amitchopra0215
 
Blockchain and Cyber Defense Strategies in new genre times
Blockchain and Cyber Defense Strategies in new genre timesBlockchain and Cyber Defense Strategies in new genre times
Blockchain and Cyber Defense Strategies in new genre times
anupriti
 
一比一原版(msvu毕业证书）圣文森山大学毕业证如何办理
一比一原版(msvu毕业证书）圣文森山大学毕业证如何办理一比一原版(msvu毕业证书）圣文森山大学毕业证如何办理
一比一原版(msvu毕业证书）圣文森山大学毕业证如何办理
uuuot
 

Recently uploaded (20)

Interaction Latency: Square's User-Centric Mobile Performance Metric
Interaction Latency: Square's User-Centric Mobile Performance MetricInteraction Latency: Square's User-Centric Mobile Performance Metric
Interaction Latency: Square's User-Centric Mobile Performance Metric
 
Calgary MuleSoft Meetup APM and IDP .pptx
Calgary MuleSoft Meetup APM and IDP .pptxCalgary MuleSoft Meetup APM and IDP .pptx
Calgary MuleSoft Meetup APM and IDP .pptx
 
UiPath Community Day Kraków: Devs4Devs Conference
UiPath Community Day Kraków: Devs4Devs ConferenceUiPath Community Day Kraków: Devs4Devs Conference
UiPath Community Day Kraków: Devs4Devs Conference
 
Running a Go App in Kubernetes: CPU Impacts
Running a Go App in Kubernetes: CPU ImpactsRunning a Go App in Kubernetes: CPU Impacts
Running a Go App in Kubernetes: CPU Impacts
 
Cookies program to display the information though cookie creation
Cookies program to display the information though cookie creationCookies program to display the information though cookie creation
Cookies program to display the information though cookie creation
 
Pigging Solutions Sustainability brochure.pdf
Pigging Solutions Sustainability brochure.pdfPigging Solutions Sustainability brochure.pdf
Pigging Solutions Sustainability brochure.pdf
 
What Not to Document and Why_ (North Bay Python 2024)
What Not to Document and Why_ (North Bay Python 2024)What Not to Document and Why_ (North Bay Python 2024)
What Not to Document and Why_ (North Bay Python 2024)
 
Knowledge and Prompt Engineering Part 2 Focus on Prompt Design Approaches
Knowledge and Prompt Engineering Part 2 Focus on Prompt Design ApproachesKnowledge and Prompt Engineering Part 2 Focus on Prompt Design Approaches
Knowledge and Prompt Engineering Part 2 Focus on Prompt Design Approaches
 
Navigating Post-Quantum Blockchain: Resilient Cryptography in Quantum Threats
Navigating Post-Quantum Blockchain: Resilient Cryptography in Quantum ThreatsNavigating Post-Quantum Blockchain: Resilient Cryptography in Quantum Threats
Navigating Post-Quantum Blockchain: Resilient Cryptography in Quantum Threats
 
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsScaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
 
5G bootcamp Sep 2020 (NPI initiative).pptx
5G bootcamp Sep 2020 (NPI initiative).pptx5G bootcamp Sep 2020 (NPI initiative).pptx
5G bootcamp Sep 2020 (NPI initiative).pptx
 
Observability For You and Me with OpenTelemetry
Observability For You and Me with OpenTelemetryObservability For You and Me with OpenTelemetry
Observability For You and Me with OpenTelemetry
 
MYIR Product Brochure - A Global Provider of Embedded SOMs & Solutions
MYIR Product Brochure - A Global Provider of Embedded SOMs & SolutionsMYIR Product Brochure - A Global Provider of Embedded SOMs & Solutions
MYIR Product Brochure - A Global Provider of Embedded SOMs & Solutions
 
Data Protection in a Connected World: Sovereignty and Cyber Security
Data Protection in a Connected World: Sovereignty and Cyber SecurityData Protection in a Connected World: Sovereignty and Cyber Security
Data Protection in a Connected World: Sovereignty and Cyber Security
 
Transcript: Details of description part II: Describing images in practice - T...
Transcript: Details of description part II: Describing images in practice - T...Transcript: Details of description part II: Describing images in practice - T...
Transcript: Details of description part II: Describing images in practice - T...
 
Coordinate Systems in FME 101 - Webinar Slides
Coordinate Systems in FME 101 - Webinar SlidesCoordinate Systems in FME 101 - Webinar Slides
Coordinate Systems in FME 101 - Webinar Slides
 
Implementations of Fused Deposition Modeling in real world
Implementations of Fused Deposition Modeling in real worldImplementations of Fused Deposition Modeling in real world
Implementations of Fused Deposition Modeling in real world
 
@Call @Girls Pune 0000000000 Riya Khan Beautiful Girl any Time
@Call @Girls Pune 0000000000 Riya Khan Beautiful Girl any Time@Call @Girls Pune 0000000000 Riya Khan Beautiful Girl any Time
@Call @Girls Pune 0000000000 Riya Khan Beautiful Girl any Time
 
Blockchain and Cyber Defense Strategies in new genre times
Blockchain and Cyber Defense Strategies in new genre timesBlockchain and Cyber Defense Strategies in new genre times
Blockchain and Cyber Defense Strategies in new genre times
 
一比一原版(msvu毕业证书）圣文森山大学毕业证如何办理
一比一原版(msvu毕业证书）圣文森山大学毕业证如何办理一比一原版(msvu毕业证书）圣文森山大学毕业证如何办理
一比一原版(msvu毕业证书）圣文森山大学毕业证如何办理
 

Single Image Super Resolution Overview

  • 1. Intelligence Machine Vision Lab Strictly Confidential Single Image Super Resolution Overview SUALAB Ho Seong Lee
  • 2. 2Type A-3 Contents • What is Super Resolution? • Applications of Super Resolution • Deep Learning for Single Image Super Resolution • Some Issues for Super Resolution
  • 3. 3Type A-3 What is Super Resolution? • Super Resolution • Restore High-Resolution(HR) image(or video) from Low-Resolution(LR) image(or video) • According to the number of input LR images, SR can be classified SISR or MISR • Single Image Super Resolution Efficient & Popular Super Resolution
  • 4. 4Type A-3 What is Super Resolution? • Single Image Super Resolution • Restore High-Resolution(HR) image(or video) from Low-Resolution(LR) image(or video) • Ill-Posed Problem.. (Regular Inverse Problem) → We can’t have ground truth from LR image HR 3 HR 2 HR 1 LR
  • 5. 5Type A-3 What is Super Resolution? • Interpolation-based Single Image Super Resolution • In image upscaling task, bicubic or bilinear or Lanczos interpolation is usually used. • Fast, easy.. but low quality.. Super Resolution Deep SR bilinear
  • 6. 6Type A-3 What is Super Resolution? • Single Image Super Resolution algorithms • Interpolation-based method • Reconstruction-based method • (Deep) Learning-based method • Today, I will cover learning-based method
  • 7. 7Type A-3 Applications of Super Resolution • Satellite image processing • Medical image processing • Multimedia Industry and Video Enhancement Reference: “Super Resolution Applications in Modern Digital Image Processing”, 2016 IJCA TV & Monitor HD(1280x720), FHD(1920x180) UHD(3840x2160)
  • 8. 8Type A-3 Deep Learning for Single Image Super Resolution • Learning-based Single Image Super Resolution • For tackling regular inverse problem, almost use this paradigm • [HR(GT) image] + [distortion & down-sampling] → [LR(input) image] • This is limitation of SISR training • Overall restoration quality is dependent on the distortion & down-sampling method Reference: “Deep Learning for Single Image Super-Resolution: A Brief Review”, 2018 IEEE Transactions on Multimedia (TMM)
  • 9. 9Type A-3 Deep Learning for Single Image Super Resolution • Learning-based Single Image Super Resolution • [HR(GT) image] + [distortion & down-sampling] → [LR(input) image] • In CVPR 2017 SR Challenge, many team showed many degradation of quality metric Reference: http://www.vision.ee.ethz.ch/~timofter/publications/NTIRE2017SRchallenge_factsheets.pdf
  • 10. 10Type A-3 Deep Learning for Single Image Super Resolution • First Deep Learning architecture for Single Image Super Resolution • SRCNN(2014) – three-layer CNN, MSE Loss, Early upsampling • Compared to traditional methods, it shows excellent performance. Reference: “Image Super-Resolution Using Deep Convolutional Networks”, 2014 ECCV
  • 11. 11Type A-3 Deep Learning for Single Image Super Resolution • Efficient Single Image Super Resolution • FSRCNN(2016), ESPCN(2016) • Use Late Upsampling with deconvolution or sub-pixel convolutional layer Inefficient in Memory, FLOPS Reference: “Image Super-Resolution Using Deep Convolutional Networks”, 2014 ECCV
  • 12. 12Type A-3 Deep Learning for Single Image Super Resolution • FSRCNN(Fast Super-Resolution Convolutional Neural Network) • Use Deconvolution layer instead of pre-processing(upsampling) • Faster and more accurate than SRCNN Reference: “Accelerating the Super-Resolution Convolutional Neural Network”, 2016 ECCV
  • 13. 13Type A-3 Deep Learning for Single Image Super Resolution • ESPCN(Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network) • Use sub-pixel convolutional layer (pixel shuffler or depth_to_space) • This sub-pixel convolutional layer is used in recent SR models Reference: “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network”, 2016 CVPR Code: https://github.com/leftthomas/ESPCN
  • 14. 14Type A-3 Deep Learning for Single Image Super Resolution • Deeper Networks for Super-Resolution • SRCNN, FSRCNN, ESPCN are shallow network → Why not deep network? • Failed to train deeper models.. → Use shallow network → how to use deeper network? Reference: “Image Super-Resolution Using Deep Convolutional Networks”, 2014 ECCV
  • 15. 15Type A-3 Deep Learning for Single Image Super Resolution • VDSR(Accurate Image Super-Resolution Using Very Deep Convolutional Networks) • VGG based deeper model(20-layer) for Super-Resolution → large receptive field • Residual learning & High learning rate with gradient clipping • MSE Loss, Early upsampling Reference: “Accurate Image Super-Resolution Using Very Deep Convolutional Networks”, 2016 CVPR
  • 16. 16Type A-3 Deep Learning for Single Image Super Resolution • Deeper Networks for Super-Resolution after VDSR • DRCN(Deeply-recursive Convolutional network), 2016 CVPR • SRResNet, 2017 CVPR • DRRN(Deep Recursive Residual Network), 2017 CVPR Reference: “Deep Learning for Single Image Super-Resolution: A Brief Review”, 2018 IEEE Transactions on Multimedia (TMM)
  • 17. 17Type A-3 Deep Learning for Single Image Super Resolution • Deeper Networks for Super-Resolution after VDSR • EDSR, MDSR (Enhanced Deep Residual Network, Multi Scale EDSR), 2017 CVPRW • DenseSR, 2017 CVPR • MemNet, 2017 CVPR Reference: “Deep Learning for Single Image Super-Resolution: A Brief Review”, 2018 IEEE Transactions on Multimedia (TMM)
  • 18. 18Type A-3 Deep Learning for Single Image Super Resolution • Generative Adversarial Network(GAN) for Super-Resolution • SRGAN(Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network) • First GAN-based SR Model, MSE Loss → Blurry Output → GAN loss + Content loss = Perceptual loss Reference: “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”, 2017 CVPR
  • 19. 19Type A-3 Deep Learning for Single Image Super Resolution • Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network • MSE Loss → Blurry Output → GAN loss + Content loss = Perceptual loss • Replace MSE loss to VGG loss (used in style transfer) and add adversarial loss Reference: “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”, 2017 CVPR
  • 20. 20Type A-3 Deep Learning for Single Image Super Resolution • Generative Adversarial Network(GAN) for Super-Resolution • SRGAN, EnhanceNet, SRFeat, ESRGAN Reference: “A Deep Journey into Super-resolution: A survey”, 2019 arXiv
  • 21. 21Type A-3 Deep Learning for Single Image Super Resolution Reference: “A Deep Journey into Super-resolution: A survey”, 2019 arXiv • Deep Learning for Single Image Super Resolution
  • 22. 22Type A-3 Deep Learning for Single Image Super Resolution Reference: “A Deep Journey into Super-resolution: A survey”, 2019 arXiv
  • 23. 23Type A-3 Deep Learning for Single Image Super Resolution • Deep Learning for Single Image Super Resolution
  • 24. 24Type A-3 Some Issues for Super Resolution • Checkerboard artifact • Deconvolution (Transposed convolution) layer can easily have “uneven overlap” • Simple solution: use “resize + conv” or “sub-pixel convolutional layer” Reference: “Deconvolution and Checkerboard Artifacts”, distill blog(https://distill.pub/2016/deconv-checkerboard/)
  • 25. 25Type A-3 Some Issues for Super Resolution • Loss function • Propose a various loss function methods in Image Restoration task • Report the best result when using mixed loss with MS-SSIM loss + 𝒍 𝟏 loss Reference: “Loss Functions for Image Restoration with Neural Networks”, 2016 IEEE TCI
  • 26. 26Type A-3 Some Issues for Super Resolution • Loss function • Recent papers almost use 𝒍 𝟏 loss Reference: “A Deep Journey into Super-resolution: A survey”, 2019 arXiv
  • 27. 27Type A-3 Some Issues for Super Resolution • Metric (Distortion measure) • Almost paper use distortion metric(PSNR and SSIM) as performance metric • But, high PSNR, SSIM do not guarantee good quality that looks good to human.. Reference: “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”, 2017 CVPR
  • 28. 28Type A-3 Some Issues for Super Resolution • Metric (Human opinion score) • But, high PSNR, SSIM do not guarantee good quality that looks good to human.. • In SRGAN Paper, Mean Opinion Score(MOS) Testing is done for quantify perceptual quality • 26 raters, score from 1(bad) to 5(excellent) • The raters were calibrated on the NN(1) and HR(5) version of 20 images from BSD300 dataset
  • 29. 29Type A-3 Some Issues for Super Resolution • Metric Paper (The Perception-Distortion Tradeoff, 2018 CVPR) • Analysis distortion measure(PSNR, SSIM, etc.) vs human opinion score(Perceptual quality) • Good supplementary video: https://www.youtube.com/watch?v=6Yid4dituqo (PR-12) Reference: “The Perception-Distortion Tradeoff”, 2018 CVPR Better quality
  • 30. 30Type A-3 References • “Image Super-Resolution Using Deep Convolutional Networks”, 2014 ECCV • “Super Resolution Applications in Modern Digital Image Processing”, 2016 IJCA • “Loss Functions for Image Restoration with Neural Networks”, 2016 IEEE TCI • “Deconvolution and Checkerboard Artifacts”, distill blog(https://distill.pub/2016/deconv-checkerboard/) • “Accurate Image Super-Resolution Using Very Deep Convolutional Networks”, 2016 CVPR • “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network”, 2016 CVPR • “Accelerating the Super-Resolution Convolutional Neural Network”, 2016 ECCV • “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”, 2017 CVPR • http://www.vision.ee.ethz.ch/~timofter/publications/NTIRE2017SRchallenge_factsheets.pdf • “The Perception-Distortion Tradeoff”, 2018 CVPR • “Deep Learning for Single Image Super-Resolution: A Brief Review”, 2018 IEEE Transactions on Multimedia (TMM) • “A Deep Journey into Super-resolution: A survey”, 2019 arXiv