单个摄像头也可以很好的变焦——谷歌Super Res Zoom的解读

1. 普通单摄数字变焦的原理

普通消费者的手机 进行数字图像拍摄时，相机传感器记录光线强度而前方的拜耳滤色镜进行转化成色彩，如扫雷一般对周边缺失的像素来推测，为什么会丢失像素呢？数字变焦会对原始图像进行裁剪并放大，放大过程有如线性插值，创造出原来没有的像素，不合适时图片质感失去。（细节丢失不可避免，而数字成像理解为对色彩的重建，叫做去马赛克）。

2. 谷歌用天文摄影的DRIZZLE方法

DRIZZLE 多帧超分辨率采样（超采样），基本理念是将多张低分辨率的连拍照片直接合并对齐到更高分辨率的像素网格中。

3. 利用图形混叠

在摄像中有个问题，图像混叠，经典的呈现方式——摩尔纹。

但对超采样来说，是好事，从中获取亮度和颜色。要“手抖”，利用光学防抖模块来抖动。

4. 存在的问题，谷歌如何解决。

问题：
1）这里“手抖”是控制下的抖动，而实际会存在许多不可预料的抖动，造成超采样实现起来有难度。
2）需要高速连拍，曝光时间短，照片存在噪点。
解决办法：
原文：

• To make the algorithm handle scenes with complex local motion (people, cars, water or tree leaves moving) reliably, we developed a robustness model that detects and mitigates alignment errors. We select one frame as a “reference image”, and merge information from other frames into it only if we’re sure that we have found the correct corresponding feature. In this way, we can avoid artifacts like “ghosting” or motion blur, and wrongly merged parts of the image.
  
• To effectively merge frames in a burst, and to produce a red, green, and blue value for every pixel without the need for demosaicing, we developed a method of integrating information across the frames that takes into account the edges of the image, and adapts accordingly. Specifically, we analyze the input frames and adjust how we combine them together, trading off increase in detail and resolution vs. noise suppression and smoothing. We accomplish this by merging pixels along the direction of apparent edges, rather than across them. The net effect is that our multi-frame method provides the best practical balance between noise reduction and enhancement of details.

我的总结：
1） 侦查纠错模型——用合理的参照，使帧与帧之间的图像吻合，才能进行超采样。
2）结合边缘检测算法——对输入采用多帧合成方式，在增加细节和抑制噪点之间进行合理权衡，保证最终画面的边缘锐度和纯净度。

参考及引用：
【1】 https://ai.googleblog.com/2018/10/see-better-and-further-with-super-res.html
【2】 https://www.toutiao.com/i6626948408345117198/?tt_from=weixin&utm_campaign=client_share&wxshare_count=2&from=singlemessage&timestamp=1542986728&app=news_article&utm_source=weixin&iid=51711616303&utm_medium=toutiao_ios&group_id=6626948408345117198&pbid=6627198234363364872