您的位置: 首页  >  文章  >  Image-to-Image Translation with Conditional Adversarial Networks

Image-to-Image Translation with Conditional Adversarial Networks

分类: 文章 • 2024-02-01 12:47:10

CVPR2017年的一篇文章，大名鼎鼎的pix2pix，paper链接https://arxiv.org/abs/1611.07004，一作是MIT的PHD，现在是UC Berkeley的postdoctoral，作者的主页https://people.eecs.berkeley.edu/~isola/，code已经released在github上面了https://github.com/phillipi/pix2pix，截至目前google citation已经585。

文章要做的事情
输入：一张图片 　　 输出：一张图片　
文章将这个任务描述为图像到图像的翻译（image-to-image translation），img2img translation示例如下所示。
Image-to-Image Translation with Conditional Adversarial Networks
文章中的一些实验结果的效果图如下所示。
Google Maps at 512x512 resolution
Image-to-Image Translation with Conditional Adversarial Networks
citescapes label2photo
Image-to-Image Translation with Conditional Adversarial Networks
facades labels2photo
Image-to-Image Translation with Conditional Adversarial Networks
day2night
Image-to-Image Translation with Conditional Adversarial Networks
edge2handbags
Image-to-Image Translation with Conditional Adversarial Networks
edge2shoes
Image-to-Image Translation with Conditional Adversarial Networks
edges2photo
Image-to-Image Translation with Conditional Adversarial Networks
failure cases
Image-to-Image Translation with Conditional Adversarial Networks
pix2pix方法
输入一张图片，然后通过U-Net产生一张新的图片，最后在通过cGAN对原始的ground-truth和产生的图片判断真假，如下左图是pix2pix的框架图，右图为U-Net的结构图。

Image-to-Image Translation with Conditional Adversarial NetworksImage-to-Image Translation with Conditional Adversarial Networks

pix2pix训练的loss function如下所示。
Image-to-Image Translation with Conditional Adversarial Networks
loss function的物理意义是使得generator产生的图片不仅能骗过discriminator，而且还能与ground-truth在L1 metric下尽可能接近，对于discriminator，文章中采用的是patchGAN，从ablation study来看这个trick还是挺work的，文中对比了4个patch，70X70最work。
文章中比较work的几个点:
1. U-Net
2. cGANs+L1
3. patchGAN

相关推荐