Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach阅读笔记

x. Questions

1. AE步骤, 怎么判断收敛不好。万一有的图片需要erase 3次，有的erase2两次会怎么样?
   
   • 实验直接选了3次
2. PSL，为什么conv7的结果做avg pooling后直接可以用作weight？conv7的feature map数量必须是class的数量?可解释性?
   
   • 应该图中fc指的是下一层,是经过avg pool 和全连接计算之后的
   • 图中中间部分每个pixel属于各类的概率，也就是HxWxK
   • 用AE的结果和weighted map作为监督信息约束网络
3. salience map多类是怎么做到的?

x. Conclusion

Pro

• AE
  
  • progressively mine discriminative object regions
• PSL
  
  • limitation of AE on capturing boundary details
  • missed pixels
• state of art

Con

• 不同的图片erase速度不一定相同
• Segmentation mask too intuitive
• hard threshold of heatmap(use graph-cut)
  
  • eg: Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

0. Abstract

• classifciation networks:
  
  • small and sparse discriminative regions
• segmentation:
  
  • dense, interior and intergral regions
• how to connect gap?
  
  • adversarial erasing approach
  • Starting with a single small object region
  • sequentially discover new and complement object regions by erasing the current mined regions in an adversarial manner.
  • eventually constitute a dense and complete object region for learning semantic segmentation
  • online prohibitive segmentation learning approach collaborate with adversarial erasing by providing auxiliary segmentation supervision

1. Introduction

1.1 methods

• proposal-based
• classification based

1.2 AE

• adversarial erasing(AE)

• shortcuts:

  • miss some object-related regions
  • introduce some noise
  • less attention on boundaries

• prohibitive segmentation learning(PSL)

  • uses the predicted image-level classification confidences to modulate the corresponding category-specific response maps
  • form them into an auxiliary segmentation mask
  • category-specific segmentation maps with low classification confidences are prohibited for contributing to the formed supervision mask

2. Related work

state of art(previous)
X. Qi, Z. Liu, J. Shi, H. Zhao, and J. Jia. Augmented feed-
back in semantic segmentation under image level supervi-
sion. In ECCV, pages 90–105, 2016.

3. Method

3.1 object region mining with AE

• iteratively performs two operations:

  • learning a classification network for localizing the object discriminative regions
  • adversarially erasing the discovered regions
  • stop until the network cannot well converge

• classification network:

  • DeepLab-CRF-LargeFOV
  • GAP applied on conv7

• AE process

  • applying a hard threshold to the heatmap
  • replacing mined region’s internal pixels by the mean pixel values of all the training images

• finding background

  • use saliency detection technology

3.2 PSL

• solves
  
  • limitation of AE on capturing boundary details
  • missed pixels

• 2 branch

  • cross-entropy loss
  • combine two loss by add

  

todos

1. 阅读Salient Object Detection: A Discriminative
   Regional Feature Integration Approach