robust scene text recognition with automatic rectification
Shi B, Wang X, Lv P, et al. Robust Scene Text Recognition with Automatic Rectification[J]. arXiv preprint arXiv:1603.03915, 2016.
本文的贡献:
1. propose a novel scene text recognition method that is robust to irregular text(不规则的文本)
1. propose a novel scene text recognition method that is robust to irregular text(不规则的文本)
2. model extends the STNframework with an attention-based model. The originalSTN is only tested on plain convolutional neural networks.
3. our model adopts a convolutional-recurrent structurein the encoder of the SRN, thus is a novel variant of theattention-based model.
总体框架:
包括 Spatial Transformer Network (STN ) 以及 Sequence Recognition Network (SRN ) 两个网络结构。其中, STN 通过 Thin-Plate-Spline 变换,能够将透射变换或者弯曲的文本图片对齐到一个正规的、更易读的图片;SRN 能够直接将输入的文本图片识别为一个文本序列。
这个系统是一个端到端的文本识别系统,在训练过程中也不需要额外标记字符串的关键点、字符位置等。同时,由于 STN 和 SRN这两个网络的共同作用,该系统在自然场景的文本识别方面取得了 state-of-the-art 的结果,特别是对于那些有着各种形变的字符图片。
四种校正的例子
SRN consists of an encoder and a decoder.
The encoder extracts a sequential representation from the input image I0.
The decoder recurrently generates a sequence conditionedon the sequential representation, by decoding the relevant contents it attends to at each step.