Text Spotting

Adaptive Bezier-Curve Network, or ABCNet, is an end-to-end framework for arbitrarily-shaped scene text spotting. It adaptively fits arbitrary-shaped text by a parameterized bezier curve. It also utilizes a feature alignment layer, BezierAlign, to calculate convolutional features of text instances in curved shapes. These features are then passed to a light-weight recognition head.

BezierAlign is a feature sampling method for arbitrarily-shaped scene text recognition that exploits parameterization nature of a compact Bezier curve bounding box. Unlike RoIAlign, the shape of sampling grid of BezierAlign is not rectangular. Instead, each column of the arbitrarily-shaped grid is orthogonal to the Bezier curve boundary of the text. The sampling points have equidistant interval in width and height, respectively, which are bilinear interpolated with respect to the coordinates. Formally given an input feature map and Bezier curve control points, we concurrently process all the output pixels of the rectangular output feature map with size $h\_{\text {out }} \times w\_{\text {out }}$. Taking pixel $g\_{i}$ with position $\left(g\_{i w}, g\_{i h}\right)$ (from output feature map) as an example, we calculate $t$ by: $$ t=\frac{g\_{i w}}{w\_{o u t}} $$ We then calculate the point of upper Bezier curve boundary $tp$ and lower Bezier curve boundary $bp$. Using $tp$ and $bp$, we can linearly index the sampling point $op$ by: $$ op=bp \cdot \frac{g\_{i h}}{h\_{\text {out }}}+tp \cdot\left(1-\frac{g\_{i h}}{h\_{\text {out }}}\right) $$ With the position of $op$, we can easily apply bilinear interpolation to calculate the result. Comparisons among previous sampling methods and BezierAlign are shown in the Figure.

PGNet is a point-gathering network for reading arbitrarily-shaped text in real-time. It is a single-shot text spotter, where the pixel-level character classification map is learned with proposed PG-CTC loss avoiding the usage of character-level annotations. With a PG-CTC decoder, we gather high-level character classification vectors from two-dimensional space and decode them into text symbols without NMS and RoI operations involved, which guarantees high efficiency. Additionally, reasoning the relations between each character and its neighbors, a graph refinement module (GRM) is proposed to optimize the coarse recognition and improve the end-to-end performance.