SAN: Structure-Aware Network for Complex and Long-tailed Chinese Text Recognition

TL;DR

A structure-aware network utilizing the hierarchical composition information to improve the recognition performance of complex characters and tail characters can significantly improve the performances of complex characters and tail characters, yielding a better overall performance.

Abstract

In text recognition, complex glyphs and tail classes have always been factors affecting model performance. Specifically for Chinese text recognition, the lack of shape-awareness can lead to confusion among close complex characters. Since such characters are often tail classes that appear less frequently in the training-set, making it harder for the model to capture its shape information. Hence in this work, we propose a structure-aware network utilizing the hierarchical composition information to improve the recognition performance of complex characters. Implementation-wise, we first propose an auxiliary radical branch and integrate it into the base recognition network as a regularization term, which distills hierarchical composition information into the feature extractor. A Tree-Similarity-based weighting mechanism is then proposed to further utilize the depth information in the hierarchical representation. Experiments demonstrate that the proposed approach can significantly improve the performances of complex characters and tail characters, yielding a better overall performance. Code is available at https://github.com/Levi-ZJY/SAN.

Figures (6)

• Figure 1: The Structure-Aware Network (SAN). The orange square frame is the ARB-TreeSim and the blue square frame is the base network. The gradient flow of the ARB-TreeSim and the VM decoder influence the feature extractor together.
• Figure 2: In figure (a), the red nodes are called structure and the blue nodes are called radical. In figure (b), the yellow circle indicates that the total weight of the subtree is 1/3.
• Figure 3: TreeSim Calculation. Nodes with a red circle are the match modes and nodes with a black circle are the mismatch nodes. The calculation results of the two radical trees are the same.
• Figure 4: Successful recognition examples using ARB. (a) (b) are complex character examples. (c) (d) are long-tailed character examples.The text strings are ABINet prediction, SAN prediction and Ground Truth.
• Figure 5: The accuracy (left), the average TreeSim (middle) and the increase ratio of TreeSim (right) divided by RSSL on character prediction samples by SAN and ABINet. RSSL represents the length of the radical structure sequence of a character.