An Adopted CNN-based Approach for Classification and Restoration of Challenging Handwritten Text

Challenging handwritten text recognition is important for several real-world applications such as digitising documents, grading student exam answers, writer identification etc. to enhance the recognizer performance. Due to unconstrained and free writing styles, handwritten text often includes both legible, neatly written text and illegible, sloppy text, unreadable, and struckout text. In this work, "challenging text" refers to handwriting that is difficult to read due to factors like sloppiness, overwriting, crossed-out lines, and shakiness. This work aims to propose a method for the classification of challenging text and restoring character shapes, such that an appropriate recognition method can be used to achieve better recognition performance. We propose a tri-channel-based CNN for classification by considering words as input. Further, to strengthen the feature extraction, the proposed work extracts seven features based on characteristics of text, such as connectivity, spacing between characters, stroke width, edge strength, and quality of text. The extracted seven features are supplied to the CNN along with the features extracted by the Tri-channel network for classification. The classified challenging text is fed to the combination of U-Net and BiLSTM networks for restoring the character shapes. The effectiveness of the proposed method is demonstrated by conducting a variety of experiments. Our method outperforms the state-of-the-art methods in terms of average classification rate.