Arabic Fonts and Scribal Scripts

Convolutional Neural Networks for Font Classification


Classifying pages or text lines into font categories aids transcription because single font Optical Character Recognition (OCR) is generally more accurate than omni-font OCR. We present a simple framework based on Convolutional Neural Networks (CNNs), where a CNN is trained to classify small patches of text into predefined font classes. To classify page or line images, we average the CNN predictions over densely extracted patches. We show that this method achieves state- of-the-art performance on a challenging dataset of 40 Arabic computer fonts with 98.8% line level accuracy. This same method also achieves the highest reported accuracy of 86.6% in predicting paleographic scribal script classes at the page level on medieval Latin manuscripts. Finally, we analyze what features are learned by the CNN on Latin manuscripts and find evidence that the CNN is learning both the defining morphological differences between scribal script classes as well as overfitting to class-correlated nuisance factors. We propose a novel form of data augmentation that improves robustness to text darkness, further increasing classification performance.

International Conference on Document Analysis and Recognition (ICDAR), IEEE.

Models and code (with corresponding docker image instructions) for this paper can be found in this github repo. The pixel annotations I created for one set of experiments can be found here.

I submitted this method to the 2016 and 2017 Competitions on Classification of Latin Medieval Manuscripts (CLaMM). Here is the code for my 2016 and 2017 submissions.

In the 2016 competition, I placed 2nd in Task 1 and 3rd in Task 2 (Results Table). 2017 results have yet to be announced.