Monte-Carlo dropout Convolutional Neural Network for Query by Example Handwritten Word spotting

Fatemeh Daraee¹ Saeed Mozaffari² Seyyed Mohammad Razavi³

1) Ph.D in Electrical Engineering ,University of Birjand
2) Associate Professor of Electrical Engineering, Semnan University
3) Associate Professor of Electrical Engineering, University of Birjand

Publication : The First Conference on Artificial Intelligence and Smart Computing(aisc2022.semnan.ac.ir)
Abstract :
Word spotting has become a popular search method in handwritten document retrieval over the last years. Recently, deep Convolutional Neural Networks (CNNs) have been used extensively for word spotting and achieved significant results. In this paper, we propose a Monte-Carlo dropout CNN model for semantic feature extraction. We used a novel total loss in our model, containing both inter-class and intra-class variations to extract discriminative features. The proposed Monte-Carlo CNN has utilized for query by example (QBE) word spotting. For this aim, the uncertainties of query image and retrieval set are predicted, and the cosine distance between them is obtained to rank the retrieved images. We evaluated our proposed approach on five standard handwritten databases. Achieving 96.65% accuracy on the IAM database attests that the proposed method is more accurate than the state-of-the-art word spotting techniques. This improvement can be mainly contributed to considering the uncertainty in the retrieval process.
Keywords : Word spotting Monte-Carlo dropout Center loss Uncertainty prediction