Date of Publication :30th October 2023

Abstract: This paper studies the topic of detecting handwritten mathematical expressions using two deep learning models - namely the convolutional neural network (CNN) and the vision transformer (ViT). We implement and train these models on a merged dataset comprising EMNIST dataset, HASYv2 dataset and a Mathematical Symbol dataset. We also compare them for the specific use case of graphing mathematical expressions in a single variable. It was found that the CNN performs better for multiple classification evaluation metrics such as accuracy, precision, recall, F1 score, training time, final model size, Cohen’s kappa score and Matthew’s correlation coefficient. Hence, it was chosen for building the application. The user facing application uses not only the aforementioned model for recognition and detection of the equations in the input images, it also displays a user interface for capturing user input using streamlit, performs morphological operations, noise reduction, and segmentation of the input images using Python code, and displays the graphical form of the equation using SymPy.

Reference :