Transformer-Based Sentiment Analysis for classification of non-depressive and suicidal thought from Bangla Text

Abstract

The growing prevalence of mental health issues, particularly depression and suicidal thoughts, points to the need to develop automated tools capable of detecting such sentiments from online communication. This study addresses some critical challenges by introducing a novel sentiment analysis framework for Bangla text, aimed at classifying content into non-depressive, depressive, and suicidal categories. We propose a hybrid deep learning model leveraging the strengths of transformer-based architectures, designed to manage long textual sequences effectively, a critical aspect in the context of natural language processing. Our model integrates RoBERTa (Robustly Optimised BERT Pre-Training Approach) with a Self-Attention Network (SAN), creating a synergistic framework for nuanced sentiment detection in Bangla social media posts, comments, and articles. This method addresses the scarcity of Bangla specific datasets by utilising a dataset curated for the study. The results demonstrate the superiority of our model, achieving an accuracy of 82.58%, alongside precision, recall, and F1-scores of 82%. Subsequently, it emphasises the potential for the proposed model to support early identification of mental health concerns, thereby enabling timely interventions and contributing to mental health awareness and prevention efforts. In the future, deploying the model as a real-time chatbot or browser extension could scan Bangla social media for depressive, non-depressive, and suicidal content and alert professionals to the risk factors.

Keywords: {'name': 'Deep Learning, RoBERTa, SAN, Bangla Language, Mental Health'}

Authors & affiliations

Md. Samiul Islam
Affiliation not supplied

Rafiul Hoque
Affiliation not supplied

Sagor Sarkar
Affiliation not supplied

Md. Rajibul Palas
Affiliation not supplied

Md. Moshiur Rahman
Affiliation not supplied

Muhammad Hoque
Affiliation not supplied

Deposited references

Supplied references, not a list of publications citing this article.

DOI: 10.4103/2224-3151.206167

DOI: 10.1177/02537176211033643

DOI: 10.3126/aet.v1i1.39660

DOI: 10.30818/jitu.2.2.2695

DOI: 10.3390/s23010506

DOI: 10.1186/s12877-017-0489-1

BNLP Toolkit: Online, 2021, Available: https://pypi.org/project/bnlp-toolkit/.

DOI: 10.1017/S0033291702006943

DOI: 10.1016/j.jad.2015.11.008

DOI: 10.18653/v1/2020.challengehml-1.1

DOI: 10.1109/ICCCI56745.2023.10128430

DOI: 10.1109/CSNT.2018.8820260

GeeksforGeeks. (2024). Confusion matrix in machine learning. https://www.geeksforgeeks.org/confusion-matrix-machine-learning/ [Accessed August 4, 2024].

DOI: 10.14311/bit.2024.02.13

DOI: 10.1109/ACCESS.2019.2957192

DOI: 10.1109/ICTC.2016.7763455

DOI: 10.18653/v1/2023.findings-acl.80

Kaggle. Kaggle code. https://www.kaggle.com/work/code [Accessed May 30, 2024].

Kaggle. Kaggle notebooks documentation. https://www.kaggle.com/docs/notebooks [Accessed May 30, 2024].

DOI: 10.1371/journal.pmed.1002905

Liu, S., Zhang, L., Yang, X., Su, H., & Zhu, J. (2021). Query2label: A simple transformer way to multi-label classification. arXiv preprint arXiv:2107.10834.

DOI: 10.1109/ACCESS.2019.2901547

DOI: 10.1109/SLT48900.2021.9383581

DOI: 10.1109/ICACCI.2014.6968484

DOI: 10.1155/2015/193729

DOI: 10.1177/0020764012444259

DOI: 10.1109/ACCESS.2020.3009626

DOI: 10.17577/IJERTV9IS050290

Organizzazione mondiale della sanità. (2014). Preventing suicide: a global imperative. World health organization.

DOI: 10.1155/2022/3498123

DOI: 10.3389/fnhum.2023.1292010

DOI: 10.1109/ACCESS.2022.3165563

Shaheen, Z., Wohlgenannt, G., & Filtz, E. (2020). Large scale legal text classification using transformer models. arXiv preprint arXiv:2010.12871.

DOI: 10.3390/app11020869

DOI: 10.1109/UBMK52708.2021.9558906

DOI: 10.1109/ACCESS.2022.3152828

TensorFlow Core. (2021). tf.keras.callbacks.EarlyStopping. https://shorturl.at/R0Nh2 [Accessed August 24, 2024].

DOI: 10.36548/jaicn.2021.3.001

DOI: 10.1109/ICBSLP47725.2019.201500

DOI: 10.18653/v1/2020.emnlp-demos.6

World Health Organization. (2021). Suicide worldwide in 2019: Global health estimates. https://www.who.int/publications/i/item/9789240026643 [Accessed May 14, 2025].

DOI: 10.1016/S2215-0366(16)30030-X

DOI: 10.1109/ICECCO.2015.7416902