On the interface between linguistics, computer science and psychiatry: analyzing textual key-factors affecting BERT-based classification of schizophrenia in social media texts.

Résumé

INTRODUCTION: This paper investigates language impairments in schizophrenia (SZ) by analyzing the decision-making process of a transformer-based model in discriminating between texts produced by persons with SZ and persons without SZ. By doing so, we integrate insights from language-centered investigations with computational approaches. Using BERT-base-cased, we explore how linguistic markers of SZ can be identified through Natural Language Processing (NLP) techniques, with emphasis on improving performance reliability via dataset refinement and approaching interpretability of deep learning outputs via statistical analyses of thematic content.METHODS: We report the fine-tuning of a BERT model for text classification of 31,278 Reddit posts (15,639 SZ, 15,639 controls). The experiment evaluated the capacity of the model to distinguish language produced by individuals with SZ.RESULTS: The model achieved moderate performance (Accuracy = 0.6969; AUC = 0.78) and remained stable across hyperparameter configurations, indicating that foundation models such as BERT fit to data and, therefore, further performance gains are more likely to be derived from dataset refinement than from additional hyperparameter optimization. There were three key factors affecting the model's performance: text length, topic of discussion and vocabulary choices. Posts that were correctly classified tended to be significantly longer (