Psychotherapist Remarks' ML classifier: Insights from LLM and Topic Modeling Application

Alexander VaninVadim Bolshev^*Anastasia Panfilova

• Institute of Psychology (RAS), Moscow, Russia

This paper presents the development of a machine learning (ML) classification model for analyzing topics in therapist remarks within a psychotherapeutic context. The study applies BERTopic, a MLbased topic modeling technique, to unstructured dialogues from two distinct groups of therapists: classical (founders of therapeutic schools, such as Carl Rogers, Fritz Perls, and Albert Ellis) and modern (representing various psychotherapeutic approaches). This methodology enables the identification and characterization of recurring topics across these groups. We detail the implementation of the BERTopic algorithm, including the construction of a vector space from therapist remarks, dimensionality reduction, clustering, and optimization of topic representations. To enhance interpretability, the study integrates expert assessment and manual refinement of the topic structure alongside automated modeling. The results reveal the most common and stable topics in therapists' discourse, providing insights into how language patterns in therapy emerge and persist across different therapeutic styles. The extracted topics are utilized as features for training a ML classifier, which is then applied to a case study examining thematic structures in therapy sessions conducted by Carl Rogers and modern Cognitive Behavioral Therapy (CBT) practitioners. The analysis uncovered distinct thematic compositions, identifying key topics that characterize each group's therapeutic practice. The resulting model is publicly available, offering broad applications in psychotherapy research and training. This study contributes to the growing intersection of ML and psychotherapy by demonstrating how automated methods can enhance both therapeutic practice, clinical supervision, and education. It underscores the potential of topic modeling as a powerful tool for deepening our understanding of therapist communication.