Intelligent Systems.
Theory and Applications

(Intellektual'nye Sistemy. Teoriya i Prilozheniya)

Abstract

The article is devoted to the validation of a short version of the questionnaire "Types of orientations in difficult situations" (TODS-36) as a tool for classifying types of perception of difficult situations and evaluating the role of text description analysis involving large language models in determining these types. The following tasks are being solved: development and validation of an algorithm for classifying types of perception of difficult life situations based on the TODS-36 profile using large language models to analyze text descriptions; comparison of the results of algorithmic classification with expert assessment; comparison of the effectiveness of various algorithm modifications. The study sample included 200 text descriptions of difficult situations received from 200 respondents (131 women, 69 men) aged 14 to 54 years. The reference classification was performed by expert psychologists based on a joint analysis of the quantitative indicators of the TODS-36 questionnaire scales and qualitative data (text descriptions of difficult situations). A series of six experiments was conducted. In experiments 1–4, the role of quantitative data and formal classification rules gradually increased. In experiment 5, classification accuracy was assessed when qualitative data were coded by a large language model; in experiment 6, it was assessed when the coding was performed by expert psychologists. The most effective among the algorithms using a large language model was the hybrid one, in which threshold rules are applied to quantitative indicators and the large language model analyzes qualitative data. Its accuracy was 85% on the full sample. At the same time, the highest accuracy in the series of experiments – 85.5% – was obtained using the same algorithm with expert coding of qualitative data. The results obtained confirm the possibility of using the TODS-36 questionnaire as a tool for classifying types of perception of difficult situations and allow us to consider large language models as a means of supporting an expert in classifying types of perception of difficult situations.

Keywords: coping, difficult life situation, situation perception, TODS-36, an algorithm for determining the types of perception of difficult situations, large language models, mixed-methods research. \par \textbf {Funding. }The study was funded by a grant from the Russian Science Foundation, project number № 25-18-00737, https://rscf.ru/project/25-18-00737/.