Abstract. This study investigates the development and performance evaluation of transformer-based models for the automatic classification of public transportation passenger reviews, aiming to enhance feedback processing while optimizing issue resolution. Efficient handling of passenger feedback is crucial for improving public transportation services, as unresolved complaints or operational inefficiencies can decrease passenger satisfaction and create logistical challenges. Traditional text classification approaches, such as keyword-based methods or classical Machine Learning (ML) algorithms, struggle with multilingual and heterogeneous textual data, particularly in low-resource languages. This study addresses this gap by systematically comparing transformer-based architectures for review classification in Russian and Kazakh, demonstrating their effectiveness in real-world applications. A key contribution of this research lies in evaluating both language-specific and multilingual transformers on passenger-generated feedback, offering insights into their generalization capabilities. Unlike previous studies, which predominantly focus on English-language datasets, this work introduces a newly created, manually labeled dataset covering diverse real-world scenarios in Russian and Kazakh, enabling an objective comparative analysis. Three transformer models DeepPavlov/rubert-base-cased, XLM-RoBERTa-base, and XLM-RoBERTa-large were trained and tested to assess their ability to process complex multilingual input. Experimental results indicate that XLM-RoBERTa-large achieves the highest classification accuracy (90%), particularly for code-mixed and multilingual reviews, whereas rubert-base-cased performs consistently well for Russian-language feedback (87.667%), reinforcing its suitability for monolingual classification tasks. XLM-RoBERTa-base exhibits a balanced trade-off between accuracy and robustness, making it a viable option for heterogeneous review processing (89.5%). Despite their effectiveness, transformer-based models still encounter challenges related to data balancing and the handling of underrepresented classes, particularly in scenarios with uneven language distributions or domain-specific terminology. These findings confirm that transformer models significantly enhance the automation of passenger feedback classification, providing a scalable solution for public transportation providers.

Keywords: Natural language processing, text classification, Transformers, BERT, DeepPavlov, XLM-RoBERTa, passenger feedback review,