Comparative Analysis of Support Vector Machine and IndoBERT Algorithms in Stance Detection on Political Issues in Social Media X: A Case Study of BPI Danantara

Abstract

Stance detection is an NLP task aimed at identifying and classifying a writer’s attitude toward a topic as supportive, opposing, or neutral based on text analysis, providing deeper insights into public opinion and supporting data-driven decision-making. This study focuses on Indonesian society’s stance toward the National Investment Management Agency (BPI Danantara), which has received positive responses for its economic potential as well as negative reactions due to concerns over governance and corruption risks. In this research, a machine learning approach using the Support Vector Machine algorithm and a deep learning approach using the IndoBERT model were applied to detect pro, contra, and neutral stances in posts from the X social media platform. A total of 6,805 tweets were collected through scraping and manually labeled by three annotators. The dataset was then processed through cleaning, undersampling, and modeling, and evaluated using accuracy, precision, recall, F1-score, and ROC-AUC metrics. Experiments were conducted across various scenarios, including binary and three-class classification as well as balanced and imbalanced datasets, to assess the effectiveness of each model. The results indicate that IndoBERT consistently outperforms SVM across all scenarios, particularly in capturing nuanced stances in Indonesian text. However, statistical evaluation using the paired t-test and the Wilcoxon signed-rank test reveals that the performance differences between the two models are generally not statistically significant, except in the three-class classification scenario with undersampling, where IndoBERT shows a significant advantage in handling balanced multi-class stance detection. These findings demonstrate the advantage of Transformer-based approaches for complex stance detection tasks and highlight their potential for developing automated public opinion monitoring systems. Nevertheless, this study has limitations, including the relatively small dataset, the focus on a single social media platform, and the methods applied. Future research could explore larger and more diverse datasets, incorporate multiple social media platforms, and employ other Transformer-based models to enhance generalization and improve stance detection accuracy.