Evaluation of TF-IDF and Word2Vec Representations Using Support Vector Machine for Multiclass Product Category Classification of Tokopedia Reviews
DOI:
https://doi.org/10.63158/journalisi.v8i5.1820Keywords:
Text Classification, TF-IDF, Word2Vec, Support Vector Machine, Tokopedia Reviews, Product Category ClassificationAbstract
As electronic commerce continues to expand, customer reviews have become an important source for automated text classification. This research evaluates and compares TF-IDF and Word2Vec representations for multiclass product category classification with Support Vector Machine (SVM). The experiment used the Tokopedia Product Reviews 2025 dataset containing 65,543 reviews across six product categories. After text preprocessing and keyword-based text refinement, 11,239 relevant reviews were retained. The dataset was divided into 80% training sets and 20% hold-out test sets. The reviews were transformed using TF-IDF and Word2Vec, while SMOTE was applied to the training data to handle the imbalance among product categories. Classification performance was assessed through five-fold cross-validation and a hold-out test set based on accuracy, macro precision, macro recall, and macro F1-score. Five-fold cross-validation showed that TF-IDF achieved mean accuracy, macro precision, macro recall, and macro F1-score of 95.95%, 93.13%, 92.78%, and 92.93%, respectively, compared with 90.80%, 84.58%, 86.02%, and 85.19% for Word2Vec. On the hold-out test set, TF-IDF achieved 95.42% accuracy and 91.88% macro F1-score, while Word2Vec achieved 91.01% and 84.91%, respectively. The Wilcoxon signed-rank test yielded a p-value of 0.0625, indicating no statistically significant difference at the 5% significance level. Nevertheless, TF-IDF consistently achieved higher descriptive performance than Word2Vec.
Downloads
References
[1] F. Tonda and M. Asif Khan, “Pengaruh Promosi dan Online Customer Reviews Terhadap Keputusan Pembelian Melalui Minat Beli (Literature Review Manajemen Pemasaran),” J. Manaj. dan Pemasar. Digit., vol. 2, no. 3, pp. 305–318, 2024, doi: 10.38035/jmpd.v2i3.213.
[2] T. Chen, P. Samaranayake, X. Y. Cen, M. Qi, and Y. C. Lan, “The Impact of Online Reviews on Consumers’ Purchasing Decisions: Evidence From an Eye-Tracking Study,” Front. Psychol., vol. 13, no. June, 2022, doi: 10.3389/fpsyg.2022.865702.
[3] Q. Li et al., “A Survey on Text Classification: From Traditional to Deep Learning,” ACM Trans. Intell. Syst. Technol., vol. 13, no. 2, 2022, doi: 10.1145/3495162.
[4] F. S. Aditama, D. Krismawati, and S. Pramana, “Multiclass Classification Of Marketplace Products With Machine Learning,” MEDIA Stat., vol. 17, no. 1, pp. 25–35, Oct. 2024, doi: 10.14710/medstat.17.1.25-35.
[5] M. F. Naufal and S. F. Kusuma, “Natural Language Processing Untuk Otomatisasi Pengenalan Pronomina Dalam Kalimat Bahasa Indonesia,” J. Teknol. Inf. dan Ilmu Komput., vol. 9, no. 5, pp. 1011–1018, 2022.
[6] A. R. Hanum et al., “ANALISIS KINERJA ALGORITMA KLASIFIKASI TEKS BERT DALAM MENDETEKSI BERITA HOAKS Assyfa,” J. Teknol. Inf. dan Ilmu Komput., vol. 11, no. 3, pp. 537–546, 2024, doi: 10.25126/jtiik2024118093.
[7] S. S. Shaker, D. Alhajim, A. Ali, T. Al-Khazaali, H. Aqeel Hussein, and A. F. Athab, “Feature Extraction based Text Classification: A review,” J. Algebr. Stat., vol. 13, no. 1, pp. 646–653, 2022.
[8] G. Salton and C. Buckley, “Term-weighting approaches in automatic text retrieval,” Inf. Process. Manag., vol. 24, no. 5, pp. 513–523, Jan. 1988, doi: 10.1016/0306-4573(88)90021-0.
[9] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed Representations of Words and Phrases and their Compositionality,” in Advances in Neural Information Processing Systems, C. J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Weinberger, Eds., Curran Associates, Inc., 2013. [Online].
[10] C. Cortes and V. Vapnik, “Support-vector networks,” Mach. Learn., vol. 20, no. 3, pp. 273–297, Sep. 1995, doi: 10.1007/BF00994018.
[11] D. R. V. Saputra, M. Maimunah, and E. R. Arumi, “Optimizing Aspect-Based Sentiment Analysis for Kyai Langgeng Park Using PSO and SVM,” J. Inf. Syst. Informatics, vol. 6, no. 4, pp. 2856–2867, 2024, doi: 10.51519/journalisi.v6i4.930.
[12] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” J. Artif. Intell. Res., vol. 16, pp. 321–357, Jun. 2002, doi: 10.1613/jair.953.
[13] F. Rifaldy, Y. Sibaroni, and S. S. Prasetiyowati, “Effectiveness of Word2Vec and TF-IDF in Sentiment Classification on Online Investment Platforms using Support Vector Machine,” JIPI (Jurnal Ilm. Penelit. dan Pembelajaran Inform., vol. 10, no. 2, pp. 863–874, 2025.
[14] J. Zhou, Z. Ye, S. Zhang, Z. Geng, N. Han, and T. Yang, “Investigating response behavior through TF-IDF and Word2vec text analysis : A case study of PISA 2012 problem-solving process data,” Heliyon, vol. 10, no. 16, p. e35945, 2024, doi: 10.1016/j.heliyon.2024.e35945.
[15] A. Al Tawil, L. Almazaydeh, D. Qawasmeh, B. Qawasmeh, M. Alshinwan, and K. Elleithy, “Comparative Analysis of Machine Learning Algorithms for Email Phishing Detection Using TF-IDF, Word2Vec, and BERT,” Comput. Mater. Contin., vol. 81, no. 2, pp. 3395–3412, 2024, doi: 10.32604/cmc.2024.057279.
[16] D. I. Sumantiawan, J. E. Suseno, and W. A. Syafei, “Sentiment Analysis of Customer Reviews Using Support Vector Machine and Smote-Tomek Links For Identify Customer Satisfaction,” J. Sist. Info. Bisnis, vol. 13, no. 1, pp. 1–9, 2023, doi: 10.21456/vol13iss1pp1-9.
[17] S. Abdurrahman, “Tokopedia Product Reviews 2025,” Kaggle. Accessed: Dec. 15, 2025. [Online]. Available: https://www.kaggle.com/ds/8992580
[18] D. Rifaldi and A. Fadlil, “Teknik Preprocessing Pada Text Mining Menggunakan Data Tweet ‘Mental Health,’” Decod. J. Pendidik. Teknol. Inf., vol. 3, no. 2, pp. 161–171, 2023.
[19] Ryzal Nur Alvandy and Arita Witianti, “Analisis Sentimen Ulasan E-Commerce Menggunakan Metode SVM,” Elkom J. Elektron. dan Komput., vol. 18, no. 2, pp. 164–175, 2025, doi: 10.51903/elkom.v18i2.3253.
[20] A. B. Mutiara, E. P. Wibowo, and P. I. Santosa, “Improving the accuracy of text classification using stemming method , a case of non ‑ formal Indonesian conversation,” J. Big Data, pp. 1–16, 2021, doi: 10.1186/s40537-021-00413-1.
[21] A. Alaiya, N. Nurdin, and C. Agusniar, “Sentiment Analysis of E-Commerce Product Reviews on Tokopedia Using Support Vector Machine,” J. Appl. Informatics Comput., vol. 9, no. 5, pp. 2869–2878, 2025, doi: 10.30871/jaic.v9i5.10977.
[22] N. P. Yunita, “Aplikasi Pencarian Hadis Menggunakan Vector Space Model Dengan Pembobotan TF-IDF Dan Confix-Stripping Stemmer,” J. Teknol. Inf. dan Ilmu Komput., vol. 10, no. 3, pp. 665–676, 2023, doi: 10.25126/jtiik.2023106736.
[23] Y. Wang, “Research on the TF–IDF algorithm combined with semantics for automatic extraction of keywords from network news texts,” J. Intell. Syst., vol. 33, no. 1, 2024, doi: 10.1515/jisys-2023-0300.
[24] Z. Jiang, B. Gao, Y. He, Y. Han, P. Doyle, and Q. Zhu, “Text Classification Using Novel Term Weighting Scheme-Based Improved TF-IDF for Internet Media Reports,” Math. Probl. Eng., vol. 2021, pp. 1–30, Mar. 2021, doi: 10.1155/2021/6619088.
[25] Yuliska, D. H. Qudsi, J. H. Lubis, K. U. Syaliman, and N. F. Najwa, “Analisis Sentimen Pada Data Saran Mahasiswa Terhadap Kinerja Departemen Di Perguruan Tinggi Menggunakan Sentiment Analysis in the Student ’ S Reviews of College Department Performance Using,” J. Teknol. Inf. dan Ilmu Komput., vol. 8, no. 5, pp. 1067–1076, 2021, doi: 10.25126/jtiik.202184842.
[26] Z. H. Kilimci and S. Akyokus, “Deep learning- and word embedding-based heterogeneous classifier ensembles for text classification,” Complexity, vol. 2018, 2018, doi: 10.1155/2018/7130146.
[27] G. B. R. Wasiat, P. S. Sasongko, E. Suharto, S. Kurniawan, I. Waspada, and Rismiyati, “Classification of Generative AI Answers Versus Human Answers on Indonesian Texts Using the TF-IDF and Support Vector Machine Methods,” in 2025 8th International Conference on Informatics and Computational Sciences (ICICoS), 2025, pp. 247–252. doi: 10.1109/ICICoS68590.2025.11329997.
[28] A. Nabilah, N. I. Sari, M. Afrina, and A. Ibrahim, “The Sentiment Analysis Of Indonesian Startup Application Reviews Using TF-IDF + SVM and FastText : A Comparative Study,” vol. 10, no. 3, pp. 125–137, 2026.
[29] M. Carpita and S. Golia, “Categorical classifiers in multiclass classification with imbalanced datasets,” Stat. Anal. Data Min., vol. 16, no. 4, pp. 391–405, 2023, doi: 10.1002/sam.11624.
[30] Wijiyanto, A. I. Pradana, and V. Atina, “Teknik K-Fold Cross Validation untuk Mengevaluasi Kinerja Mahasiswa,” J. Algoritm., vol. 21, no. 1, pp. 239–248, 2024, doi: 10.33364/algoritma/v.21-1.1618.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Information Systems and Informatics
This work is licensed under a Creative Commons Attribution 4.0 International License.
Author's Declaration
- The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
- The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
- The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
- The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
- The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
- The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
- If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
- The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights
