Benchmarking Classical Machine Learning and Deep Learning Models on the Cleveland Heart Disease Binary Classification Dataset
DOI:
https://doi.org/10.63158/journalisi.v8i5.1794Keywords:
Ridge Classifier, Tree based Models, BiLSTM CNN Hybrid, Class Imbalance (SMOTE), Cleveland Heart Disease DatasetAbstract
Many high-stakes applications relied on binary classification, yet practitioners lacked equitable, methodical benchmarks that evaluated diverse model families under identical conditions. This study used the Cleveland Heart Disease tabular dataset to conduct a controlled, empirical benchmark of a regularised linear model (Ridge Classifier), tree-based ensembles (XGBoost, Random Forest), and a hybrid deep learning architecture combining Bidirectional Long Short-Term Memory and Convolutional Neural Network layers (BiLSTM-CNN) on a mildly imbalanced binary classification task. A strict protocol was applied, comprising uniform pre-processing, Synthetic Minority Oversampling Technique (SMOTE) applied to training folds only, a hybrid hold-out design with internal five-fold stratified crossvalidation on the training set (70%), and evaluation using accuracy, positive-class precision, recall, F1 score, and area under the curve (AUC) on a fixed test set (15%). Cross-validation results showed that no single model dominated every metric: the Ridge Classifier unexpectedly achieved the best AUC (0.8945), indicating strong ranking ability; Random Forest and XGBoost achieved the highest accuracy (81.67%); and the BiLSTM-CNN hybrid achieved the highest recall and F1 score (0.8478 and 0.8365, respectively). These findings indicated that, for this dataset, classical and ensemble approaches remained competitive with the more complex hybrid deep learning model, suggesting their continued relevance for similar small-scale, tabular binary classification tasks.
Downloads
References
[1] A. N. Edmund, I. D. Emmanuel, and E. Ebenezer, “Comparative Evaluation of BiLSTM-CNN , XGBoost , and Ridge Regression for Heart Disease Classification on the Cleveland Dataset,” J. Inf. Syst. Informatics, vol. 8, no. 3, pp. 3819–3851, 2026, doi: 10.63158/journalisi.v8i3.1668.
[2] P. H. Hutagalung, S. Informasi, and U. Nasional, “Heart Disease Classification Using Optimised XGBoost and Random Forest with SHAP Explanations,” Sink. J. dan Penelit. Tek. Inform., vol. 10, no. 1, pp. 330–342, 2026, doi: 10.33395/sinkron.v10i1.15544 e-ISSN.
[3] M. A. Kadhim and A. M. Radhi, “Heart disease classification using optimized Machine learning algorithms Heart disease classification using optimized Machine learning algorithms,” Iraqi J. Comput. Sci. Math., vol. 4, no. 2, pp. 31–42, 2023, doi: 10.52866/ijcsm.2023.02.02.004.
[4] S. W. P. M. Janosi Andras and R. Detrano, “Heart Disease,” Heart Disease. [Online]. Available: https://archive.ics.uci.edu/ml/datasets/Heart+Disease
[5] T. Karadeniz, G. Tokdemir, and H. H. Maraş, “A Structure-Aware Classifier to Predict Heart Disease BT - Intelligent Systems and Pattern Recognition,” T. Ensari, A. Bennour, B. Bouaziz, I. Rida, and W. Mahdi, Eds., Cham: Springer Nature Switzerland, 2026, pp. 315–326.
[6] M. T. Yaqoob, A. M. Khan, A. A. Shaikh, and N. Khan, “Review on Cleveland Heart Disease Dataset using Machine Learning,” QUEST Res. J., vol. 21, no. 01, pp. 87–98, 2023.
[7] K. V. V. Reddy, I. Elamvazuthi, A. A. Aziz, S. Paramasivam, H. N. Chua, and S. Pranavanand, “Heart disease risk prediction using machine learning classifiers with attribute evaluators,” Appl. Sci., vol. 11, no. 18, Art. no. 8352, 2021.
[8] Aaryashree and A. Devi, “Integrating machine learning with electrochemical sensors for intelligent food safety monitoring,” Artif. Intell. Chem., vol. 4, no. 1, p. 100105, 2026, doi: 10.1016/j.aichem.2025.100105.
[9] Y. Yang and Y. YANG, “Hybrid Prediction Method for Wind Speed Combining Ensemble Empirical Mode Decomposition and Bayesian Ridge Regression,” IEEE Access, vol. 8, no. 2, pp. 71206–71218, 2020, doi: 10.1109/ACCESS.2020.2984020.
[10] N. V Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE : Synthetic Minority Over-sampling Technique,” J. Artif. Intell. Res., vol. 16, no. 2, pp. 321–357, 2002.
[11] Z. Shangli, Z. Lili, Q. I. U. Kuanmin, L. U. Ying, and C. A. I. Baigen, “Variable Selection in Logistic Regression Model ∗,” Chinese J. Electron., vol. 24, no. 4, pp. 813–818, 2015, doi: 10.1049/cje.2015.10.025.
[12] N. E. Ajimah and O. N. Iloanusi, “Biometric voice recognition system in the context of multiple languages : using traditional means of identification of individuals in Nigeria languages and English language,” Res. Stat. (Taylor Fr. group, LLC), vol. 2, no. i, pp. 1–16, 2024, doi: 10.1080/27684520.2024.2362298.
[13] A. Al-hamadani, “Benchmarking the Mapping of Address and Port With Encapsulation Border Relay Routers,” IEEE Access, vol. 13, no. June, pp. 124631–124650, 2025.
[14] K. Kumar, R. Garg, and N. Goel, “Heart Disease Prediction Using Machine Learning: A Comparative Approach on CHSLB and Cleveland Datasets with Advanced Modeling Techniques BT,” in Data Science and Applications, S. J. Nanda, R. P. Yadav, M. Prasad, and M. Saraswat, Eds., Cham: Springer Nature Switzerland, 2026, pp. 38–50. doi: DOI: 10.1007/978-3-032-12827-0_3.
[15] N. E. Ajimah, A. A. Christopher, O. T. Oluwaseun, L. I. Agbotiname, and D. S. Tanko, “Artififcial Intelligence- Assisted Internet of Medical Things Enabling Medical Image Processing,” in Handbook of Security and Privacy of AI-Enabled Healthcare Systems and Internet of Medical Things, vol. 1, Routledge Publishers: Taylor & Francis Group, 2023, pp. 309–334. doi: 9781003370321.
[16] A. Kostovska, G. S. Member, D. Vermetten, C. Doerr, and S. Džeroski, “OPTION : OPTImization Algorithm Benchmarking ONtology,” IEEE Trans. Evol. Comput., vol. 27, no. 6, pp. 1618–1632, 2023, doi: 10.1109/TEVC.2022.3232844.
[17] K. Panetta, L. Kezebou, and G. S. Member, “Benchmark Dataset and Underwater Image Enhancement With GAN,” IEEE J. Ocean. Eng., vol. 47, no. 1, pp. 59–75, 2022, doi: 10.1109/JOE.2021.3086907.
[18] R. Ramm, Y. Li, A. Oberdörster, S. Heist, G. Notni, and P. Kühmstedt, “Sensor applications Benchmarking AI-Based Monocular Depth Estimators in Terms of Their Metrological Potential Following 3-D Sensor Guideline VDI / VDE 2634,” Sens. Appl., vol. 10, no. 2, pp. 4–7, 2026.
[19] N. Fazakis, S. Kotsiantis, and Y. Dimakopoulos, “MAGA : A Controlled Minority-Class Generative-Augmentation Framework With Fairness-Aware Selection for Imbalanced Diabetes Tabular Classification,” IEEE Access, vol. 14, no. March 2026, 2026.
[20] S. K. Patel, S. Member, J. Surve, and G. S. Member, “Encoding and Tuning of THz Metasurface-Based Refractive Index Sensor With Behavior Prediction Using XGBoost Regressor,” IEEE Access, vol. 10, no. 1, pp. 24797–24814, 2022, doi: 10.1109/ACCESS.2022.3154386.
[21] M. Alzubi, M. Almseidin, and M. Alkasassbeh, “Comparative Analysis of Fuzzy Rule Interpolation Techniques Across Various Scenarios Using a Set of Benchmarks,” IEEE Access, vol. 12, no. March, pp. 33140–33153, 2024, doi: 10.1109/ACCESS.2024.3371858.
[22] H. Khan, W. Ahmed, M. Z. Alam, M. Saifuddin, A. Ahmad, and S. Ahmed, “Preprocessing Techniques for Heart Disease Prediction Using Machine Learning BT - Advanced Computing Techniques in Engineering and Technology,” R. C. Bansal, V. Devedzic, R. Nayak, B. Agarwal, A. Tandon, and P. Jain, Eds., Cham: Springer Nature Switzerland, 2026, pp. 307–313.
[23] N. Ajimah et al., “Overview of Voice Biometric System: VoiceE Person Identification and Challenges,” in 2020 LGT-ECE-UNN International Conference- Technological Innovation for Holistic Sustainable Development (TECHISD2020), 2020.
[24] Y. Chen, S.-S. LIN, Y. SHI, T.-Y. HO, and X. XU, “MCC : Multi-Cluster Contrastive Semi-Supervised Segmentation Framework for Echocardiogram Videos,” IEEE Access, vol. 13, no. February, pp. 30543–30554, 2025, doi: 10.1109/ACCESS.2025.3541173.
[25] L. I. U. Chunhui, Q. I. Yue, and D. Wenrui, “The Data-Reusing MCC-Based Algorithm and Its Performance Analysis ∗,” Chinese J. Electron., vol. 25, no. 4, pp. 719–725, 2016, doi: 10.1049/cje.2016.06.019.
[26] K. Govindaraju and G. Kalimuthu, “Heart Disease Analysis and Prediction with Machine Learning Techniques Using Cleveland Dataset BT - Intelligent Computing Systems,” A. Safi, A. Martin-Gonzalez, C. Brito-Loeza, and V. Castañeda-Zeman, Eds., Cham: Springer Nature Switzerland, 2025, pp. 20–43.
[27] T. Chen and C. Guestrin, “XGBoost : A Scalable Tree Boosting System,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 785–794. doi: 10.1145/2939672.2939785.
[28] L. E. O. Breiman, “Random Forests,” Mach. Learn., vol. 45, no. 1, pp. 5–32, 2001, doi: 10.1023/A:1010933404324.
[29] A. E. Hoerl and R. W. Kennard, “Ridge Regression: Biased Estimation for Nonorthogonal Problems,” Technometrics, vol. 12, no. 1, pp. 55–67, Feb. 1970, doi: 10.1080/00401706.1970.10488634.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Information Systems and Informatics
This work is licensed under a Creative Commons Attribution 4.0 International License.
Author's Declaration
- The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
- The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
- The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
- The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
- The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
- The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
- If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
- The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights
