An Uncertainty-Aware Hybrid Symbolic-Lexicon Framework for Multi-Domain Sentiment Classification
DOI:
https://doi.org/10.63158/journalisi.v8i5.1872Keywords:
multi-domain sentiment classification, selective prediction, deferment, domain shift, lexicon adaptation, rough-set evidence, algorithmic traceabilityAbstract
Multi-domain sentiment classifiers may return a polarity label even when the evidence behind that decision becomes fragile under domain shift. This study evaluates an uncertainty-aware hybrid symbolic-lexicon framework designed to pair each polarity prediction with a traceable accept-or-defer status. The framework combines a lexical branch adapted only on training data with training-only fitted symbolic feature states, variable-precision rough evidence, and nested score fusion. Evaluation uses the UCI Sentiment Labelled Sentences dataset, comprising 3,000 balanced English sentences from Amazon Cells, IMDb, and Yelp, under repeated within-source, pooled multi-domain, and leave-one-source-out protocols. In pooled evaluation, the framework achieved 83.73% accuracy and 83.76% F1-score, improving F1 by 0.50 points over the adaptive lexical branch. The boundary mechanism deferred 53.55% of cases and retained 92.54% accuracy among accepted predictions; however, matched-coverage logistic regression and SVM confidence rankings produced lower selective risk. The contribution is therefore a reproducible, algorithmically traceable polarity-and-deferment mechanism rather than a claim of universal predictive or uncertainty-ranking superiority.
Downloads
References
[1] M. Wankhade, A. C. S. Rao, and C. Kulkarni, "A survey on sentiment analysis methods, applications, and challenges," Artif. Intell. Rev., vol. 55, no. 7, pp. 5731–5780, Oct. 2022, doi: 10.1007/s10462-022-10144-1.
[2] K. Alahmadi, S. Alharbi, J. Chen, et al., "Generalizing sentiment analysis: A review of progress, challenges, and emerging directions," Soc. Netw. Anal. Min., vol. 15, no. 1, 2025, doi: 10.1007/s13278-025-01461-8.
[3] S. Wang, J. Zhou, Q. Chen, Q. Zhang, T. Gui, and X. Huang, "Domain generalization via causal adjustment for cross-domain sentiment analysis," pp. 5286–5298, 2024, arXiv: 2402.14536.
[4] D. Xin, K. Zhao, J. Sun, and Y. Li, “CDA²: Counterfactual diffusion augmentation for cross-domain adaptation in low-resource sentiment analysis,” in Proc. 31st Int. Conf. Comput. Linguistics, Jan. 2025, pp. 61–72.
[5] W. Zhang, Y. Deng, B. Liu, et al., "Sentiment analysis in the era of large language models: A reality check," in Findings of the Association for Computational Linguistics: NAACL 2024, pp. 3881–3906, 2024, doi: 10.18653/v1/2024.findings-naacl.246.
[6] G. Xie, Y. Zhang, J. Bao, et al., "Comprehensive and efficient distillation for lightweight sentiment analysis models," in Proc. 2025 Conf. Empirical Methods in Natural Language Processing, pp. 22070–22091, 2025, doi: 10.18653/v1/2025.emnlp-main.1122.
[7] D. Herrera-Poyatos, C. Peláez-González, C. Zuheros, A. Herrera-Poyatos, V. Tejedor, F. Herrera, and R. Montes, "An overview of model uncertainty and variability in LLM-based sentiment analysis: Challenges, mitigation strategies, and the role of explainability," Front. Artif. Intell., 2025, doi: 10.3389/frai.2025.1609097.
[8] C. J. Hutto and E. Gilbert, "VADER: A parsimonious rule-based model for sentiment analysis of social media text," in Proc. 8th Int. AAAI Conf. Web and Social Media, vol. 8, no. 1, pp. 216–225, 2014, doi: 10.1609/icwsm.v8i1.14550.
[9] K. Barik and S. Misra, "Analysis of customer reviews with an improved VADER lexicon classifier," J. Big Data, vol. 11, no. 1, art. no. 10, 2024, doi: 10.1186/s40537-023-00861-x.
[10] Z. Pawlak, "Rough sets," Int. J. Comput. Inf. Sci., vol. 11, no. 5, pp. 341–356, 1982.
[11] W. Ziarko, "Variable precision rough set model," J. Comput. Syst. Sci., vol. 46, no. 1, pp. 39–59, 1993.
[12] Y. Geifman and R. El-Yaniv, "SelectiveNet: A deep neural network with an integrated reject option," in Proc. 36th Int. Conf. Machine Learning, PMLR, vol. 97, 2019, arXiv: 1901.09192.
[13] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, "On calibration of modern neural networks," in Proc. 34th Int. Conf. Machine Learning, PMLR, vol. 70, pp. 1321–1330, 2017.
[14] Y. Ovadia, E. Fertig, J. Ren, et al., "Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift," in Proc. 33rd Conf. Neural Information Processing Systems, 2019, arXiv: 1906.02530.
[15] A. N. Angelopoulos and S. Bates, "Conformal prediction: A gentle introduction," Found. Trends Mach. Learn., 2023, doi: 10.1561/2200000101.
[16] K. Gunaratne, S. Kobourov, and J. A. Miller, "Class angular distortion index for dimensionality reduction," Comput. Graph. Forum, 2026, doi: 10.1111/cgf.70465.
[17] D. Kotzias, M. Denil, N. de Freitas, et al., "From group to individual labels using deep features," in Proc. 21st ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, pp. 597–606, 2015, doi: 10.1145/2783258.2783380.
[18] S. Kapoor and A. Narayanan, "Leakage and the reproducibility crisis in machine-learning-based science," Patterns, vol. 4, no. 9, art. no. 100804, 2023, doi: 10.1016/j.patter.2023.100804.
[19] J. Pineau, P. Vincent-Lamarre, K. Sinha, et al., "Improving reproducibility in machine learning research (A report from the NeurIPS 2019 reproducibility program)," J. Mach. Learn. Res., vol. 22, pp. 1–20, 2021.
[20] M. Ock, N. Yi, J. Ahn, and M. Jo, "How many alternatives can be ranked? A comparison of the paired comparison and ranking methods," Value Health, vol. 19, no. 5, pp. 655–660, 2016, doi: 10.1016/j.jval.2016.03.1836.
[21] S. Holm, "A simple sequentially rejective multiple test procedure," Scand. J. Statist., vol. 6, no. 2, pp. 65–70, 1979.
[22] A. Ramponi and B. Plank, "Neural unsupervised domain adaptation in NLP — A survey," in Proc. 28th Int. Conf. Computational Linguistics, pp. 6838–6855, 2020, doi: 10.18653/v1/2020.coling-main.603.
[23] K. Zhou, Z. Liu, Y. Qiao, T. Xiang, and C. C. Loy, "Domain generalization: A survey," IEEE Trans. Pattern Anal. Mach. Intell., 2022, doi: 10.1109/TPAMI.2022.3195549.
[24] A. Jacovi and Y. Goldberg, "Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?," in Proc. 58th Annu. Meeting Association for Computational Linguistics, pp. 4198–4205, 2020, doi: 10.18653/v1/2020.acl-main.386.
[25] Q. Lyu, M. Apidianaki, and C. Callison-Burch, "Towards faithful model explanation in NLP: A survey," Comput. Linguist., 2024, doi: 10.1162/coli_a_00511.
[26] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, "DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter," arXiv: 1910.01108, 2019.
[27] W. Wang, F. Wei, L. Dong, et al., "MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers," in Proc. Advances in Neural Information Processing Systems, 2020.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Information Systems and Informatics
This work is licensed under a Creative Commons Attribution 4.0 International License.
Author's Declaration
- The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
- The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
- The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
- The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
- The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
- The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
- If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
- The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights
