A Statistical Analysis of System Usability Scale (SUS) Evaluations in Online Learning Platform

Authors

  • Ozzi Suria Mercu Buana University of Yogyakarta, Indonesia
Pages Icon

DOI:

https://doi.org/10.51519/journalisi.v6i2.750

Keywords:

online learning, System Usability Scale (SUS), Moodle, statistical analysis, usability improvement

Abstract

This study evaluates the usability of a Moodle-based online learning system at a university using the System Usability Scale (SUS). Introduced in 2019, the platform has been instrumental in facilitating access to educational resources, enhancing the distribution of course materials, and supporting academic activities through interactive features. The SUS was employed to gather subjective feedback from 120 student respondents, providing insights into the system's efficiency, effectiveness, and user satisfaction. Statistical analysis was performed using methods including the Shapiro-Wilk test to address the skewed distribution of the original dataset with a mean SUS score of 71.52 and a high standard deviation of 16.18, indicating varied user experiences. Analysis of sample means was also conducted to achieve a more stable estimate of usability, resulting in a mean of 71.51 with a significantly reduced standard deviation of 2.44, suggesting a more consistent assessment across users. The final SUS score placed the system in the 'C+' category according to the Sauro and Lewis grading scale, indicating that while usability is acceptable, substantial improvements are necessary.

Downloads

Download data is not yet available.

References

U. Mukarromah and W. Wijayanti, “Implementation of the online learning at vocational high school during Covid-19: Between obligations and barriers,” Jurnal Pendidikan Vokasi, vol. 11, no. 1, Mar. 2021, doi: 10.21831/jpv.v11i1.37110.

V. Singh and A. Thurman, “How Many Ways Can We Define Online Learning? A Systematic Literature Review of Definitions of Online Learning (1988-2018),” American Journal of Distance Education, vol. 33, no. 4, pp. 289–306, Oct. 2019, doi: 10.1080/08923647.2019.1663082.

E. Enoh, B. Qamaruzzaman, and Q. Y. Zaqiyah, “Pengembangan dan Implementasi E-Learning di Universitas Islam Bandung,” Ta’dib Jurnal Pendidikan Islam, vol. 12, no. 2, pp. 573–584, Aug. 2023, doi: 10.29313/TJPI.V12I2.12312.

H. C. Kuong, “Enhancing Online Learning Experience: From Learners’ Perspective,” Procedia Soc Behav Sci, vol. 191, pp. 1002–1005, Jun. 2015, doi: 10.1016/j.sbspro.2015.04.403.

N. K. A. Suwastini, K. I. R. Dewi, and G. R. Dantes, “Benefits of online learning according to recent studies,” Jurnal Inovasi dan Teknologi Pembelajaran, vol. 10, no. 1, pp. 32–42, Mar. 2023, doi: 10.17977/UM031V10I12023P032.

L. Pei and H. Wu, “Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis,” Med Educ Online, vol. 24, no. 1, p. 1666538, Jan. 2019, doi: 10.1080/10872981.2019.1666538.

P. Paudel, “Online Education: Benefits, Challenges and Strategies During and After COVID-19 in Higher Education,” International Journal on Studies in Education, vol. 3, no. 2, pp. 70–85, Sep. 2021, doi: 10.46328/IJONSE.32.

M. D. B. Castro and G. M. Tumibay, “A literature review: efficacy of online learning courses for higher education institution using meta-analysis,” Educ Inf Technol (Dordr), vol. 26, no. 2, pp. 1367–1385, Mar. 2021, doi: 10.1007/S10639-019-10027-Z/METRICS.

H. N. Y. Nhi and T. T. Nhan, “Online Education During The Covid-19 Pandemic - Opportunities And Challenges: A Case Study In The South Of Vietnam,” European Journal of Open Education and E-learning Studies, vol. 7, no. 1, Apr. 2022, doi: 10.46827/ejoe.v7i1.4222.

A. Kilburn, B. Kilburn, and T. Cates, “Drivers of Student Retention: System Availability, Privacy, Value And Loyalty In Online Higher Education,” Academy of Educational Leadership Journal, vol. 18, no. 4, 2014.

W. Bhuasiri, O. Xaymoungkhoun, H. Zo, J. J. Rho, and A. P. Ciganek, “Critical success factors for e-learning in developing countries: A comparative analysis between ICT experts and faculty,” Comput Educ, vol. 58, no. 2, pp. 843–855, Feb. 2012, doi: 10.1016/J.COMPEDU.2011.10.010.

J. Brooke, “SUS: a retrospective,” J Usability Stud, vol. 8, no. 2, pp. 29–40, 2013.

B. Klug, “An Overview of the System Usability Scale in Library Website and System Usability Testing,” Weave: Journal of Library User Experience, vol. 1, no. 6, Apr. 2017, doi: 10.3998/weave.12535642.0001.602.

J. Brooke, “SUS-a quick and dirty usability scale,” in Usability Evaluation In Industry, P. W. Jordan, B. Thomas, and I. Lyall, Eds., London, Bristol: Taylor & Francis, 1996, pp. 189–194.

S. Manikandan, “Measures of central tendency: The mean,” J Pharmacol Pharmacother, vol. 2, no. 2, pp. 140–142, Jun. 2011, doi: 10.4103/0976-500X.81920.

M. A. Barri, “A Simulation Showing the Role of Central Limit Theorem in Handling Non-Normal Distributions,” Am J Educ Res, vol. 7, no. 8, pp. 591–598, Aug. 2019, doi: 10.12691/education-7-8-8.

N. Clark, M. Dabkowski, P. J. Driscoll, D. Kennedy, I. Kloo, and H. Shi, “Empirical Decision Rules for Improving the Uncertainty Reporting of Small Sample System Usability Scale Scores,” Int J Hum Comput Interact, vol. 37, no. 13, pp. 1191–1206, Aug. 2021, doi: 10.1080/10447318.2020.1870831.

P. Vlachogianni and N. Tselios, “Perceived usability evaluation of educational technology using the System Usability Scale (SUS): A systematic review,” Journal of Research on Technology in Education, vol. 54, no. 3, pp. 392–409, May 2022, doi: 10.1080/15391523.2020.1867938.

F. G. Sembodo, G. F. Fitriana, and N. A. Prasetyo, “Evaluasi Usability Website Shopee Menggunakan System Usability Scale (SUS),” 2021.

K. Sagar and A. Saha, “Quantitative usability assessment relying on experiential and specific task-based SUS ratings,” Journal of Statistics and Management Systems, vol. 23, no. 2, pp. 333–348, Feb. 2020, doi: 10.1080/09720510.2020.1736316.

J. Sauro and J. R. Lewis, Quantifying the User Experience: Practical Statistics for User Research, 2nd Edition. Cambridge: Elsevier, 2016. doi: 10.1016/C2010-0-65192-3.

J. P. Royston, “An Extension of Shapiro and Wilk’s W Test for Normality to Large Samples,” J R Stat Soc Ser C Appl Stat, vol. 31, no. 2, pp. 115–124, Jun. 1982, doi: 10.2307/2347973.

S. G. Kwak and J. H. Kim, “Introduction Basic Concepts of Central Limit Theorem Central limit theorem: the cornerstone of modern statistics KJA,” Korean J Anesthesiol, vol. 70, no. 2, pp. 144–156, 2017, doi: 10.4097/kjae.2017.70.2.144.

R. Bono, M. J. Blanca, J. Arnau, and J. Gómez-Benito, “Non-normal distributions commonly used in health, education, and social sciences: A systematic review,” Front Psychol, vol. 8, no. SEP, p. 267072, Sep. 2017, doi: 10.3389/FPSYG.2017.01602/BIBTEX.

Downloads

Published

2024-06-13

Issue

Vol. 6 No. 2 (2024): June

Section

Articles

License

Authors Declaration

  1. The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
  2. The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
  3. The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
  4. The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
  5. The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
  6. The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
  7. If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
  8. The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights

How to Cite

[1]
O. Suria, “A Statistical Analysis of System Usability Scale (SUS) Evaluations in Online Learning Platform”, journalisi, vol. 6, no. 2, pp. 992–1007, Jun. 2024, doi: 10.51519/journalisi.v6i2.750.