Usability Testing of the E-learning System at SMK TI Bali Jimbaran Using the Concurrent Think Aloud (CTA) and System Usability Scale (SUS) Methods
DOI:
https://doi.org/10.21154/asanka.v7i1.13635Keywords:
Concurrent Think Aloud, System Usability Scale, Satisfaction, User Satisfaction, E-LearningAbstract
E-learning serves as a support tool for teachers in fulfilling their responsibilities as educators and homeroom teachers. This system has been implemented as a learning medium; however, some users, particularly new users, still experience difficulties adapting to its interface, which differs from those of commonly used systems. This study aims to evaluate the e-learning system of SMK TI Bali Global Jimbaran in terms of effectiveness, efficiency, and user satisfaction. The usability testing method was selected to evaluate the system based on these three aspects. The techniques used in this study include Concurrent Think Aloud (CTA) and the System Usability Scale (SUS). This evaluation used a sample of 20 teacher respondents. The sample was separated by level of technology proficiency: 10 representatives from the advanced group and 10 representatives from the beginner group. In qualitative research, including CTA, the number of participants is determined based on the principle of data saturation, which occurs when the collected data begin to show repetitive patterns and no significant new findings emerge. With 15–20 users, coverage of usability problems can reach nearly 100% of relevant issues. Therefore, 20 participants are considered sufficient to ensure that the major usability problems in the e-learning system have been identified. To measure user satisfaction, the System Usability Scale (SUS) method was employed, while interface improvement recommendations were derived using the Concurrent Think Aloud (CTA) method. The respondents involved in the SUS method were the same as those participating in the CTA method. The results of the CTA and SUS analyses will serve as a basis for recommendations to improve the e-learning system in the future.
References
Amelia, Putu, Chrysilla Gunawan, Nengah Widya Utami, Eka Grana, and Aristyana Dewi. (2024). “Penggunaan Retrospective Think Aloud dan System Usability Scale pada Edlink Mobile pada Universitas Primakara.” 9: 958–68.
Antika, Ayu, and Evi Yulianingsih. 2023. “Analisa Sistem E-Learning pada Universitas PGRI Palembang dengan Metode System Usability Scale (SUS).” Smatika Jurnal 13(01): 53–61. doi:10.32664/smatika.v13i01.721.
Baihaqy, M. Alif Muwafiq, Abidarin Rosidi, and A. Syamsul Syahdan. 2016. “Analisis Usability Aplikasi E-Learning.” Jurnal PPKM III (hakaman 60): 159–73.
Candiasa, I Made, I Gede Aris Gunadi, and I Nyoman Wahyu Semeru Putra. 2023. “UX Evaluation Using Firstclick, Performance Measurement, RTA, and Questionnaire on E-Commerce Website.” Sinkron 8(1): 451–60. doi:10.33395/sinkron.v8i1.12037.
Dwipayana, I Gusti Made Surya, I Made Candiasa, and Luh Joni Erawati Dewi. 2025. “Usability Evaluation of Lecturer Information System in ITB STIKOM Bali.” Sinkron 9(1): 149–59. doi:10.33395/sinkron.v9i1.14315.
Fatmawati, Azizah. 2021. “Evaluasi Usability pada Learning Management System Open Learning Menggunakan System Usability Scale.” INOVTEK Polbeng - Seri Informatika 6(1): 120. doi:10.35314/isi.v6i1.1881.
Indra Pratama, Putu Ary, Nengah Widya Utami, and Putu Trisna Hady Permana S. 2023. “Evaluasi Usability pada Aplikasi M-Pise LPD Digital di Kabupaten Jembrana dengan Metode Usability Testing.” Jurnal Informatika dan Rekayasa Perangkat Lunak 5(2): 162. doi:10.36499/jinrpl.v5i2.8913.
Iryanti, Emi, La Ode Mohamad Zulfiqar, Sri Suning Kusumawardani, and Indriana Hidayah. 2022. “Pengukuran Kepuasan Pengguna E-Learning Menggunakan Metode Evaluasi Heuristik dan System Usability Scale.” Jurnal Teknologi Informasi dan Ilmu Komputer 9(3): 469–78. doi:10.25126/jtiik.2022924631.
Kamilia, Tarisa, Seftin Fitri Ana Wati, and Dhian Satria Yudha Kartika. 2024. “Think Aloud dan Heuristic Evaluation Metode Evaluasi Aplikasi RSBA untuk Perancangan Ulang UI/UX Menggunakan UCD.” Jurnal Informatika dan Teknik Elektro Terapan 12(3). doi:10.23960/jitet.v12i3.5027.
Mu’ayyadah, Mu’ayyadah, and Noor Fatmawati. 2021. “Pemanfaatan Teknologi Sebagai Pembelajaran E-Learning Google Classroom pada Mata Pelajaran IPS.” ASANKA : Journal of Social Science and Education 2(1): 31–42. doi:10.21154/asanka.v2i1.2364.
Nugraha, Putu Tulus, I Made Sukarsa, and Ni kadek Dwi Rusjayanthi. 2022. “Usability Testing Sistem Love Bali Menggunakan Teknik Performance Measurement dan Concurrent Think Aloud (CTA).” JITTER : Jurnal Ilmiah Teknologi dan Komputer 3(2): 1137. doi:10.24843/jtrti.2022.v03.i02.p13.
Putera, W. A., and I. M. Candiasa. 2021. “Analysis of E-Learning User Satisfaction ITB Stikom Bali Using End User Computing Satisfaction (EUCS) Method.” Journal of Physics: Conference Series 1810(1). doi:10.1088/1742-6596/1810/1/012017.
Putera, W A, I G E Sandiyasa, and ... 2022. “Evaluation of The E-Learning System Usability Using The System Usability Scale (SUS).” International …: 322–26.
Sulistiya, Medika, Zainul Mu, Sri Rahayu Natasia, and Muhammad Yusuf. 2016. “Penerapan Metode Think Aloud Untuk Evaluasi Usability pada Website Dinas Pendidikan dan Kebudayaan Kota MNO.” Telematika 16(1): 25–32.
Thaib, Faisal, Miswar Papuangan, and Imam Hizbullah. 2022. “Evaluasi Usability Menggunakan Metode Think Aloud pada Aplikasi Mobile Mister Aladin.” Jurnal Teknik SILITEK 1(02): 80–89. doi:10.51135/jts.v1i02.21.
Wibisono, Prasetyo Margaret, Retno Indah Rokhmawati, and Buce Trias Hanggara. 2023. “Evaluasi Usability Aplikasi Perangkat Bergerak Sipindo Menggunakan Metode Think Aloud dan Heuristic Evaluation.” Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer 7(3): 1247–56.
Yuono, Erfan, Satrio Hadi Wijoyo, and Retno Indah Rokhmawati. 2021. “Evaluasi Usability Pada E-Learning Menggunakan Metode Human Centered Design dan Usability Testing One Click Service SMK PGRI 3 Malang.” Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer 5(7): 2809–18.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 I Putu Agus Eka Yatna Cipta, Rifky Lana Rahardian, Aldho Wardana, Muhammad Agha Firmansyah, I Made Arya Satyana
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal the right of first publication with the work simultaneously licensed under Creative Commons Attribution-NonCommercial 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
