Premium
Determining the number of participants needed for the usability evaluation of e‐learning resources: A M onte C arlo simulation
Author(s) -
Davids Mogamat Razeen,
Harvey Justin,
Halperin Mitchell L.,
Chikte Usuf M. E.
Publication year - 2015
Publication title -
british journal of educational technology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.79
H-Index - 95
eISSN - 1467-8535
pISSN - 0007-1013
DOI - 10.1111/bjet.12336
Subject(s) - usability , computer science , web usability , resource (disambiguation) , cognitive walkthrough , usability lab , usability inspection , usability goals , human–computer interaction , usability engineering , computer network
The usability of computer interfaces has a major influence on learning. Optimising the usability of e‐learning resources is therefore essential. However, this may be neglected because of time and monetary constraints. User testing is a common approach to usability evaluation and involves studying typical end‐users interacting with the application being tested. Determining the minimum number of users that are required for such an evaluation is important as it has a direct bearing on the costs and time requirements. This issue has long been a subject of debate and the widely cited recommendation of five users being enough has been questioned. We conducted a usability evaluation of an e‐learning resource for electrolyte and acid‐base disorders by studying the interaction of medical doctors with the application. A total of 15 serious usability problems were detected, most of these related to an interactive treatment simulation. With this report, we are making available the data on the detection of serious usability problems by each of our participants. We have used these data to run a M onte C arlo simulation and examine how many users would be sufficient to test our application. The simulation revealed that we would have required 10 participants to detect 80% of the serious usability problems. The MATLAB ® code for running the simulation is supplied, as are our calculations of problem discovery rates. The e‐learning resource which was evaluated is freely available, together with a revised version with all identified usability problems addressed.