Premium
The usability of personal digital assistants (PDAs) for assessment of practical performance
Author(s) -
Treadwell Ina
Publication year - 2006
Publication title -
medical education
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.776
H-Index - 138
eISSN - 1365-2923
pISSN - 0308-0110
DOI - 10.1111/j.1365-2929.2006.02543.x
Subject(s) - usability , computer science , context (archaeology) , handwriting , significant difference , multimedia , medical education , human–computer interaction , medicine , artificial intelligence , paleontology , biology
Context The administration of an objective structured clinical examination (OSCE) using paper checklists presents problems such as illegible handwriting, missing student names and/or numbers and lost checklists. Calculating and entering results is not only time‐consuming, but is subject to human errors, and feedback to students is rarely available. To rectify these problems, personal digital assistants (PDAs) and HaPerT software were acquired to replace paper checklists and provide automated results and feedback. This study sought to determine the usability of the PDA assessment system. Methods The usability of the PDA system was evaluated according to effectiveness, efficiency and user satisfaction. Effectiveness was judged by comparing the results of an OSCE conducted in 2003 (paper‐based method) and repeated in 2004 and 2005 (PDA method). Efficiency was determined by calculating the amount of time required to organise the logistics for 2 consecutive PDA OSCEs and deliver results and grades, compared with the time required for the paper‐based OSCE. User satisfaction was established by using questionnaires to obtain feedback on the assessors' experiences during their first assessments. Results An independent groups t ‐test used to compare the means of scores achieved by students in the PDA and paper‐based OSCEs, respectively, showed that the difference in effectiveness was not significant. In terms of efficiency, 77% less time was used for the PDA OSCE in 2004 and 93% less in 2005. Assessor feedback on PDA assessment was overwhelmingly positive. Assessment by PDA was found to be just as effective as and more efficient than paper‐based assessment in practical examinations, and was highly rated by assessors.