Premium
In reference to Objective assessment in residency‐based training for transoral robotic surgery
Author(s) -
Gomez Ernest D.,
Hashimoto Daniel A.,
Aggarwal Rajesh,
O'Malley Bert W.,
Weinstein Gregory S.
Publication year - 2013
Publication title -
the laryngoscope
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.181
H-Index - 148
eISSN - 1531-4995
pISSN - 0023-852X
DOI - 10.1002/lary.23920
Subject(s) - medicine , otorhinolaryngology , head and neck surgery , general surgery , head and neck , surgery
Dear Editor: We commend the authors of ‘‘Objective assessment in residency-based training for transoral robotic surgery’’ for taking the first steps toward developing a structured robotic-surgery training curriculum for residents. However, we would like to raise several points regarding the study’s design and analysis that may be of value. First, the authors should describe the type of instruction or feedback provided to subjects during the training modules. These details are essential to determining whether performance improvements between sessions result from an educational intervention or from the natural learning effects of repeating the tasks. If an intervention is being applied, a control group receiving an alternative or no intervention should be present in order to make valid claims regarding the value of the curriculum. Second, the authors note that the average kappa values ranging from 0.3 to 0.54 suggest moderate to strong interobserver agreement, but the surgical education literature generally (and arbitrarily) considers values above 0.8 to be ‘‘good.’’ When using Cohen’s kappa for the OSATS global rating scale, expert observers should consider meeting prior to rating study videos in order to develop a consensus regarding the value of each item on the global rating scale by evaluating pilot study videos for each task. This practice is helpful because kappa is sensitive to single-point differences in Likert-scale ratings, which can contribute to the lower values reported by the authors. Finally, the authors mention that learning curves were computed for each task, but neither the comparison groups nor the statistical test are specified. There are many viable options for the statistical analysis of learning curves. ANOVA and the Friedman test can be used to compare mean performance levels of trainees at different time points. Least squares regression can also be used to measure performance as a function of experience. It is important to use statistical techniques when analyzing learning curves as the traditional practice of commenting on the appearance of performance trends is obsolescent. The authors have presented an excellent example of the work that is essential to improving the timeand cost-efficiency of resident training on novel surgical techniques that are entering the field of otolaryngology. The points we have made touch upon difficult concepts in surgical education research that are continually evolving and that at times are controversial. We hope that our commentary might provide helpful insight to the authors and others in the field as we conduct research on surgical skill assessment.