UMLS Concept Indexing for Production Databases: A Feasibility Study

To the Editor:— In the recently published study by Nadkarni et al.,1 the authors used text-mining software to extract concepts from clinical documents. Matching of these concepts was attempted with the UMLS 99 Metathesaurus. Matches were then categorized as true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) from 8,745 terms in a “training set” and 1,701 terms in a “test set,” for a total of 10,446 terms. True positives were reported as 82.6 percent for the training set and 76.3 percent for the test set.In 1999, we carried out an almost identical study using the identical version of the UMLS, on a larger scale, which resulted in very similar results that were presented at the 1999 AMIA Annual Symposium.2 In our study, 4,994 of the most frequently referenced terms were chosen from 1,000,000 terms randomly extracted from the general Mayo Clinic Master Sheet Index and the Impression/Report/Plan section of the Mayo Clinic clinical notes system, to form a general medicine set. The Mayo Clinic Department of Dermatology independently developed a lexicon of 9,050 …