Empirical Measure of Learnability: A Tool for Semantic Map Validation

The many approaches to semantic mapping developed recently demand a precise measuring device that would, on the one hand, be sensitive to human subjective experiences (and therefore must involve a human in the loop), and on the other hand, allow comparative study and validation of consistency of individual semantic maps. The idea explored in this work is to measure the ability of a human subject to learn a given semantic map, and in this sense to be able to “make sense” of the map, as estimated based on a given set of test words. The paradigm includes allocating previously unseen test words in the map coordinates. The quantitative measure is the Pearson's correlation between actual map coordinates of test words and coordinates assigned by subjects. The preliminary study indicates that the proposed measure is sufficiently sensitive to discriminate individual semantic maps from each other and to rank them by their learnability, related to their internal consistency. Potential applications include evaluation of methods for automated semantic map construction, as well as diagnostics of semantic dementia, affective and personality disorders