Premium
A probabilistic nearest neighbour method for statistical pattern recognition
Author(s) -
Holmes C. C.,
Adams N. M.
Publication year - 2002
Publication title -
journal of the royal statistical society: series b (statistical methodology)
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.523
H-Index - 137
eISSN - 1467-9868
pISSN - 1369-7412
DOI - 10.1111/1467-9868.00338
Subject(s) - probabilistic logic , nearest neighbour , computer science , neighbourhood (mathematics) , k nearest neighbors algorithm , data mining , statistical model , set (abstract data type) , point (geometry) , artificial intelligence , pattern recognition (psychology) , algorithm , machine learning , mathematics , mathematical analysis , geometry , programming language
Summary. Nearest neighbour algorithms are among the most popular methods used in statistical pattern recognition. The models are conceptually simple and empirical studies have shown that their performance is highly competitive against other techniques. However, the lack of a formal framework for choosing the size of the neighbourhood k is problematic. Furthermore, the method can only make discrete predictions by reporting the relative frequency of the classes in the neighbourhood of the prediction point. We present a probabilistic framework for the k ‐nearest‐neighbour method that largely overcomes these difficulties. Uncertainty is accommodated via a prior distribution on k as well as in the strength of the interaction between neighbours. These prior distributions propagate uncertainty through to proper probabilistic predictions that have continuous support on (0, 1). The method makes no assumptions about the distribution of the predictor variables. The method is also fully automatic with no user‐set parameters and empirically it proves to be highly accurate on many bench‐mark data sets.