Premium
Feature extraction through local learning
Author(s) -
Sun Yijun,
Wu Dapeng
Publication year - 2009
Publication title -
statistical analysis and data mining: the asa data science journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.381
H-Index - 33
eISSN - 1932-1872
pISSN - 1932-1864
DOI - 10.1002/sam.10028
Subject(s) - margin (machine learning) , computer science , feature extraction , generalization , artificial intelligence , pattern recognition (psychology) , feature (linguistics) , principal component analysis , decomposition , function (biology) , algorithm , data mining , machine learning , mathematics , mathematical analysis , linguistics , philosophy , ecology , evolutionary biology , biology
RELIEF is considered one of the most successful algorithms for assessing the quality of features. It has been recently proved that RELIEF is an online learning algorithm that solves a convex optimization problem with a margin‐based objective function. Starting from this mathematical interpretation, we propose a novel feature extraction algorithm, referred to as local feature extraction (LFE), as a natural generalization of RELIEF. LFE collects discriminant information through local learning and can be solved as an eigenvalue decomposition problem with a closed‐form solution. A fast implementation of LFE is derived. Compared to principal component analysis, LFE also has a clear physical meaning and can be implemented easily with a comparable computational cost. Compared to other feature extraction algorithms, LFE has an explicit mechanism to remove irrelevant features. Experiments on synthetic and real‐world data are presented. The results demonstrate the effectiveness of the proposed algorithm. © 2009 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 2: 34‐47, 2009