Premium
Leveraging for big data regression
Author(s) -
Ma Ping,
Sun Xiaoxiao
Publication year - 2014
Publication title -
wiley interdisciplinary reviews: computational statistics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.693
H-Index - 38
eISSN - 1939-0068
pISSN - 1939-5108
DOI - 10.1002/wics.1324
Subject(s) - computer science , big data , data science , sample (material) , computational statistics , analytics , dimension (graph theory) , data mining , cloud computing , dimensionality reduction , construct (python library) , sampling (signal processing) , key (lock) , machine learning , chemistry , mathematics , filter (signal processing) , chromatography , pure mathematics , computer vision , programming language , operating system , computer security
Rapid advance in science and technology in the past decade brings an extraordinary amount of data, offering researchers an unprecedented opportunity to tackle complex research challenges. The opportunity, however, has not yet been fully utilized, because effective and efficient statistical tools for analyzing super‐large dataset are still lacking. One major challenge is that the advance of computing resources still lags far behind the exponential growth of database. To facilitate scientific discoveries using current computing resources, one may use an emerging family of statistical methods, called leveraging. Leveraging methods are designed under a subsampling framework, in which one samples a small proportion of the data (subsample) from the full sample, and then performs intended computations for the full sample using the small subsample as a surrogate. The key of the success of the leveraging methods is to construct nonuniform sampling probabilities so that influential data points are sampled with high probabilities. These methods stand as the very unique development of their type in big data analytics and allow pervasive access to massive amounts of information without resorting to high performance computing and cloud computing. WIREs Comput Stat 2015, 7:70–76. doi: 10.1002/wics.1324 This article is categorized under: Statistical and Graphical Methods of Data Analysis > Dimension Reduction Statistical and Graphical Methods of Data Analysis > Sampling