Premium
Precision aggregated local models
Author(s) -
Edwards Adam M.,
Gramacy Robert B.
Publication year - 2021
Publication title -
statistical analysis and data mining: the asa data science journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.381
H-Index - 33
eISSN - 1932-1872
pISSN - 1932-1864
DOI - 10.1002/sam.11547
Subject(s) - divide and conquer algorithms , computer science , quadratic equation , partition (number theory) , gaussian process , independence (probability theory) , gaussian , mathematical optimization , scaling , global positioning system , algorithm , mathematics , statistics , telecommunications , physics , geometry , combinatorics , quantum mechanics
Large‐scale Gaussian process (GP) regression is infeasible for large training data due to cubic scaling of flops and quadratic storage involved in working with covariance matrices. Remedies in recent literature focus on divide‐and‐conquer, for example, partitioning into subproblems and inducing functional (and thus computational) independence. Such approximations can be speedy, accurate, and sometimes even more flexible than ordinary GPs. However, a big downside is loss of continuity at partition boundaries. Modern methods like local approximate GPs (LAGPs) imply effectively infinite partitioning and are thus both good and bad in this regard. Model averaging, an alternative to divide‐and‐conquer, can maintain absolute continuity but often over‐smooths, diminishing accuracy. Here we propose putting LAGP‐like methods into a local experts‐like framework, blending partition‐based speed with model‐averaging continuity, as a flagship example of what we call precision aggregated local models (PALM). Using K LAGPs, each selecting n from N total data pairs, our scheme is at most cubic in n , quadratic in K , and linear in N . Extensive empirical illustration shows how PALM is at least as accurate as LAGP, can be much faster, and furnishes continuous predictions. Finally, we propose sequential updating scheme that greedily refines a PALM predictor up to a computational budget.