Regularized S‐map for inference and forecasting with noisy ecological time series

AbstractIt is well known that fluctuations of species abundances observed in ecological time series emerge from an interplay between deterministic nonlinear dynamics and stochastic forces. Importantly, nonlinearity and stochasticity introduce significant challenges to the analysis of ecological time series, such as the inference of the effect of species interactions on community dynamics and forecasting of species abundances. Local linear fits with state‐space‐dependent kernel functions, known as S‐maps, provide an efficient method to infer Jacobian coefficients (a proxy for the local effect of species interactions) and to make reliable forecasts from nonlinear time series. Yet, while it has been shown that the S‐map outperforms existing methods for nonparametric inference and forecasting, the methodology is sensitive to process noise. To overcome this limitation, we integrate the S‐map with different regularization schemes. To validate our approach, we test our methodology against different levels of noise and nonlinearity using three standard population dynamics models. We show that an appropriate choice of the regularization scheme, alongside an accurate choice of the kernel functions, can significantly improve the in‐sample inference of Jacobian coefficients and the out‐of‐sample forecast of species abundances in the presence of process noise. We further validate our methodology using two empirical time series of marine microbial communities. Our results illustrate that the regularized S‐map is an efficient method for nonparametric inference and forecasting from noisy, nonlinear, ecological time series. Yet, attention must be paid on the regularization scheme and the structure of the kernel for whether inference or forecasting is the ultimate goal of a research study.