An information theoretic alternative to model a natural system using observational information alone

How to define a system? This is a problem faced routinely in science and engineering, with solutions developed from our understanding of the processes inherent, to assessing the underlying structure based on observational evidence alone. In general, system specification involves identifying a few meaningful predictors (from a large enough set that is plausibly related to the response) and formulating a relation between them and the system response being modeled. For systems where physical relationships are less apparent, and sufficient observational records exist, a range of statistical alternatives have been investigated as a possible way of specifying the underlying form. Here we introduce partial information (PI) as a new means for specifying the system, its key advantage being the relative lack of major assumptions about the processes being modeled in order to characterize the complete system. In addition to PI which offers a means of identifying the system predictors of interest, we also introduce the concept of partial weights (PWs) which uses the identified predictors to formulate a predictive model that acknowledges the relative contributions, predictor variables make to the prediction of the response. We assess the utility of the PI‐PW framework using synthetically generated data sets from known linear, nonlinear, and high‐dimensional dynamic yet chaotic systems and demonstrate the efficacy of the procedure in ascertaining the underlying true system with varying extents of observational evidence available. We highlight how this framework can be invaluable in formulating prediction models for natural systems which are modeled using empirical or semiempirical alternatives and discuss current limitations that still need to be overcome.