An Integrated Approach to Parameter Learning in Infinite-Dimensional Space

The availability of sophisticated modern physics codes has greatly extended the ability of domain scientists to understand the processes underlying their observations of complicated processes, but it has also introduced the curse of dimensionality via the many user-set parameters available to tune. Many of these parameters are naturally expressed as functional data, such as initial temperature distributions, equations of state, and controls. Thus, when attempting to find parameters that match observed data, being able to navigate parameter-space becomes highly non-trivial, especially considering that accurate simulations can be expensive both in terms of time and money. Existing solutions include batch-parallel simulations, high-dimensional, derivative-free optimization, and expert guessing, all of which make some contribution to solving the problem but do not completely resolve the issue. In this work, we explore the possibility of coupling together all three of the techniques just described by designing user-guided, batch-parallel optimization schemes. Our motivating example is a neutron diffusion partial differential equation where the time-varying multiplication factor serves as the unknown control parameter to be learned. We find that a simple, batch-parallelizable, random-walk scheme is able to make some progress on the problem but does not by itself produce satisfactory results. After reducing the dimensionality of the problem using functional principal component analysis (fPCA), we are able to track the progress of the solver in a visually simple way as well as viewing the associated principle components. This allows a human to make reasonable guesses about which points in the state space the random walker should try next. Thus, by combining the random walker’s ability to find descent directions with the human’s understanding of the underlying physics, it is possible to use expensive simulations more efficiently and more quickly arrive at the desired parameter set. 1 Three existing techniques Parameter search problems can be tackled using up to three techniques, which we propose to combine. First, parameter searches in computer simulation are often amenable to batch parallel approaches, in which many simulations are executed at the same time using multi-processor computing. The gains from parallel computing are almost perfect in this context, since there is generally no dependency at all between the different runs. The weakness of relying on batch parallelism alone is that it is susceptible to the curse of dimensionality, as the number of grid points required for a thorough search grows exponentially with the dimension of the problem. This means that some judicious approach is needed when selecting points at which to run simulations. The next two techniques help with that problem. Second, parameter search can be conducted using an automated optimizer. When working with a complicated simulation package and trying to find parameters that reproduce observed data approximately, we are essentially working with an optimization problem where we attempt to minimize the difference between the simulated solution and the observed data. The problem will be made precise below, but we can already make some important observations regarding the nature of the task. First observation: There is no particular reason to believe that the problem is convex, since the relationship between input parameters and simulation output is complicated, which was the very reason that simulations were invoked. Therefore, local search methods will not be adequate. In fact, one can prove that unless some additional assumption is made about the structure of the problem, any algorithm that can successfully optimize arbitrary continuous objective functions must have a search pattern that is dense in