Local Kernels and the Geometric Structure of Data

We introduce a theory of local kernels, which generalize the kernels used in the standard diffusion maps construction of nonparametric modeling. We prove that evaluating a local kernel on a data set gives a discrete representation of the generator of a continuous Markov process, which converges in the limit of large data. We explicitly connect the drift and diffusion coefficients of the process to the moments of the kernel. Moreover, when the kernel is symmetric, the generator is the Laplace-Beltrami operator with respect to a geometry which is influenced by the embedding geometry and the properties of the kernel. In particular, this allows us to generate any Riemannian geometry by an appropriate choice of local kernel. In this way, we continue a program of Belkin, Niyogi, Coifman and others to reinterpret the current diverse collection of kernel-based data analysis methods and place them in a geometric framework. We show how to use this framework to design local kernels invariant to various features of data. These data-driven local kernels can be used to construct conformally invariant embeddings and reconstruct global diffeomorphisms

Linear theory for filtering nonlinear multiscale systems with model error

We study filtering of multiscale dynamical systems with model error arising from unresolved smaller scale processes. The analysis assumes continuous-time noisy observations of all components of the slow variables alone. For a linear model with Gaussian noise, we prove existence of a unique choice of parameters in a linear reduced model for the slow variables. The linear theory extends to to a non-Gaussian, nonlinear test problem, where we assume we know the optimal stochastic parameterization and the correct observation model. We show that when the parameterization is inappropriate, parameters chosen for good filter performance may give poor equilibrium statistical estimates and vice versa. Given the correct parameterization, it is imperative to estimate the parameters simultaneously and to account for the nonlinear feedback of the stochastic parameters into the reduced filter estimates. In numerical experiments on the two-layer Lorenz-96 model, we find that parameters estimated online, as part of a filtering procedure, produce accurate filtering and equilibrium statistical prediction. In contrast, a linear regression based offline method, which fits the parameters to a given training data set independently from the filter, yields filter estimates which are worse than the observations or even divergent when the slow variables are not fully observed

Nonparametric Uncertainty Quantification for Stochastic Gradient Flows

This paper presents a nonparametric statistical modeling method for quantifying uncertainty in stochastic gradient systems with isotropic diffusion. The central idea is to apply the diffusion maps algorithm to a training data set to produce a stochastic matrix whose generator is a discrete approximation to the backward Kolmogorov operator of the underlying dynamics. The eigenvectors of this stochastic matrix, which we will refer to as the diffusion coordinates, are discrete approximations to the eigenfunctions of the Kolmogorov operator and form an orthonormal basis for functions defined on the data set. Using this basis, we consider the projection of three uncertainty quantification (UQ) problems (prediction, filtering, and response) into the diffusion coordinates. In these coordinates, the nonlinear prediction and response problems reduce to solving systems of infinite-dimensional linear ordinary differential equations. Similarly, the continuous-time nonlinear filtering problem reduces to solving a system of infinite-dimensional linear stochastic differential equations. Solving the UQ problems then reduces to solving the corresponding truncated linear systems in finitely many diffusion coordinates. By solving these systems we give a model-free algorithm for UQ on gradient flow systems with isotropic diffusion. We numerically verify these algorithms on a 1-dimensional linear gradient flow system where the analytic solutions of the UQ problems are known. We also apply the algorithm to a chaotically forced nonlinear gradient flow system which is known to be well approximated as a stochastically forced gradient flow.Comment: Find the associated videos at: http://personal.psu.edu/thb11

Kalman-Takens filtering in the presence of dynamical noise

The use of data assimilation for the merging of observed data with dynamical models is becoming standard in modern physics. If a parametric model is known, methods such as Kalman filtering have been developed for this purpose. If no model is known, a hybrid Kalman-Takens method has been recently introduced, in order to exploit the advantages of optimal filtering in a nonparametric setting. This procedure replaces the parametric model with dynamics reconstructed from delay coordinates, while using the Kalman update formulation to assimilate new observations. We find that this hybrid approach results in comparable efficiency to parametric methods in identifying underlying dynamics, even in the presence of dynamical noise. By combining the Kalman-Takens method with an adaptive filtering procedure we are able to estimate the statistics of the observational and dynamical noise. This solves a long standing problem of separating dynamical and observational noise in time series data, which is especially challenging when no dynamical model is specified