Next month, 22–24 August 2018, along with Matt Dunlop (Helsinki), Tapio Helin (Helsinki), and Simo Särkkä (Aalto), I will be giving guest lectures at a Summer School / Workshop on Computational Mathematics and Data Science at the University of Oulu, Finland.
While the the other lecturers will treat aspects such as machine learning using deep Gaussian processes, filtering, and MAP estimation, my lectures will tackle the fundamentals of the Bayesian approach to inverse problems in the function-space context, as increasingly demanded by modern applications.
“Well-posedness of Bayesian inverse problems in function spaces: analysis and algorithms”
The basic formalism of the Bayesian method is easily stated, and appears in every introductory probability and statistics course: the posterior probability is proportional to the prior probability times the likelihood. However, for inference problems in high or even infinite dimension, the Bayesian formula must be carefully formulated and its stability properties mathematically analysed. The paradigm advocated by Andrew Stuart and collaborators since 2010 is that one should study the infinite-dimensional Bayesian inverse problem directly and delay discretisation until the last moment. These lectures will study the role of various choices of prior distribution and likelihood and how they lead to well-posed or ill-posed Bayesian inverse problems. If time permits, we will also consider the implications for algorithms, and how Bayesian posterior are summarised (e.g. by maximum a posteriori estimators).
Han Cheng Lie and I have just uploaded a revised preprint of our paper, “Equivalence of weak and strong modes of measures on topological vector spaces”, to the arXiv. This addresses a natural question in the theory of modes (or maximum a posteriori estimators, in the case of posterior measure for a Bayesian inverse problem) in an infinite-dimensional space \(X\). Such modes can be defined either strongly (a la Dashti et al. (2013), via a global maximisation) or weakly (a la Helin & Burger (2015), via a dense subspace \(E \subset X\)). The question is, when are strong and weak modes equivalent? The answer turns out to be rather subtle: under reasonable uniformity conditions, the two kinds of modes are indeed equivalent, but finite-dimensional counterexamples exist when the uniformity conditions fail.
Abstract. A strong mode of a probability measure on a normed space \(X\) can be defined as a point \(u \in X\) such that the mass of the ball centred at \(u\) uniformly dominates the mass of all other balls in the small-radius limit. Helin and Burger weakened this definition by considering only pairwise comparisons with balls whose centres differ by vectors in a dense, proper linear subspace \(E\) of \(X\), and posed the question of when these two types of modes coincide. We show that, in a more general setting of metrisable vector spaces equipped with non-atomic measures that are finite on bounded sets, the density of \(E\) and a uniformity condition suffice for the equivalence of these two types of modes. We accomplish this by introducing a new, intermediate type of mode. We also show that these modes can be inequivalent if the uniformity condition fails. Our results shed light on the relationships between among various notions of maximum a posteriori estimator in non-parametric Bayesian inference.
Han Cheng Lie, Aretha Teckentrup, and I have just uploaded a preprint of our latest article, “Random forward models and log-likelihoods in Bayesian inverse problems”, to the arXiv. This paper considers the effect of approximating the likelihood in a Bayesian inverse problem by a random surrogate, as frequently happens in applications, with the aim of showing that the perturbed posterior distribution is close to the exact one in a suitable sense. This article considers general randomisation models, and thereby expands upon the previous investigations of Stuart and Teckentrup (2017) in the Gaussian setting.
Abstract. We consider the use of randomised forward models and log-likelihoods within the Bayesian approach to inverse problems. Such random approximations to the exact forward model or log-likelihood arise naturally when a computationally expensive model is approximated using a cheaper stochastic surrogate, as in Gaussian process emulation (kriging), or in the field of probabilistic numerical methods. We show that the Hellinger distance between the exact and approximate Bayesian posteriors is bounded by moments of the difference between the true and approximate log-likelihoods. Example applications of these stability results are given for randomised misfit models in large data applications and the probabilistic solution of ordinary differential equations.
The final version of “Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors” has now been published online in Inverse Problems and Imaging; the print version will be available in October.
From 29 August–1 September 2017, the Alan Turing Institute will host a summer school on Mathematical Aspects of Inverse Problems organised by Claudia Schillings (Mannheim) and Aretha Teckentrup (Edinburgh and Alan Turing Institute), two of my former colleagues at the University of Warwick. The invited lecturers are:
- Masoumeh Dashti (Sussex): Bayesian Approach to Inverse Problems
- Michela Ottobre (Maxwell Institute, Edinburgh): Hamiltonian Monte Carlo
- Carola-Bibiane Schönlieb and Clarice Poon (Cambridge): A Comparison of Variational Methods and Deep Neural Networks for Inverse Problems
- Alison Fowler (Reading): The Value of Observations for Numerical Weather Prediction