Regression with I-priors

The event is taking part on the Friday, Apr 28th 2017 at 11.00
Theme/s: Statistics
Location of Event: Alan Turing 306
This event is a: Public Seminar

Abstract: The aim of this talk is to describe an (empirical) Bayes estimator for parametric and nonparametric regression functions with good frequentist properties. Our estimator is the posterior distribution based on a proper objective prior for the regression function, which we call I-prior. We show that the posterior mean under the I-prior has some potential advantages over the Tikhonov regularizer. This is because, at least in some common settings, the regularizer undersmoothes and the I-prior estimator does not. Our small sample simulations, asymptotic results, and real data analyses all point towards an advantage of the I-prior methodology.

We assume the regression function lies in a reproducing kernel Hilbert space (RKHS) over some covariate space. The I-prior is obtained by maximizing entropy over a suitable subset of distributions, and in the present case turns out to be Gaussian, with mean chosen a priori, and covariance kernel proportional to the Fisher information for the regression function. This has the intuitively appealing property that the more information is available about a linear functional of the regression function, the larger its prior variance, and, broadly speaking, the less influential the prior is on the posterior.

The I-prior methodology has some particularly nice properties if the regression function is assumed to lie in the fractional Brownian motion (FBM) RKHS over Euclidean space. Firstly, one parameter less needs to be estimated than using standard kernel methods, e.g., based on exponential or Matern kernels. Secondly, the functions in an FBM RKHS, and the corresponding I-prior realizations, provide a suitable range of smoothness for many applications. This is in contrast to FBM process realizations, which may be too rough. In particular, we show that the functions in an FBM RKHS with Hurst coefficient $\gamma$, as well as the associated I-prior realizations, are locally H\"older of order $2\gamma$, implying differentiability if $\gamma\ge 1/2$. This may be compared with FBM process realizations, which are H\"older only of order $\gamma$ and are not differentiable.

Report an error on this page

External Speakers

Wicher Bergsma, London School of Economics