Posterior predictive distribution

This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these template messages) This article may be too technical for most readers to understand. Please help improve it to make it understandable to non-experts, without removing the technical details. (March 2016) (Learn how and when to remove this message) This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Posterior predictive distribution" – news · newspapers · books · scholar · JSTOR (August 2014) (Learn how and when to remove this message) (Learn how and when to remove this message)

Part of a series on
Bayesian statistics
Posterior = Likelihood × Prior ÷ Evidence
Background
Bayesian inference Bayesian probability Bayes' theorem Bernstein–von Mises theorem Coherence Cox's theorem Cromwell's rule Principle of indifference Principle of maximum entropy
Model building
Weak prior ... Strong prior Conjugate prior Linear regression Empirical Bayes Hierarchical model
Posterior approximation
Markov chain Monte Carlo Laplace's approximation Integrated nested Laplace approximations Variational inference Approximate Bayesian computation
Estimators
Bayesian estimator Credible interval Maximum a posteriori estimation
Evidence approximation
Evidence lower bound Nested sampling
Model evaluation
Bayes factor Model averaging Posterior predictive
Mathematics portal
v t e

In Bayesian statistics, the posterior predictive distribution is the distribution of possible unobserved values conditional on the observed values.^[1][2]

Given a set of N i.i.d. observations ${\displaystyle \mathbf {X} =\{x_{1},\dots ,x_{N}\}}$, a new value ${\displaystyle {\tilde {x}}}$ will be drawn from a distribution that depends on a parameter ${\displaystyle \theta \in \Theta }$, where ${\displaystyle \Theta }$ is the parameter space.

${\displaystyle p({\tilde {x}}|\theta )}$

It may seem tempting to plug in a single best estimate ${\displaystyle {\hat {\theta }}}$ for ${\displaystyle \theta }$, but this ignores uncertainty about ${\displaystyle \theta }$, and because a source of uncertainty is ignored, the predictive distribution will be too narrow. Put another way, predictions of extreme values of ${\displaystyle {\tilde {x}}}$ will have a lower probability than if the uncertainty in the parameters as given by their posterior distribution is accounted for.

A posterior predictive distribution accounts for uncertainty about ${\displaystyle \theta }$. The posterior distribution of possible ${\displaystyle \theta }$ values depends on ${\displaystyle \mathbf {X} }$:

${\displaystyle p(\theta |\mathbf {X} )}$

And the posterior predictive distribution of ${\displaystyle {\tilde {x}}}$ given ${\displaystyle \mathbf {X} }$ is calculated by marginalizing the distribution of ${\displaystyle {\tilde {x}}}$ given ${\displaystyle \theta }$ over the posterior distribution of ${\displaystyle \theta }$ given ${\displaystyle \mathbf {X} }$:

${\displaystyle p({\tilde {x}}|\mathbf {X} )=\int _{\Theta }p({\tilde {x}}|\theta )\,p(\theta |\mathbf {X} )\operatorname {d} \!\theta }$

Because it accounts for uncertainty about ${\displaystyle \theta }$, the posterior predictive distribution will in general be wider than a predictive distribution which plugs in a single best estimate for ${\displaystyle \theta }$.