STAT 238 - Bayesian Statistics Lecture Three Spring 2026, UC Berkeley

In the last lecture, we started discussing this example.

Example 3: Inference from measurements¶

Suppose a scientist makes 6 numerical measurements 26.6, 38.5, 34.4, 34, 31, 23.6 on an unknown real-valued physical quantity $\theta$. On the basis of these measurements, what can be inferred about $\theta$?

Here is the Bayesian solution to this problem. The first step is modeling where we have to write the likelihood and prior. The likelihood represents the probability of the observed data conditional on parameter values. Here the main parameter is $\theta$. In order to write the probability of the observed data, it is helpful to introduce another parameter $\sigma$ which represents the scale of the noise inherent in the measurement process.

So our parameter vector is $(\theta, \sigma)$. We work with the normal likelihood:

where $n = 6$ and $x_1 = 26.6, x_2 = 38.5, x_3 = 34.4, x_4 = 34, x_5 = 31, x_6 = 23.6$ denote the observed data points. More formally, you can arrive at this likelihood in the following way. Denote potential measurements by $X_1, \dots, X_n$. Each actual measurement will have some rounding error so the data point 26.6 should be understood as belonging to the interval $[26.6 - \delta, 26.6 + \delta]$ for some small rounding error $\delta$. So the likelihood is:

Assuming $\delta$ is small, we can use probability-density approximation to write

We are now assuming that:

This leads to the likelihood (6) (note that $\delta^n$ is being dropped as it is a constant of proportionality which does not affect any further calculations).

Here is a short digression on the likelihood and the assumption (9).

A lot of the time, the likelihood assumption (9) is written as:

Strictly speaking (9) and (5) are not the same. This is because (5) is equivalent to

(6) is much stronger than (9) because $u_1. \dots, u_n$ are completely arbitrary while in (9) the points $x_1, \dots, x_n$ are not arbitrary (they simply equal the observed data).

For example, note that if we assume

then again we arrive at the same likelihood. But now (5) is no longer true.

To complete the modeling step, we need to describe the prior on $\theta, \sigma$. We assume

for a very large positive constant $C$. The idea here is that we are allowing $\theta$ and $\log \sigma$ to take values essentially on the entire real line and not expressing preference for any one value over any other. In terms of the densities, (8) is the same as

To now get the posterior (which is the joint density of $\theta, \sigma$ given the observed data), we use Bayes rule as

The constant underlying proportionality above is determined by the overall integral being one:

This is the joint posterior density of $\theta$ and $\sigma$. If we only want the posterior density for $\theta$, we use the sum rule of probability to integrate out $\sigma$:

Because $C$ is large, the limits of the integral can be taken to be 0 and $\infty$ leading to

The factor $2^{(n/2) - 1} \Gamma(n/2)$ does not depend on $\theta$ so it can be absorbed in the constant of proportionality as well leading to:

where $S(\theta)$ is the sum of squares term:

If $C$ is large, then the indicator can be dropped (because it will essentially be always 1) so the posterior becomes:

Thus the posterior is inversely proportional to $S(\theta)^{n/2}$. This means that the posterior model will be at the least squares estimator which is the mean $\hat{\theta} = \bar{x} = (x_1 + \dots + x_n)/n$. It is cleaner to write the above posterior as:

Because $S(\theta) = S(\hat{\theta}) + n(\theta - \hat{\theta})^2$, we can also rewrite the posterior as:

It can be shown (left as exercise) that this is related to the $t$-density (see {https://en.wikipedia.org/wiki/Student

the above is equivalent to

where $t_{n-1}$ denotes the $t$-density with $n-1$ degrees of freedom. Note that $\hat{\theta} = \bar{x}$ and $S(\hat{\theta}) = \sum_{i=1}^n (x_i - \bar{x})^2$.

So the Bayesian point estimate is simply $\hat{\theta} = \bar{x}$ (this is the posterior mean, median and mode!). A $100(1 - \alpha)$% uncertainty interval for $\theta$ is given by:

where $t_{n-1, \alpha/2}$ is the $(1 - \alpha/2)$ quantile of the Student $t$-distribution with $n-1$ degrees of freedom. This uncertainty interval is sometimes referred to as the Bayesian Credible Interval.

Check that $\alpha = 0.05$ leads to the interval $[25.598, 37.102]$.

Frequentist Solution¶

It turns out that the Bayesian credible interval (20) is also the standard frequentist confidence interval in this problem. Specifically, we have (below $\bar{X} = (X_1 + \dots + X_n)/n$)

under the assumption:

This is because

In the above probability statement, $\theta$ is held fixed, and the probability is taken with respect to $X_1, \dots, X_n$ which are i.i.d $N(0, \sigma^2)$.

Observe the difference between (19) and (22). In (19), the data is held fixed at the observed values and the probability is with respect to $\theta$. In (22), $\theta$ is held fixed and the probability is with respect to the random variables $X_1, \dots, X_n$ which are supposed to represent data.

In this problem, the standard Bayesian inference and standard frequentist inference exactly coincide.

However, it is very easy to break this coincidence. We shall discuss this in the next lecture.