STAT 238 - Bayesian Statistics Lecture Thirty Three

Spring 2026, UC Berkeley

Hamiltonian Monte Carlo¶

In HMC, the proposal $y$ is generated by solving the following ODE:

\begin{align} x(0) = x ~~~~ \dot{x}(0) = z ~~~~ \ddot{x}(t) = \nabla \log \pi(x(t)) ~~~~ y = x(\sigma). \end{align}

(1)

Note that if $\nabla \log \pi(x(t))$ is replaced by $\nabla \log \pi(x)$ , then it is easy to check that $y$ equals the MALA proposal $y = x + (\sigma^2/2)\nabla \log \pi(x) + \sigma z$ .

The ODE (30) can be written in the following alternative first order form:

\begin{align} \begin{pmatrix} \dot{x}(t) \\ \dot{v}(t) \end{pmatrix} = \begin{pmatrix} v(t) \\ \nabla \log \pi(x(t)) \end{pmatrix} ~\text{with initialization}~ \begin{pmatrix} x(0) \\ v(0) \end{pmatrix} = \begin{pmatrix} x \\ z \end{pmatrix} \end{align}

(2)

It is easy to see that (30) and (2) are equivalent. Here is another common way of writing (2). Let

\begin{align} H(x, v) = H(x_1, \dots, x_d, v_1, \dots, v_d) := -\log \pi(x) + \frac{1}{2}\|v\|^2 \end{align}

(3)

This quantity $H(x, v)$ is called the Hamiltonian associated with the ODE (2). It is then easy to see that

\begin{align*} \frac{\partial H}{\partial x} = \left(\frac{\partial H}{\partial x_1}, \dots, \frac{\partial H}{\partial x_d} \right)^T = -\nabla \log \pi(x) \end{align*}

(4)

and

\begin{align*} \frac{\partial H}{\partial v} = \left(\frac{\partial H}{\partial v_1}, \dots, \frac{\partial H}{\partial v_d} \right)^T = v. \end{align*}

(5)

As a result, (2) is equivalent to:

\begin{align} \begin{pmatrix} \dot{x}(t) \\ \dot{v}(t) \end{pmatrix} = \begin{pmatrix} \frac{\partial H}{\partial v} \\ -\frac{\partial H}{\partial x} \end{pmatrix} ~\text{with initialization}~ \begin{pmatrix} x(0) \\ v(0) \end{pmatrix} = \begin{pmatrix} x \\ z \end{pmatrix} \end{align}

(6)

One important property of this ODE is that the Hamiltonian $H(x(t), v(t))$ remains constant in $t$ for all times $t$ :

\begin{align} H(x(t), v(t)) = \text{constant} = H(x(0), v(0)) = H(x, z) = -\log \pi(x) + \frac{1}{2} \|z\|^2. \end{align}

(7)

To see (7), just note that

\begin{align*} \frac{d}{dt} H(x(t), v(t)) &= \sum_{i=1}^d \left( \frac{\partial H}{\partial x_i} \dot{x}_i(t) + \frac{\partial H}{\partial v_i} \dot{v}_i(t) \right) \\ &= \sum_{i=1}^d \left(\frac{\partial H}{\partial x_i} \frac{\partial H}{\partial v_i} + \frac{\partial H}{\partial v_i} \left(- \frac{\partial H}{\partial x_i}\right) \right) = \sum_{i=1}^d \left(\frac{\partial H}{\partial x_i} \frac{\partial H}{\partial v_i} - \frac{\partial H}{\partial v_i} \frac{\partial H}{\partial x_i} \right) = 0 \end{align*}

(8)

Stationarity of $\pi$ for the HMC¶

The transition kernel given by (30) satisfies detailed balance with respect to $\pi$ for every fixed $\sigma > 0$ . A consequence of this is that $\pi$ is invariant (stationary) for this Markov Chain (for every $\sigma$ ). We will sketch a proof of this invariance (and skip the proof of detailed balance).

Stationarity of $\pi$ means that if $x \sim \pi$ , then $y$ given by (30) is also distributed as $\pi$ . In order to verify this, we shall use the following result on density evolutions of ODEs. This result has been popular in the recent literature on generative modeling (see e.g., lai2025principles).

Theorem 1

Consider the ODE

\begin{align} \dot{S}(x, t) = V(t, S(x, t)) ~~\text{with initial condition $S(x, 0) = x$}. \end{align}

(9)

Here $\dot{S}(x, t) = \frac{d}{dt}S(x, t)$ , $x \in \R^d$ and $S(t, \cdot), V(t, \cdot)$ are functions from $\R^d$ to $\R^d$ . Suppose $\rho_b$ is a density on $\R^d$ and let $\rho(t, \cdot)$ be the density of $S(X, t)$ with $X \sim \rho_b$ . Then $\rho(t, x)$ satisfies the following PDE:

\begin{align} \partial_t \rho(t, x) = -\nabla \cdot \left(V(t, x) \rho(t, x) \right) ~~ \text{ with $\rho(0, \cdot) = \rho_b$}. \end{align}

(10)

Here $\nabla \cdot$ , also known as the divergence, is defined as follows. $\nabla \cdot G(x) = \sum_{i=1}^d \frac{\partial}{\partial x_i} G_i(x)$ for a function $G(x) = (G_1(x), \dots, G_d(x))$ . So $\nabla \cdot (V \rho) = \mathrm{div} (V \rho) = \sum_{i=1}^d \frac{\partial}{\partial x_i} (V_i(x, t) \rho(x, t))$ .

In the next lecture, we shall see how this result implies that $\pi$ is invariant for the HMC chain. We will basically apply Theorem 1 with $V$ given by:

\begin{align*} V(t, x, v) = \begin{pmatrix} v \\ \nabla \log \pi(x) \end{pmatrix} = \begin{pmatrix} \frac{\partial H}{\partial v} \\ -\frac{\partial H}{\partial x} \end{pmatrix}. \end{align*}

(11)

We shall also discuss the leapfrog discretization of the Hamiltonian ODE (2), and its Metropolization.

STAT 238 - Bayesian Statistics Lecture Thirty Three

Hamiltonian Monte Carlo¶

Stationarity of π\piπ for the HMC¶

Stationarity of $\pi$ for the HMC¶