de Bruijn identity

Wiki card — concept node for exp-families-stability.

(i) Formal statement

Let $X$ be an $\mathbb{R}^d$-valued random variable with density $p$ (smooth, sufficiently integrable), and let $X_t = X + \sqrt{t}\, Z$ with $Z \sim \mathcal{N}(0, I_d)$ independent. Write $p_t := \mathrm{law}(X_t) = p * \gamma_{\sqrt{t}}$ and $h(X_t) := -\int p_t \log p_t\, dx$ for the differential entropy. Then

\[ \boxed{\;\frac{d}{dt}\, h(X_t) \;=\; \tfrac12\, J(X_t)\;} \tag{dB} \]

where

\[ J(X_t) \;:=\; \int p_t(x)\, \|\nabla \log p_t(x)\|^2\, dx \;=\; \mathbb{E}\bigl[\|\nabla\log p_t(X_t)\|^2\bigr] \]

is the Fisher information of $X_t$ (location Fisher information). Attributed to de Bruijn in Stam (1959); independent appearance in Linnik and in Rényi.

Pointwise lift. Writing $f_t := -\log p_t$, the heat equation $\partial_t p_t = \tfrac12 \Delta p_t$ and Cole–Hopf give $\partial_t f_t = \tfrac12 \Delta f_t - \tfrac12\|\nabla f_t\|^2$. Integrating against $p_t$: \[ \frac{d}{dt} h(X_t) = -\int (\partial_t p_t) \log p_t - \int p_t \frac{\partial_t p_t}{p_t} = -\int p_t\, \partial_t f_t = \int p_t \bigl(\tfrac12 \|\nabla f_t\|^2 - \tfrac12 \Delta f_t\bigr). \] The $\tfrac12 \Delta f_t$ term integrates to $0$ after IBP (using $\nabla p_t = -p_t \nabla f_t$), leaving $\tfrac12 \mathbb{E}\|\nabla f_t\|^2 = \tfrac12 J(X_t)$.

(ii) Role in Q1 / Q2 / Q3

Q3 (integrated vs pointwise). de Bruijn is the integrated form of the Hamilton–Jacobi equation (6). Integrating (6) against $p_{\theta_t}$: $\tfrac{d}{dt}\log Z_{\theta_t} + \dot\theta_t^\top\eta_t = -\tfrac12 \mathbb{E}[\|(\nabla\varphi)\theta_t\|^2] + \tfrac12 \theta_t^\top \mathbb{E}[\Delta\varphi]$. The first term on the RHS is $-\tfrac12 J(X_t)$ (since $\nabla f_t = (\nabla\varphi)\theta_t$); the second term integrates by parts to $-\tfrac12 J(X_t)$ also. So (6) integrated is (dB).
Q1 (obstruction of Shannon persona). A statistic $\varphi$ fails to be stable under $*\gamma_\sigma$ iff Fisher information does not stay expressible as a linear function of moments of $\varphi$ along the heat flow. Since $J(X_t)$ must be affine in the sufficient statistics for every $\theta$, (dB) imposes a rigidity constraint that fails for $\deg\varphi \ge 3$.
Q2. For the Gaussian family, $J(\mathcal{N}(\mu, \Sigma)) = \mathrm{tr}(\Sigma^{-1})$, so $\tfrac{d}{dt}h = \tfrac12 \mathrm{tr}(\Sigma_t^{-1})$ with $\Sigma_t = \Sigma + t I$ — direct verification of the Riccati flow (§5 of invariant-reformulation.md).

(iii) References

Stam (1959) — Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon. Original proof; de Bruijn attribution. doi:10.1016/S0019-9958(59)90348-1.
Blachman (1965) — The Convolution Inequality for Entropy Powers. doi:10.1109/TIT.1965.1053768.
Dembo, Cover, Thomas (1991) — Information-Theoretic Inequalities. doi:10.1109/18.104312.
Barron (1986) — Entropy and the Central Limit Theorem. doi:10.1214/aop/1176992632.

(iv) Worked miniature — Gaussian sanity check

Take $X \sim \mathcal{N}(0, 1)$ in $d = 1$, so $X_t \sim \mathcal{N}(0, 1 + t)$.

Entropy. $h(X_t) = \tfrac12 \log(2\pi e (1 + t))$, hence $\tfrac{d}{dt}h = \tfrac{1}{2(1 + t)}$.
Fisher info. For $\mathcal{N}(0, \sigma^2)$, $\log p = -\tfrac{x^2}{2\sigma^2}
- \mathrm{cst}$, $\nabla\log p = -x/\sigma^2$, so $J = \mathbb{E}[X^2]/\sigma^4 = 1/\sigma^2 = 1/(1+t)$.
Check. $\tfrac12 J = \tfrac{1}{2(1+t)} = \tfrac{d}{dt}h$. ✓

Observation. On the Gaussian submanifold, Fisher info and entropy derivative are each expressible in the parameters ($\sigma^2 = 1 + t$). This is the hallmark of the stable class — Q1 is asking: for which $\varphi$ does this happen identically in $\theta$?