Multivariate normal distribution is normalized (of course): A proof

Let $\mathbf{X}=(X_1,X_2,\cdots ,X_n{)}^{\mathrm{\top }}$ be a vector of random variables. We say it follows the multivariate normal (Gaussian) distribution if its density is given by

$$\begin{array}{}\text{(Eq:density)}& f(\mathbf{x})=\frac{1}{\sqrt{(2\pi {)}^n|\mathrm{\Sigma }|}}\exp (-\frac{1}{2}(\mathbf{x}-\mathit{\mu }{)}^{\mathrm{\top }}{\mathrm{\Sigma }}^{-1}(\mathbf{x}-\mathit{\mu }))\end{array}$$

where $\mathit{\mu }=({\mu }_1,{\mu }_2,\cdots ,{\mu }_n{)}^{\mathrm{\top }}\in {\mathbb{R}}^n$ is a vector, $\mathrm{\Sigma }$ is a symmetric positive definite $n\times n$ matrix, and ${\mathrm{\Sigma }}^{-1}$ and $|\mathrm{\Sigma }|$ are the inverse and determinant of $\mathrm{\Sigma }$, respectively. It turns out that $\mathit{\mu }$ and $\mathrm{\Sigma }$ are the mean vector and covariance matrix of $\mathbf{X}$, respectively. But we will not prove that here. In this post, we will show this density (Eq:density) is normalized (of course). That is, we prove that

$${\int }_{{\mathbb{R}}^n}f(\mathbf{x})d\mathbf{x}=1.$$

We assume that you already know how to prove the univariate normal distribution is normalized:

$${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\frac{1}{\sqrt{2\pi {\sigma }^2}}\exp (-\frac{(x-\mu {)}^2}{2{\sigma }^2})dx=1.$$

Let's start!

First, by changing the variables $\mathbf{y}=\mathbf{x}-\mathit{\mu }$, we need to prove

$$\begin{array}{}\text{(Eq:Goal)}& {\int }_{{\mathbb{R}}^n}\exp (-\frac{1}{2}{\mathbf{y}}^{\mathrm{\top }}{\mathrm{\Sigma }}^{-1}\mathbf{y})d\mathbf{y}=\sqrt{(2\pi {)}^n|\mathrm{\Sigma }|}.\end{array}$$

What's annoying about this integral is that it contains the cross terms between vector components of $\mathbf{y}$ such as $a_{ij}{y}_i{y}_j$ where $a_{ij}=({\mathrm{\Sigma }}^{-1}{)}_{ij}$. If there were no cross terms, then the only terms are of the form $a_{ii}{y}_i^2$, and we can split the multivariate integral into the product of univariate integrals. We can do this by diagonalizing ${\mathrm{\Sigma }}^{-1}$.

Since $\mathrm{\Sigma }$ is real symmetric, we can diagonalize it by some orthogonal matrix $U$:

$$\mathrm{\Sigma }=US{U}^{\mathrm{\top }}$$

where $S=\mathrm{diag}(s_1,s_2,\cdots ,s_n)$ is the diagonal matrix of the eigenvalues of $\mathrm{\Sigma }$. Since $\mathrm{\Sigma }$ is positive definite, all the eigenvalues are positive. Accordingly (exercise!), the inverse matrix of $\mathrm{\Sigma }$ is diagonalized as

$${\mathrm{\Sigma }}^{-1}=U{S}^{-1}{U}^{\mathrm{\top }}$$

where $S^{-1}=\mathrm{diag}(1/s_1,1/s_2,\cdots ,1/s_n)$ is the inverse matrix of $S$. Let's substitute this into the exponent of the density. We have

$$\begin{array}{rcl}{\mathbf{y}}^{\mathrm{\top }}{\mathrm{\Sigma }}^{-1}\mathbf{y}& =& {\mathbf{y}}^{\mathrm{\top }}U{S}^{-1}{U}^{\mathrm{\top }}\mathbf{y}\\ & =& (U^{\mathrm{\top }}\mathbf{y}{)}^{\mathrm{\top }}{S}^{-1}(U^{\mathrm{\top }}\mathbf{y})\\ & =& {\mathbf{z}}^{\mathrm{\top }}{S}^{-1}\mathbf{z}\end{array}$$

where we set $\mathbf{z}=U^{\mathrm{\top }}\mathbf{y}$. Since $U$ is orthogonal, $|U|=detU=\pm 1$. Therefore, $d\mathbf{y}=|detU|d\mathbf{z}=d\mathbf{z}$, so that

$${\int }_{{\mathbb{R}}^n}\exp (-\frac{1}{2}{\mathbf{y}}^{\mathrm{\top }}{\mathrm{\Sigma }}^{-1}\mathbf{y})d\mathbf{y}={\int }_{{\mathbb{R}}^n}\exp (-\frac{1}{2}{\mathbf{z}}^{\mathrm{\top }}{S}^{-1}\mathbf{z})d\mathbf{z}.$$

Since $S^{-1}$ is a diagonal matrix, the exponent becomes 

$${\mathbf{z}}^{\mathrm{\top }}{S}^{-1}\mathbf{z}=z_1^2/s_1+z_2^2/s_2+\cdots z_n^2/s_n.$$

Thus,

$$\begin{array}{rcl}{\int }_{{\mathbb{R}}^n}\exp (-\frac{1}{2}{\mathbf{z}}^{\mathrm{\top }}{S}^{-1}\mathbf{z})d\mathbf{z}& =& {\int }_{{\mathbf{R}}^n}\prod _{i=1}^n\exp (-\frac{z_i^2}{2s_i})d\mathbf{z}\\ & =& \prod _{i=1}^n{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\exp (-\frac{z_i^2}{2s_i})d{z}_i\\ & =& \prod _{i=1}^n\sqrt{2\pi s_i}\\ & =& \sqrt{(2\pi {)}^n{s}_1{s}_2\cdots s_n}\\ & =& \sqrt{(2\pi {)}^n|\mathrm{\Sigma }|}\end{array}$$

as

$$|\mathrm{\Sigma }|=|US{U}^{\mathrm{\top }}|=|U||S||U^{\mathrm{\top }}|=|S|=s_1{s}_2\cdots s_n.$$

Thus, (Eq:Goal) holds.