Differential operators

-


Given the function \(f(x,y)\), we have considered its partial derivatives, such as

\[\begin{eqnarray*} f_x(x,y) &=&\frac{\partial}{\partial x}f(x,y),\\ f_y(x,y) &=&\frac{\partial}{\partial y}f(x,y). \end{eqnarray*}\]

We may interpret partial differentiation in the following manner:

The derivative \(f_x(x,y)\) is obtained by applying \(\frac{\partial}{\partial x}\) to the function \(f(x,y)\) from the left.

\(\frac{\partial}{\partial x}\) is neither a number nor a function. It's something different that we call a (partial) differential operator. The same argument applies to \(\frac{\partial}{\partial y}\). This interpretation of differential operators turns out to be useful in many situations. For example, for any constants \(a, b\in \mathbb{R}\), we may consider the following operator \(D\):

\[D = a\frac{\partial}{\partial x} + b\frac{\partial}{\partial y}.\]

If we apply this operator to the function \(f(x,y)\) from the left, we obtain \(af_x(x,y) + bf_y(x,y)\) as a result. That is,
\[Df(x,y) = af_x(x,y) + bf_y(x,y).\]
Note that the result is different if we apply the operator to \(f(x,y)\) from the right, which will be another differential operator rather than a function:
\[f(x,y)D = af(x,y)\frac{\partial}{\partial x} + bf(x,y)\frac{\partial}{\partial y}.\]
In other words, the "product" between an operator and a function is not commutative.


Let \(D\) and \(E\) be differential operators. We define their "product" (composition) \(DE\) as follows: For the function \(f(x,y)\),
\[DEf(x,y) = D(Ef(x,y)).\]
That is, first apply \(E\) to \(f(x,y)\), then apply \(D\) to the result. When \(D = E\), we write \(D^2 = DD\). For example,
\[\begin{eqnarray*} \frac{\partial^2}{\partial x^2} &=& \left(\frac{\partial}{\partial x}\right)^2,\\ \frac{\partial^2}{\partial x\partial y} &=& \frac{\partial}{\partial x}\cdot \frac{\partial}{\partial y}. \end{eqnarray*}\]

Example (\(\Delta\), Laplacian). The differential operator defined by
\[\Delta = \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2}\]
is called the (two-variable) Laplacian after the French mathematician Pierre-Simon Laplace (1749-1827).
\[\Delta f(x,y) = f_{xx}(x,y) + f_{yy}(x,y).\]
The Laplacian appears in many physical problems such as heat transfer and diffusion. □

Example (\(\nabla\), nabla). We can formally define an ordered pair of the partial differential operators:
\[\nabla = \left(\frac{\partial}{\partial x}, \frac{\partial}{\partial y}\right).\]
This operator is called the nabla. When this operator is applied to a function \(f(x,y)\), we have an ordered pair of partial derivatives:
\[\nabla f(x,y) = \left(\frac{\partial}{\partial x}f(x,y), \frac{\partial}{\partial y}f(x,y)\right) = (f_x(x,y), f_y(x,y)).\]
The result is a vector function composed of the partial derivatives of \(f(x,y)\).
The nabla may be regarded as a vector. Suppose we have a vector function \(\mathbf{u}(x,y) = (u(x,y), v(x,y))\). Then, we can take their dot (scalar) product:
\[\nabla \cdot \mathbf{u}(x,y) = \frac{\partial}{\partial x}u(x,y) + \frac{\partial}{\partial y}v(x,y) = u_x(x,y) + v_y(x,y).\]
We can also calculate the scalar product between the nabla and itself:
\[\nabla\cdot\nabla = \frac{\partial}{\partial x}\frac{\partial}{\partial x} + \frac{\partial}{\partial y}\frac{\partial}{\partial y} = \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} = \Delta.\]
The result is the Laplacian. □
See also: Nabla symbol

Example. For \(a,b\in\mathbb{R}\), let \(a\frac{\partial}{\partial x} + b\frac{\partial}{\partial y}\) be a differential operator that operates on functions of class \(C^{\infty}\). Then, \(\frac{\partial^2}{\partial x\partial y} = \frac{\partial^2}{\partial y\partial x}\). Accordingly, the following holds:
\[\begin{equation*} \left(a\frac{\partial}{\partial x} + b\frac{\partial}{\partial y}\right)^2 = a^2\frac{\partial^2}{\partial x^2} + 2ab\frac{\partial^2}{\partial x\partial y} + b^2\frac{\partial^2}{\partial y^2}. \end{equation*}\]
More generally, we have
\[\left(a\frac{\partial}{\partial x} + b\frac{\partial}{\partial y}\right)^n=\sum_{k=0}^{n}\binom{n}{k}a^kb^{n-k}\frac{\partial^n}{\partial x^k\partial y^{n-k}}\tag{Eq:DObinom} \]
where \(\binom{n}{k}\) is the binomial coefficient:
\[\binom{n}{k} = \frac{n!}{k!(n-k)!}.\]
□

Exercise. Prove Eq. (Eq:DObinom). (Hint: Mathematical induction.)□



Comments

Post a Comment

Popular posts from this blog

Multivariate normal distribution is normalized (of course): A proof

-
Image
Let \(\mathbf{X} = (X_1, X_2, \cdots, X_n)^\top\) be a vector of random variables. We say it follows the multivariate normal (Gaussian) distribution if its density is given by \[f(\mathbf{x}) = \frac{1}{\sqrt{(2\pi)^n|\Sigma|}}\exp\left(-\frac{1}{2}(\mathbf{x} - \boldsymbol{\mu})^\top\Sigma^{-1}(\mathbf{x} - \boldsymbol{\mu})\right)\tag{Eq:density}\] where \(\boldsymbol{\mu} = (\mu_1, \mu_2, \cdots, \mu_n)^\top \in \mathbb{R}^n\) is a vector, \(\Sigma\) is a symmetric   positive definite \(n\times n\) matrix, and \(\Sigma^{-1}\) and \(|\Sigma|\) are the inverse and determinant of \(\Sigma\), respectively. It turns out that \(\boldsymbol{\mu}\) and \(\Sigma\) are the mean vector and covariance matrix of \(\mathbf{X}\), respectively. But we will not prove that here. In this post, we will show this density (Eq:density) is normalized (of course). That is, we prove that \[\int_{\mathbb{R}^n}f(\mathbf{x})d\mathbf{x} = 1.\] We assume that you already know how to prove the univariate nor...
Read more

Any covariance matrix is positive semi-definite: A proof

-
Let \(\mathbf{X} = (X_1, X_2, \cdots, X_n)^\top \in \mathbb{R}^n\) be a vector of random variables. The covariance matrix \(\Sigma\) of \(\mathbf{X}\) is a square (\(n\times n\)) matrix whose elements are covariances between the components of \(\mathbf{X}\). That is, \[\Sigma_{ij} = \mathrm{Cov}(X_i,X_j)\] where \(\mathrm{Cov}(X_i,X_j)\) is the covariance between \(X_i\) and \(X_j\) , \(i,j = 1, 2, \cdots, n\):  \[\mathrm{Cov}(X_i, X_j) = \mathbb{E}[(X_i - \mathbb{E}[X_i])(X_j - \mathbb{E}[X_j])].\] Here, \(\mathbb{E}[\cdot]\) indicates the expectation value of a random variable . Any covariance matrix has the following properties: Symmetric. That is, \[\Sigma = \Sigma^\top.\] Positive semi-definite. That is,\[\forall \mathbf{v} \in \mathbb{R}^n, \mathbf{v}^\top\Sigma\mathbf{v} \geq 0.\] See also : Positive definite matrix (Wolfram MathWorld) The symmetry is obvious from the definition of the covariance matrix.  Now, let us prove that the covariance matrix is positive semi-...
Read more

Computing integrals (3): Integration by parts

-
Image
 Sometimes, we may simplify integration by using the product rule of differentiation. This technique is called integration by parts. Theorem (Integration by parts) Let \(f(x)\) and \(g(x)\) be differentiable functions on an open interval \(I\). Then,  \(\int f(x)g'(x)dx = f(x)g(x) - \int f'(x)g(x)dx\); For any \(a, b \in I\), \[\int_a^bf(x)g'(x)dx = \left[f(x)g(x)\right]_a^b - \int_a^bf'(x)g(x)dx.\] Proof . By the product rule, \[[f(x)g(x)]' = f'(x)g(x) + f(x)g'(x)\] so \[f(x)g'(x) = [f(x)g(x)]' - f'(x)g(x).\] By integrating both sides, we have the desired results. ■ Example . Let us find \(\int x\cosh x dx\). \[ \begin{eqnarray*} \int x\cosh x dx &=& \int x(\sinh x)'dx \\ &=& x \sinh x - \int 1 \cdot \sinh x dx\\ &=& x \sinh x - \cosh x + C. \end{eqnarray*} \] Example (eg:recur) . Let us study how we can compute \[I_n = \int \frac{dx}{(x^2 + 1)^n}\] for \(n\in \mathbb{N}\). Note \[I_{n} = \int \fr...
Read more