Kernels arising from the Fourier series

-

Here, we present a different way of looking at the Fourier series using kernels. We see that the partial Fourier sum of a function \(f(x)\) can be expressed as the convolution between \(f\) and the Dirichlet kernel. We have already seen that the partial Fourier sum converges uniformly to \(f\) if \(f(x)\) is continuously differentiable. Similarly, we show that the convolution of the function \(f\) and the Fejér kernel converges uniformly to \(f\), but this time, if \(f(x)\) is continuous. These kernels are used to prove the \(L^2\) convergence of the Fourier series.



Definition (Kernel function)

Let \(X\) be a set, and \(K: X\times X \to \mathbb{R}\) be a two-variable function. \(K(x,y)\) is said to be a kernel function (or integral kernel or nucleus) if it satisfies the following conditions:

  1. (symmetry) For any \(x, y\in X\), \[K(x,y) = K(y,x).\]
  2.  (positive semi-definiteness) For any \(n \in \mathbb{N}\), \(\{x_1, x_2, \cdots, x_n\} \subset X\), \(\{c_1, c_2, \cdots, c_n\} \subset \mathbb{R}\), \[\sum_{i=1}^{n}\sum_{j=1}^{n}c_ic_jK(x_i,x_j) \geq 0.\]

We now consider kernel representations of the Fourier series.

The Dirichlet kernel

Consider the partial sum \(S_n(x)\) of \(S[f]\):
\[\begin{eqnarray*} S_n(x) &=& \sum_{|k|\leq n}c_ke^{ikx}\\ &=& \sum_{|k|\leq n}\left(\frac{1}{2\pi}\int_{-\pi}^{\pi}f(y)e^{-iky}dy\right)e^{ikx}\\ &=& \int_{-\pi}^{\pi}\frac{1}{2\pi}\sum_{k=-n}^{n}e^{ik(x-y)}f(y)dy. \end{eqnarray*}\]
Let us define \(D_n(\theta) ~ (\theta \in \mathbb{R})\) by
\[D_n(\theta) = \frac{1}{2\pi}\sum_{k=-n}^{n}e^{ik\theta}.\tag{eq:Ddef}\]
Noting
\[\begin{eqnarray*} \sum_{k=-n}^{n}e^{ik\theta} & = & \frac{e^{-in\theta} - e^{i(n+1)\theta}}{1 - e^{i\theta}}\\ &=& \frac{e^{i(n+1/2)\theta} - e^{-i(n+1/2)\theta}}{e^{i\theta/2} - e^{-i\theta/2}}\\ &=& \frac{\sin\left(n + \frac{1}{2}\right)\theta}{\sin\frac{\theta}{2}}, \end{eqnarray*}\]
we have
\[S_n[f](x) = S_n(x) = \int_{-\pi}^{\pi}D_n(x-y)f(y)dy\tag{eq:Dfconv}\]
where
\[D_n(x-y) = \frac{1}{2\pi}\frac{\sin\left(n + \frac{1}{2}\right)(x-y)}{\sin\frac{x-y}{2}} ~ (n = 0, 1, \cdots)\]
is called the Dirichlet kernel. As the name suggests, it is a kernel function. \(D_n(\theta)\) is an even function of \(\theta\), and has a period of \(2\pi\). Thus, \(D_n(x - y)\) is symmetric with respect to \(x\) and \(y\). Using (eq:Ddef), we can show that it is also positive semi-definite (exercise!).
For the constant function \(f(x) = 1\),  \(c_n = 0\) for \(n \neq 0\) and \(c_0 = 1\) so that \(S_n[f](x) = 1\) for all \(n\). Therefore
\[\int_{-\pi}^{\pi}D_n(x-y)dy = 1 ~~~ (n = 0, 1, \cdots).\]
The following figure shows the Dirichlet kernels with \(n = 0, 1, 5, 20\).

The integral of the form of (Eq:Dfconv) is so common that it has a name.

Definition (Convolution)

Let \(f, g \in \mathcal{P}_{2\pi}\) . The convolution of \(f\) and \(g\), often denoted as \(f\ast g\), is defined as
\[(f\ast g)(x) = \int_{-\pi}^{\pi}f(x - y)g(y)dy.\]

See also: Convolution (Wikipedia)

Thus, we have
\[S_n(x) = (D_n\ast f)(x).\]

The Fejér kernel

Given the sequence \(\{a_n\}_{n=0}^{\infty}\), define another sequence \(\{b_n\}\) by
\[b_n = \frac{1}{n + 1}\sum_{k=0}^{n}a_k = \frac{a_0 + a_1 + \cdots + a_n}{n+1}.\]
It is known that, if \(\{a_n\}\) converges, then \(\{b_n\}\) converges to the same value and the convergence is generally ``faster'' than \(\{a_n\}\). Based on this observation, we introduce another kernel representation of the Fourier series. For the partial sum \(S_n\) of the Fourier series of \(f\), we define Fejér's partial sum \(\sigma_n[f](x)\) by
\[\sigma_n[f](x) = \sigma_n(x) = \frac{1}{n+1}\sum_{k=0}^{n}S_k(x) ~~~ (n = 0, 1, \cdots).\tag{eq:fejer}\]
The integral representation of \(\sigma_n(x)\) can be obtained as follows.
\[\begin{eqnarray*} \sigma_n(x) &=& \frac{1}{n+1}\sum_{k=0}^{n}\int_{-\pi}^{\pi}D_k(x-y)f(y)dy\\ &=& \int_{-\pi}^{\pi}\left(\frac{1}{n+1}\sum_{k=0}^{n}D_k(x-y)\right)f(y)dy \end{eqnarray*}\]
where \(D_n\) is the Dirichlet kernel. Noting
\[\begin{eqnarray*} \sum_{k=0}^{n}\sin\left(k + \frac{1}{2}\right)\theta &=& \frac{1}{\sin\frac{\theta}{2}}\sum_{k=0}^{n}\sin\left(k + \frac{1}{2}\right)\theta\cdot {\sin\frac{\theta}{2}}\\ &=& \frac{1}{2\sin\frac{\theta}{2}}\sum_{k=0}^{n}(\cos k\theta - \cos(k+1)\theta)\\ &=& \frac{1}{2\sin\frac{\theta}{2}}(1 - \cos(n+1)\theta)\\ &=& \frac{\sin^2\frac{n+1}{2}\theta}{\sin\frac{\theta}{2}}, \end{eqnarray*}\]
we have
\[\sigma_n[f](x) = \sigma_n(x) = \int_{-\pi}^{\pi}F_n(x-y)f(y)dy = (F_n\ast f)(x)\]
where
\[F_n(x-y) = \frac{1}{2(n+1)\pi}\frac{\sin^2\left(\frac{n+1}{2}(x-y)\right)}{\sin^2\frac{x-y}{2}} ~~~ (n = 0, 1,\cdots).\]
\(F_n(x-y)\) is called the Fejér kernel. \(F_n(\theta)\) is an even function and has a period of \(2\pi\), so \(F_n(x-y)\) is symmetric. The positive semi-definiteness is inherited from that of the Dirichlet kernel.
For the constant function \(f(x) = 1\), \(S_n \equiv 1\) (\(n=0, 1, \cdots\)) so that \(\sigma_n(x) \equiv 1\), and hence
\[\int_{-\pi}^{\pi}F_n(x-y)dy = 1.\tag{eq:Fnormal}\]
Unlike \(D_n\), \(F_n\) is non-negative:
\[F_n(x-y) \geq 0 ~~~ (\forall x,y).\]
The following figure shows the Fejér kernels for \(n = 0, 1, 5, 20\).


Lemma (Uniform convergence of \(\sigma_n[f]\))

If \(f\in C_{2\pi}^{0}\), then \(\sigma_n[f]\) converges uniformly to \(f\). 

Proof. Using (eq:Fnormal), we have
\[f(x) = \int_{-\pi}^{\pi}F_n(x-y)f(x)\,dy.\]
Therefore,
\[f(x) - \sigma_n(x) = \int_{-\pi}^{\pi}F_n(x-y)\{f(x) - f(y)\}dy \tag{eq:fms}\]
where the range of integration could be any one period.

For any \(\varepsilon > 0\), there exists a \(\delta > 0\) such that 
\[|x - y| < \delta \implies |f(x) - f(y)| < \varepsilon\]
as \(f\) is uniformly continuous. Fixing this \(\delta\), we split the right-hand side of (eq:fms) into two parts:
\[f(x) - \sigma_n(x) = I_1 + I_2\]
where
\[\begin{eqnarray*} I_1 &=& \int_{|y - x|<\delta}F_n(x-y)\{f(x) - f(y)\}dy,\\ I_2 &=& \int_{\delta\leq |y - x|\leq \pi}F_n(x-y)\{f(x) - f(y)\}dy. \end{eqnarray*}\]
Since \(F_n\) is non-negative and normalized (eq:Fnormal),
\[|I_1| \leq \varepsilon\int_{|y-x|< \delta}F_n(x-y)dy \leq \varepsilon\int_{|y-x|< \pi}F_n(x-y)dy = \varepsilon.\]

Next, if \(|x-y|\geq \delta\), noting that
\[F_n(x-y)\leq \frac{1}{2(n+1)\pi}\cdot\frac{1}{\sin^2\frac{\delta}{2}}\]
and letting \(M = \max_{|y|\leq \pi}|f(y)|\),
\[\begin{eqnarray*} |I_2| &\leq &\int_{\delta\leq |y-x|\leq \pi}\frac{1}{2(n+1)\pi}\cdot\frac{1}{\sin^2\frac{\delta}{2}}\cdot 2Mdy\\ &\leq& \frac{2M}{(n+1)\sin^2\frac{\delta}{2}}. \end{eqnarray*}\]
Thus, for a sufficiently large \(N_1\), we have
\[n \geq N_1 \implies |I_2| < \varepsilon.\]
Putting these together, we have
\[|f(x) - \sigma_n(x)| < 2\varepsilon ~~~ (n \geq N_1).\]
Here, \(N_1\) does not depend on \(x\), which proves that \(\sigma_n\) uniformly converges to \(f\) with respect to \(x\). ■


Comments

Post a Comment

Popular posts from this blog

Multivariate normal distribution is normalized (of course): A proof

-
Image
Let \(\mathbf{X} = (X_1, X_2, \cdots, X_n)^\top\) be a vector of random variables. We say it follows the multivariate normal (Gaussian) distribution if its density is given by \[f(\mathbf{x}) = \frac{1}{\sqrt{(2\pi)^n|\Sigma|}}\exp\left(-\frac{1}{2}(\mathbf{x} - \boldsymbol{\mu})^\top\Sigma^{-1}(\mathbf{x} - \boldsymbol{\mu})\right)\tag{Eq:density}\] where \(\boldsymbol{\mu} = (\mu_1, \mu_2, \cdots, \mu_n)^\top \in \mathbb{R}^n\) is a vector, \(\Sigma\) is a symmetric   positive definite \(n\times n\) matrix, and \(\Sigma^{-1}\) and \(|\Sigma|\) are the inverse and determinant of \(\Sigma\), respectively. It turns out that \(\boldsymbol{\mu}\) and \(\Sigma\) are the mean vector and covariance matrix of \(\mathbf{X}\), respectively. But we will not prove that here. In this post, we will show this density (Eq:density) is normalized (of course). That is, we prove that \[\int_{\mathbb{R}^n}f(\mathbf{x})d\mathbf{x} = 1.\] We assume that you already know how to prove the univariate nor...
Read more

Any covariance matrix is positive semi-definite: A proof

-
Let \(\mathbf{X} = (X_1, X_2, \cdots, X_n)^\top \in \mathbb{R}^n\) be a vector of random variables. The covariance matrix \(\Sigma\) of \(\mathbf{X}\) is a square (\(n\times n\)) matrix whose elements are covariances between the components of \(\mathbf{X}\). That is, \[\Sigma_{ij} = \mathrm{Cov}(X_i,X_j)\] where \(\mathrm{Cov}(X_i,X_j)\) is the covariance between \(X_i\) and \(X_j\) , \(i,j = 1, 2, \cdots, n\):  \[\mathrm{Cov}(X_i, X_j) = \mathbb{E}[(X_i - \mathbb{E}[X_i])(X_j - \mathbb{E}[X_j])].\] Here, \(\mathbb{E}[\cdot]\) indicates the expectation value of a random variable . Any covariance matrix has the following properties: Symmetric. That is, \[\Sigma = \Sigma^\top.\] Positive semi-definite. That is,\[\forall \mathbf{v} \in \mathbb{R}^n, \mathbf{v}^\top\Sigma\mathbf{v} \geq 0.\] See also : Positive definite matrix (Wolfram MathWorld) The symmetry is obvious from the definition of the covariance matrix.  Now, let us prove that the covariance matrix is positive semi-...
Read more

Properties of differentiation

-
Image
 We show some basic properties of differentiation, such as linearity, the product rule, the quotient rule, the chain rule, etc. We also introduce higher-order derivatives and differentiability classes. Theorem (Properties of differentiation) Let \(f(x)\) and \(g(x)\) be differentiable functions on an open interval \(I\). (Linearity) \[\frac{d}{dx}[kf(x) + lg(x)] = kf'(x) + lg'(x)\] where \(k, l\) are constants.  (Product rule) \[\frac{d}{dx}[f(x)g(x)] = f'(x)g(x) + f(x)g'(x).\]  (Quotient rule) \(\frac{f(x)}{g(x)}\) is differentiable on \(\{x \mid g(x) \neq 0, x\in I\}\) and \[\frac{d}{dx}\left[\frac{f(x)}{g(x)}\right] = \frac{f'(x)g(x) - f(x)g'(x)}{[g(x)]^2}.\] Proof .  Exercise. Since \(f(x)\) is differentiable at \(x=a\), it is continuous at \(x=a\) so \(f(x) \to f(a)\) as \(x \to a\). For any \(a\in I\), \[\begin{eqnarray*} \frac{f(x)g(x) - f(a)g(a)}{x - a} &=& \frac{f(x)g(x) - f(x)g(a) + f(x)g(a) - f(a)g(a)}{x - a}\\ &=& f(x)\cdot\frac{g(x) ...
Read more