Vector space of functions

We study the collection of functions \(\mathcal{R}_{2\pi}^2\) (square-integrable functions with period \(2\pi\)) as a vector space. We define the \(L^2\) norm and \(L^2\) inner product on this vector space so that we can investigate the ``geometric'' structure of the space of functions. 

Let us show that \(\mathcal{R}_{2\pi}^{2}\), the set of square-integrable functions with period \(2\pi\), is a vector space over \(\mathbb{C}\). First, we need to define addition and scalar multiplication. Let \(f, g\in \mathcal{R}_{2\pi}^2\). We define \(f + g \in \mathcal{R}_{2\pi}^2\) by

\[(f+g)(x) = f(x) + g(x), ~ x \in \mathbb{R}.\tag{eq:add}\]

Note that the ``\(+\)'' on the left-hand side is defined between the two functions \(f\) and \(g\), whereas the ``\(+\)'' on the right-hand side is the addition between two complex numbers \(f(x)\) and \(g(x)\). Next, we define scalar multiplication. Let \(\alpha\in \mathbb{C}\) and \(f \in \mathcal{R}_{2\pi}^2\). We define \(\alpha f\) by

\[(\alpha f)(x) = \alpha f(x).\tag{eq:scale}\]

Note that the product on the left-hand side is between the scalar \(\alpha\) and the function \(f\), whereas the product on the right-hand side is between the two complex numbers \(\alpha\) and \(f(x)\).

Lemma 

\(\mathcal{R}_{2\pi}^2\) is a vector space with vector addition (eq:add) and scalar multiplication (eq:scale).

Proof. We show that \(\mathcal{R}_{2\pi}^2\) satisfies all the axioms of the vector space. In the following, \(f, g, h\in\mathcal{R}_{2\pi}^2\) and \(\alpha, \beta \in \mathbb{C}\).

1. \(\mathcal{R}_{2\pi}^2\) is closed under vector addition.

Suppose \(f, g \in \mathcal{R}_{2\pi}^2\). We show \(f + g\in\mathcal{R}_{2\pi}^2\).

By assumption, \(\int_{\pi}^{\pi}|f(x)|^2\,dx < +\infty\) and \(\int_{-\pi}^{\pi}|g(x)|^2\,dx < +\infty\). Thus,

\[\begin{eqnarray} \int_{-\pi}^{\pi}|f(x)+g(x)|^2\,dx &\leq&\int_{-\pi}^{\pi}(|f(x)| + |g(x)|)^2\,dx ~ (\text{the triangle inequality})\\ & = & \int_{-\pi}^{\pi}|f(x)|^2\,dx + \int_{-\pi}^{\pi}|g(x)|^2\,dx + 2\int_{-\pi}^{\pi}|f(x)||g(x)|\,dx\\ &\leq&\int_{-\pi}^{\pi}|f(x)|^2\,dx + \int_{-\pi}^{\pi}|g(x)|^2\,dx + 2\int_{-\pi}^{\pi}\frac{|f(x)|^2 + |g(x)|^2}{2}\,dx\\ &=& 2\left(\int_{-\pi}^{\pi}|f(x)|^2\,dx + \int_{-\pi}^{\pi}|g(x)|^2\,dx \right)\\ & < & +\infty ~ (\text{by assumption}). \end{eqnarray}\]

2. Addition is commutative.

For any \(f, g\in \mathcal{R}_{2\pi}^2\),

\[(f+g)(x) = f(x) + g(x) = g(x) + f(x) = (g+f)(x).\]

3. Addition is associative.

For any \(f, g, h \in \mathcal{R}_{2\pi}^2\),

\[\begin{eqnarray}((f+g)+h))(x) &=& (f+g)(x) + h(x)\\ &=& (f(x) + g(x)) + h(x)\\ &=& f(x) + (g(x)+h(x))\\ &=& f(x) + (g+h)(x)\\ & =& (f+(g+h))(x). \end{eqnarray}\]

4. The existence of the zero element (additive identity).

Let \(\mathbf{0}(x) = 0\) be the identically zero function. Clearly, \(\mathbf{0}\in \mathcal{R}_{2\pi}^2\), and for any \(f\in\mathcal{R}_{2\pi}^2\),

\[(\mathbf{0} + f)(x) = \mathbf{0}(x) + f(x) = f(x).\]

5. Additive inverses exist.

For any \(f\in\mathcal{R}_{2\pi}^{2}\), define \(-f\) by \((-f)(x) = -f(x)\). Clearly, \(-f\in\mathcal{R}_{2\pi}^2\), and 

\[(f + (-f))(x) = f(x) + (-f(x)) = 0 = \mathbf{0}(x).\]

6. \(\mathcal{R}_{2\pi}^2\) is closed under scalar multiplication.

For each \(\alpha \in\mathbb{C}\) and \(f \in\mathcal{R}_{2\pi}^2\),

\[\int_{-\pi}^{\pi}|\alpha f(x)|^2\,dx=|\alpha|^2\|f\|^2 < +\infty.\]

7. Multiplication by the sum of scalars: \((c+d)f = cf + df\).

For each \(f\in \mathcal{R}_{2\pi}^2\) and \(\alpha, \beta \in \mathbb{C}\),

\[((\alpha+\beta)f)(x) = (\alpha+\beta)f(x) = \alpha f(x) + \beta f(x) = (\alpha f)(x) + (\beta f)(x) = (\alpha f + \beta f)(x).\]

Thus, \((\alpha +\beta )f = \alpha f + \beta f\).

8. Scalar multiplication of the sum of vectors: \(\alpha (f + g) = \alpha f + \alpha g\).

For all \(f, g\in\mathcal{R}_{2\pi}^2\), and \(\alpha \in\mathbb{C}\),

\[(\alpha (f + g))(x) = \alpha (f(x) + g(x)) = \alpha f(x) + \alpha g(x) = (\alpha f+\alpha g)(x).\]

9. The existence of the multiplicative identity scalar.

For any \(f\in\mathcal{R}_{2\pi}^{2}\), \(1 \in \mathbb{C}\),

\[(1\cdot f)(x) = 1\cdot f(x) = f(x).\]

10. Scalar multiplication is associative: \((\alpha\beta )f = \alpha(\beta f)\).

For any \(\alpha, \beta\in \mathbb{C}\), and \(f \in \mathcal{R}_{2\pi}^2\), 

\[((\alpha\beta)f)(x) = (\alpha\beta)f(x) = \alpha(\beta f(x)) = \alpha(\beta f)(x) = (\alpha (\beta f))(x).\]

Thus, \(\mathcal{R}_{2\pi}^{2}\) is a vector space over \(\mathbb{C}\). ■

Definition (\(L^2\)-norm, mean-square norm)

Let \(f\) be a function on \((-\pi, \pi)\) that is square-integrable, i.e.,

\[\int_{-\pi}^{\pi}|f(x)|^2dx < +\infty.\]

We define the mean-square norm or \(L^2\) norm \(\|f\|\) by

\[\|f\| = \sqrt{\int_{-\pi}^{\pi}|f(x)|^2dx}.\]

Remark. The \(L\) in \(L^2\) stands for ``Lebesgue.'' suggesting that we should use the Lebesgue integral rather than the Riemann integral. But the term ``\(L^2\)'' is so widespread that we use it, although we only use the Riemann integral. □

See also: Norm (Wikipedia)

Definition (Normed space)

A vector space on which a norm is defined is called a normed vector space or, simply, normed space. 

Definition (\(L^2\) inner product)

Let \(f, g\) be complex-valued, square-integrable functions on \((-\pi, \pi)\). We define the \(L^2\) inner product between \(f\) and \(g\) as

\[(f, g) = \int_{-\pi}^{\pi}f(x)\overline{g(x)}\,dx\]

where \(\overline{g(x)}\) is the complex conjugate of \(g(x)\). 

We need to verify that the integral on the right-hand side of the above equation does exist. In fact,

\[\begin{eqnarray*} \left|\int_{-\pi}^{\pi}f(x)\overline{g(x)}dx\right| &\leq& \int_{-\pi}^{\pi}|f(x)\overline{g(x)}|dx\\ &=& \int_{-\pi}^{\pi}|f(x)|\cdot |g(x)|dx \\ &\leq&\int_{-\pi}^{\pi}\frac{|f(x)|^2 + |g(x)|^2}{2}dx \\ &=& \frac{1}{2}\int_{-\pi}^{\pi}|f(x)|^2dx + \frac{1}{2}\int_{-\pi}^{\pi}|g(x)|^2dx\\ &<& +\infty \end{eqnarray*}\]

Remark. The \(L^2\) inner product corresponds to the scalar (dot) product in a vector space. □

In general, an inner product is defined as follows.

Definition (Inner product (general))

Let \(V\) be a vector space over the field \(K\). An inner product \((\cdot, \cdot): V\times V \to K\) is a map with the following properties for all vectors \(x,y,z\in V\) and all scalars \(\alpha, \beta \in K\):

1. (Conjugate symmetry) \[(x,y) = \overline{(y,x)}.\]
2. (Linearity in the first argument) \[(\alpha x + \beta y, z) = \alpha(x,z) + \beta(y,z)\]
3. (Positive definiteness) If \(x \neq 0\), \[(x,x) > 0.\]

Example. Consider the vector space \(\mathbb{R}^n\). For \(x = (x_1, x_2, \cdots, x_n), y = (y_1, y_2, \cdots, y_n) \in \mathbb{R}^n\), the scalar product is defined as

\[(x,y) = \sum_{i=1}^{n}x_iy_i.\]

For each \(a \in \mathbb{R}\), its ``conjugate'' is the same \(a\): \(\overline{a} = a\). Thus, 

\[\begin{eqnarray}\overline{(y,x)} &=& (y,x)\\ &=& \sum_{i=1}^{n}y_ix_i\\ &=& \sum_{i=1}^{n}x_iy_i\\ &=& (x,y).\end{eqnarray}\]

For \(a,b\in\mathbb{R}\) and \(x,y,z \in \mathbb{R}^n\),

\[\begin{eqnarray}(ax+by,z) &=& \sum_{i=1}^{n}(ax_i+by_i)z_i\\ &=& a\sum_{i=1}^{n}x_iz_i + b\sum_{i=1}^{n}y_iz_i\\ &=& a(x,z) + b(y,z). \end{eqnarray}\]

If \(x \in \mathbb{R}^n\) is non-zero, at least one of its components, \(x_1, x_2, \cdots, x_n\), is non-zero. Thus,

\[(x,x) = \sum_{i=1}^{n}x_i^2 > 0.\]

Thus, the scalar product in \(\mathbb{R}^n\) is an inner product. □

Exercise. Show that \(L^2\) inner product satisfies the conditions of an inner product. □

Definition (Inner product space)

A vector space on which an inner product is defined is called an inner product space.

Example. \(\mathcal{R}_{2\pi}^2\) with the \(L^2\) inner product is an inner product space. □

Lemma (Cauchy-Schwarz inequality)

For \(f, g \in \mathcal{R}_{2\pi}^2\), we have

\[|(f, g)| \leq \|f\|\cdot \|g\|.\]

Proof. Exercise. ■

Exercise. Using the Cauchy-Schwarz inequality, show the triangle inequality:

\[\|f+g\| \leq \|f\| + \|g\|.\] □

Theorem 

Let \(f, f_1, f_2, g, g_1, g_2\) be square-integrable and \(\alpha\in \mathbb{C}\) be an arbitrary constant. The following hold:

\[\begin{eqnarray} \|f\|^2 &=& (f, f),\\ (f, g) &=& \overline{(g, f)},\\ (f_1 + f_2, g) &=& (f_1, g) + (f_2, g),\\ (f, g_1 + g_2) &=& (f, g_1) + (f, g_2),\\ (\alpha f, g) &=& \alpha(f, g),\\ (f, \alpha g) &=& \bar{\alpha}(f,g). \end{eqnarray}\]

Proof. Exercise. ■

Using the inner product, the Fourier coefficient of \(f\) can be written as

\[c_k = \frac{1}{2\pi}(f, e^{ikx}) ~~~ (k \in \mathbb{Z}).\]

The orthogonality of \(\{e^{inx}\}\) can be expressed as

\[(e^{inx}, e^{imx}) = 2\pi\delta_{n,m}\]

where \(\delta_{n,m}\) is Kronecker's delta.

Remark. If the function \(f\) has a period of \(2\pi\), the range of integration in

\[\|f\|^2 = \int_{-\pi}^{\pi}|f(x)|^2\,dx\]

can be anything, i.e., \(\int_a^{b}|f(x)|^2dx\) as long as \(b - a = 2\pi\). The same argument applies to the inner product.