Vector space of functions

We study the collection of functions ${\mathcal{R}}_{2\pi }^2$ (square-integrable functions with period $2\pi$) as a vector space. We define the $L^2$ norm and $L^2$ inner product on this vector space so that we can investigate the ``geometric'' structure of the space of functions. 

Let us show that ${\mathcal{R}}_{2\pi }^2$, the set of square-integrable functions with period $2\pi$, is a vector space over $\mathbb{C}$. First, we need to define addition and scalar multiplication. Let $f,g\in {\mathcal{R}}_{2\pi }^2$. We define $f+g\in {\mathcal{R}}_{2\pi }^2$ by

$$\begin{array}{}\text{(eq:add)}& (f+g)(x)=f(x)+g(x),x\in \mathbb{R}.\end{array}$$

Note that the ``$+$'' on the left-hand side is defined between the two functions $f$ and $g$, whereas the ``$+$'' on the right-hand side is the addition between two complex numbers $f(x)$ and $g(x)$. Next, we define scalar multiplication. Let $\alpha \in \mathbb{C}$ and $f\in {\mathcal{R}}_{2\pi }^2$. We define $\alpha f$ by

$$\begin{array}{}\text{(eq:scale)}& (\alpha f)(x)=\alpha f(x).\end{array}$$

Note that the product on the left-hand side is between the scalar $\alpha$ and the function $f$, whereas the product on the right-hand side is between the two complex numbers $\alpha$ and $f(x)$.

Lemma 

${\mathcal{R}}_{2\pi }^2$ is a vector space with vector addition (eq:add) and scalar multiplication (eq:scale).

Proof. We show that ${\mathcal{R}}_{2\pi }^2$ satisfies all the axioms of the vector space. In the following, $f,g,h\in {\mathcal{R}}_{2\pi }^2$ and $\alpha ,\beta \in \mathbb{C}$.

1. ${\mathcal{R}}_{2\pi }^2$ is closed under vector addition.

Suppose $f,g\in {\mathcal{R}}_{2\pi }^2$. We show $f+g\in {\mathcal{R}}_{2\pi }^2$.

By assumption, ${\int }_{\pi }^{\pi }|f(x){|}^{2}dx<+\mathrm{\infty }$ and ${\int }_{-\pi }^{\pi }|g(x){|}^{2}dx<+\mathrm{\infty }$. Thus,

$$\begin{array}{rcl}{\int }_{-\pi }^{\pi }|f(x)+g(x){|}^{2}dx& \le & {\int }_{-\pi }^{\pi }(|f(x)|+|g(x)|{)}^{2}dx(\text{the triangle inequality})\\ & =& {\int }_{-\pi }^{\pi }|f(x){|}^{2}dx+{\int }_{-\pi }^{\pi }|g(x){|}^{2}dx+2{\int }_{-\pi }^{\pi }|f(x)||g(x)|dx\\ & \le & {\int }_{-\pi }^{\pi }|f(x){|}^{2}dx+{\int }_{-\pi }^{\pi }|g(x){|}^{2}dx+2{\int }_{-\pi }^{\pi }\frac{|f(x){|}^2+|g(x){|}^2}{2}dx\\ & =& 2({\int }_{-\pi }^{\pi }|f(x){|}^{2}dx+{\int }_{-\pi }^{\pi }|g(x){|}^{2}dx)\\ & <& +\mathrm{\infty }(\text{by assumption}).\end{array}$$

2. Addition is commutative.

For any $f,g\in {\mathcal{R}}_{2\pi }^2$,

$$(f+g)(x)=f(x)+g(x)=g(x)+f(x)=(g+f)(x).$$

3. Addition is associative.

For any $f,g,h\in {\mathcal{R}}_{2\pi }^2$,

$$\begin{array}{rcl}((f+g)+h))(x)& =& (f+g)(x)+h(x)\\ & =& (f(x)+g(x))+h(x)\\ & =& f(x)+(g(x)+h(x))\\ & =& f(x)+(g+h)(x)\\ & =& (f+(g+h))(x).\end{array}$$

4. The existence of the zero element (additive identity).

Let $\mathbf{0}(x)=0$ be the identically zero function. Clearly, $\mathbf{0}\in {\mathcal{R}}_{2\pi }^2$, and for any $f\in {\mathcal{R}}_{2\pi }^2$,

$$(\mathbf{0}+f)(x)=\mathbf{0}(x)+f(x)=f(x).$$

5. Additive inverses exist.

For any $f\in {\mathcal{R}}_{2\pi }^2$, define $-f$ by $(-f)(x)=-f(x)$. Clearly, $-f\in {\mathcal{R}}_{2\pi }^2$, and 

$$(f+(-f))(x)=f(x)+(-f(x))=0=\mathbf{0}(x).$$

6. ${\mathcal{R}}_{2\pi }^2$ is closed under scalar multiplication.

For each $\alpha \in \mathbb{C}$ and $f\in {\mathcal{R}}_{2\pi }^2$,

$${\int }_{-\pi }^{\pi }|\alpha f(x){|}^{2}dx=|\alpha {|}^2\Vert f{\Vert }^2<+\mathrm{\infty }.$$

7. Multiplication by the sum of scalars: $(c+d)f=cf+df$.

For each $f\in {\mathcal{R}}_{2\pi }^2$ and $\alpha ,\beta \in \mathbb{C}$,

$$((\alpha +\beta )f)(x)=(\alpha +\beta )f(x)=\alpha f(x)+\beta f(x)=(\alpha f)(x)+(\beta f)(x)=(\alpha f+\beta f)(x).$$

Thus, $(\alpha +\beta )f=\alpha f+\beta f$.

8. Scalar multiplication of the sum of vectors: $\alpha (f+g)=\alpha f+\alpha g$.

For all $f,g\in {\mathcal{R}}_{2\pi }^2$, and $\alpha \in \mathbb{C}$,

$$(\alpha (f+g))(x)=\alpha (f(x)+g(x))=\alpha f(x)+\alpha g(x)=(\alpha f+\alpha g)(x).$$

9. The existence of the multiplicative identity scalar.

For any $f\in {\mathcal{R}}_{2\pi }^2$, $1\in \mathbb{C}$,

$$(1\cdot f)(x)=1\cdot f(x)=f(x).$$

10. Scalar multiplication is associative: $(\alpha \beta )f=\alpha (\beta f)$.

For any $\alpha ,\beta \in \mathbb{C}$, and $f\in {\mathcal{R}}_{2\pi }^2$, 

$$((\alpha \beta )f)(x)=(\alpha \beta )f(x)=\alpha (\beta f(x))=\alpha (\beta f)(x)=(\alpha (\beta f))(x).$$

Thus, ${\mathcal{R}}_{2\pi }^2$ is a vector space over $\mathbb{C}$. ■

Definition ($L^2$-norm, mean-square norm)

Let $f$ be a function on $(-\pi ,\pi )$ that is square-integrable, i.e.,

$${\int }_{-\pi }^{\pi }|f(x){|}^{2}dx<+\mathrm{\infty }.$$

We define the mean-square norm or $L^2$ norm $\Vert f\Vert$ by

$$\Vert f\Vert =\sqrt{{\int }_{-\pi }^{\pi }|f(x){|}^{2}dx}.$$

Remark. The $L$ in $L^2$ stands for ``Lebesgue.'' suggesting that we should use the Lebesgue integral rather than the Riemann integral. But the term ``$L^2$'' is so widespread that we use it, although we only use the Riemann integral. □

See also: Norm (Wikipedia)

Definition (Normed space)

A vector space on which a norm is defined is called a normed vector space or, simply, normed space. 

Definition ($L^2$ inner product)

Let $f,g$ be complex-valued, square-integrable functions on $(-\pi ,\pi )$. We define the $L^2$ inner product between $f$ and $g$ as

$$(f,g)={\int }_{-\pi }^{\pi }f(x)\overline{g(x)}dx$$

where $\overline{g(x)}$ is the complex conjugate of $g(x)$. 

We need to verify that the integral on the right-hand side of the above equation does exist. In fact,

$$\begin{array}{rcl}|{\int }_{-\pi }^{\pi }f(x)\overline{g(x)}dx|& \le & {\int }_{-\pi }^{\pi }|f(x)\overline{g(x)}|dx\\ & =& {\int }_{-\pi }^{\pi }|f(x)|\cdot |g(x)|dx\\ & \le & {\int }_{-\pi }^{\pi }\frac{|f(x){|}^2+|g(x){|}^2}{2}dx\\ & =& \frac{1}{2}{\int }_{-\pi }^{\pi }|f(x){|}^{2}dx+\frac{1}{2}{\int }_{-\pi }^{\pi }|g(x){|}^{2}dx\\ & <& +\mathrm{\infty }\end{array}$$

Remark. The $L^2$ inner product corresponds to the scalar (dot) product in a vector space. □

In general, an inner product is defined as follows.

Definition (Inner product (general))

Let $V$ be a vector space over the field $K$. An inner product $(\cdot ,\cdot ):V\times V\to K$ is a map with the following properties for all vectors $x,y,z\in V$ and all scalars $\alpha ,\beta \in K$:

1. (Conjugate symmetry) $$(x,y)=\overline{(y,x)}.$$
2. (Linearity in the first argument) $$(\alpha x+\beta y,z)=\alpha (x,z)+\beta (y,z)$$
3. (Positive definiteness) If $x\ne 0$, $$(x,x)>0.$$

Example. Consider the vector space ${\mathbb{R}}^n$. For $x=(x_1,x_2,\cdots ,x_n),y=(y_1,y_2,\cdots ,y_n)\in {\mathbb{R}}^n$, the scalar product is defined as

$$(x,y)=\sum _{i=1}^n{x}_i{y}_i.$$

For each $a\in \mathbb{R}$, its ``conjugate'' is the same $a$: $\bar{a}=a$. Thus, 

$$\begin{array}{rcl}\overline{(y,x)}& =& (y,x)\\ & =& \sum _{i=1}^n{y}_i{x}_i\\ & =& \sum _{i=1}^n{x}_i{y}_i\\ & =& (x,y).\end{array}$$

For $a,b\in \mathbb{R}$ and $x,y,z\in {\mathbb{R}}^n$,

$$\begin{array}{rcl}(ax+by,z)& =& \sum _{i=1}^n(a{x}_i+b{y}_i)z_i\\ & =& a\sum _{i=1}^n{x}_i{z}_i+b\sum _{i=1}^n{y}_i{z}_i\\ & =& a(x,z)+b(y,z).\end{array}$$

If $x\in {\mathbb{R}}^n$ is non-zero, at least one of its components, $x_1,x_2,\cdots ,x_n$, is non-zero. Thus,

$$(x,x)=\sum _{i=1}^n{x}_i^2>0.$$

Thus, the scalar product in ${\mathbb{R}}^n$ is an inner product. □

Exercise. Show that $L^2$ inner product satisfies the conditions of an inner product. □

Definition (Inner product space)

A vector space on which an inner product is defined is called an inner product space.

Example. ${\mathcal{R}}_{2\pi }^2$ with the $L^2$ inner product is an inner product space. □

Lemma (Cauchy-Schwarz inequality)

For $f,g\in {\mathcal{R}}_{2\pi }^2$, we have

$$|(f,g)|\le \Vert f\Vert \cdot \Vert g\Vert .$$

Proof. Exercise. ■

Exercise. Using the Cauchy-Schwarz inequality, show the triangle inequality:

$$\Vert f+g\Vert \le \Vert f\Vert +\Vert g\Vert .$$ □

Theorem 

Let $f,f_1,f_2,g,g_1,g_2$ be square-integrable and $\alpha \in \mathbb{C}$ be an arbitrary constant. The following hold:

$$\begin{array}{rcl}\Vert f{\Vert }^2& =& (f,f),\\ (f,g)& =& \overline{(g,f)},\\ (f_1+f_2,g)& =& (f_1,g)+(f_2,g),\\ (f,g_1+g_2)& =& (f,g_1)+(f,g_2),\\ (\alpha f,g)& =& \alpha (f,g),\\ (f,\alpha g)& =& \overline{\alpha }(f,g).\end{array}$$

Proof. Exercise. ■

Using the inner product, the Fourier coefficient of $f$ can be written as

$$c_k=\frac{1}{2\pi }(f,e^{ikx})(k\in \mathbb{Z}).$$

The orthogonality of $\{e^{inx}\}$ can be expressed as

$$(e^{inx},e^{imx})=2\pi {\delta }_{n,m}$$

where ${\delta }_{n,m}$ is Kronecker's delta.

Remark. If the function $f$ has a period of $2\pi$, the range of integration in

$$\Vert f{\Vert }^2={\int }_{-\pi }^{\pi }|f(x){|}^{2}dx$$

can be anything, i.e., ${\int }_a^b|f(x){|}^{2}dx$ as long as $b-a=2\pi$. The same argument applies to the inner product.