Euclidean spaces

-

We would like to study multivariate functions (i.e., functions of many variables), continuous multivariate functions in particular. To define continuity, we need a measure of "closeness" between points. One measure of closeness is the Euclidean distance. The set \(\mathbb{R}^n\) (with \(n \in \mathbb{N}\)) with the Euclidean distance function is called a Euclidean space. This is the space where our functions of interest live.

The real line is a geometric representation of \(\mathbb{R}\), the set of all real numbers. That is, each \(a \in \mathbb{R}\) is represented as the point \(a\) on the real line.

The coordinate plane, or the \(x\)-\(y\) plane, is a geometric representation of \(\mathbb{R}^2\), the set of all pairs of real numbers. Each pair of real numbers \((a, b)\) is visualized as the point \((a, b)\) in the plane.

Remark. Recall that \(\mathbb{R}^2 = \mathbb{R}\times\mathbb{R} = \{(x, y) | x, y \in \mathbb{R}\}\) is the Cartesian product of \(\mathbb{R}\) with itself, its elements such as \((x, y)\) are ordered pairs of real numbers.

The coordinate space, or the \(x\)-\(y\)-\(z\) space, is a geometric representation of \(\mathbb{R}^3\), the set of all triples of real numbers.

Remark. Recall that \(\mathbb{R}^3 = \mathbb{R}\times\mathbb{R}\times\mathbb{R} = \{(x, y, z) | x, y, z \in \mathbb{R}\}\) is also a Cartesian product, its elements such as \((x, y, z)\) are ordered triples of real numbers.

We can naturally extend this idea. For any \(n\in\mathbb{N}\), we can consider an \(n\)-tuple of real numbers \((a_1, a_2, \cdots, a_n)\) and the set \(\mathbb{R}^n\) of all such \(n\)-tuples. We can "visualize" each element of \(\mathbb{R}^n\) as a "point"' in the \(n\)-dimensional space. For example, \((a_1, a_2, \cdots, a_n) \in \mathbb{R}^n\) is a point where the \(x_1\)-coordinate is \(a_1\), \(x_2\)-coordinate is \(a_2\), and so on.

Univariate functions (i.e., functions with one variable) are often defined on an interval. We would like to extend the notion of an interval to \(\mathbb{R}^n\). But first, we need the notion of distance.

Definition (Euclidean distance)

  Let \(x = (x_1, x_2, \cdots, x_n)\) and \(y = (y_1, y_2, \cdots, y_n)\) be points in \(\mathbb{R}^n\). The (Euclidean) distance \(d(x,y)\) between \(x\) and \(y\) is defined as \[d(x,y) = \sqrt{\sum_{i=1}^{n}(x_i - y_i)^2}.\tag{Eq:Distance}\]

Remark. The distance \(d(x,y)\) is also denoted as \(\|x - y\|\) or \(\|x - y \|_2\). Recall the definition of the length of an \(n\)-dimensional vector.

Definition (Euclidean space)

The set \(\mathbb{R}^n\) equipped with the distance defined in (Eq:Distance) is called the \(n\)-dimensional Euclidean space.
Remark. Sometimes, we say the pair \((\mathbb{R}^n, d)\), where \(d\) is the distance function, is the Euclidean space.
Remark. In mathematics, we generally use the term space to mean a set with some "structure." In the case of Euclidean space, the "structure" is specified by the distance. Other examples of spaces include vector space, probability space, topological space, Hilbert space, etc.

You might have learned the following lemma in Linear Algebra:

Lemma (Cauchy-Schwarz inequality)

For \(a = (a_1, a_2, \cdots, a_n), b = (b_1, b_2, \cdots, b_n) \in \mathbb{R}^n\), we have
\[\left|\sum_{i=1}^{n}a_ib_i\right| \leq \sqrt{\sum_{i=1}^{n}a_i^2}\sqrt{\sum_{i=1}^{n}b_i^2}.\tag{Eq:CauchySchwarzIneq}\]
Proof. The result is trivial if \(a = (0, 0, \cdots, 0)\). Suppose \(a \neq (0, 0, \cdots, 0)\). For any \(t \in \mathbb{R}\), 
\[\begin{eqnarray} 0 &\leq & \sum_{i=1}^{n}(a_it+b_i)^2\\ &=& \left(\sum_{i=1}^{n}a_i^2\right)t^2 + 2\left(\sum_{i=1}^{n}a_ib_i\right)t + \left(\sum_{i=1}^{n}b_i^2\right).   \end{eqnarray}\]
The last quadratic form of \(t\) has at most one real root because it is non-negative;  hence its discriminant is non-positive:
\[\left(\sum_{i=1}^{n}a_ib_i\right)^2 - \left(\sum_{i=1}^{n}a_i^2\right)\left(\sum_{i=1}^{n}b_i^2\right) \leq 0,\]
from which we conclude that
\[\left(\sum_{i=1}^{n}a_ib_i\right)^2 \leq \left(\sum_{i=1}^{n}a_i^2\right)\left(\sum_{i=1}^{n}b_i^2\right).\]
Taking the square root of both sides, we have (Eq:CauchySchwarzIneq). ■
Remark: If we regard \(a, b\in\mathbb{R}^n\) as vectors in a vector space with the scalar product (i.e., dot product) \(\langle a, b\rangle\) and induced norm \(\|\cdot\|\), the Cauchy-Schwarz inequality reads:
\[|\langle a, b\rangle| \leq \|a\|\|b\|.\]

Theorem (Distance axioms)

  1. (Non-negativity) For any \(x, y \in \mathbb{R}^n\), \(d(x,y) \geq 0\). In particular, \(d(x,y) = 0\) if and only if \(x = y\).
  2. (Symmetry) For any \(x, y\in\mathbb{R}^n\), \(d(x,y) = d(y,x)\).
  3. (Triangle inequality) For \(x,y,z\in \mathbb{R}^n\), \[d(x,z) \leq d(x,y) + d(y,z).\]
Proof. (1) and (2) are trivial. We show (3) only.
Let \(x = (x_1, x_2, \cdots, x_n), y = (y_1, y_2, \cdots, y_n), z=(z_1, z_2,\cdots,z_n)\), and \(a_i = x_i - y_i, b_i = y_i - z_i\). Note that \(x_i - z_i = (x_i - y_i) + (y_i - z_i) = a_i + b_i\). Then, we need to prove
\[\sqrt{\sum_{i=1}^{n}(a_i + b_i)^2} \leq \sqrt{\sum_{i=1}^{n}a_i^2} + \sqrt{\sum_{i=1}^{n}b_i^2}.\] 
Noting that both sides are non-negative, squaring both sides gives
\[\sum_{i=1}^{n}(a_i + b_i)^2 \leq \sum_{i=1}^{n}a_i^2 + \sum_{i=1}^{n}b_i^2 + 2\sqrt{\sum_{i=1}^{n}a_i^2}\sqrt{\sum_{i=1}^{n}b_i^2}.\tag{Eq:Ineq1}\]
The left-hand side is
\[\sum_{i=1}^{n}(a_i + b_i)^2 = \sum_{i=1}^{n}a_i^2 + \sum_{i=1}^{n}b_i^2 + 2\sum_{i=1}^{n}a_ib_i.\] Canceling common terms, the above inequality (Eq:Ineq1) to be proved becomes
\[\sum_{i=1}^{n}a_ib_i \leq \sqrt{\sum_{i=1}^{n}a_i^2}\sqrt{\sum_{i=1}^{n}b_i^2}.\]
But this is trivial from the Cauchy-Schwarz inequality (Eq:CauchySchwarzIneq) in the above Lemma. Now, trace this argument backward, and the triangle inequality follows from the Cauchy-Schwarz inequality. ■

Remark. For any set \(S\), if a function \(d: S\times S \to \mathbb{R}\) satisfies the above properties of distance, then this function \(d\) may be considered as a distance function in \(S\). The above properties can be used as axioms to define a distance in any set (if possible). Generally, a set \(S\) with a distance function \(d\) is called a metric space.

Example. For \(x, y \in \mathbb{R}^n\), let us define the following function:
\[d_1(x,y) = \sum_{i=1}^{n}|x_i - y_i|.\] This function satisfies all the distance axioms. Thus, \((\mathbb{R}^n, d_1)\) is a metric space. The function \(d_1\) is sometimes called the L1 distance. In comparison, the Euclidean distance is also called the L2 distance.

Definition (\(\varepsilon\)-neighbor)

For \(x \in \mathbb{R}^n\) and \(\varepsilon > 0\), the \(\varepsilon\)-neighbor of \(x\) is defined as
\[N_{\varepsilon}(x) = \{y \in \mathbb{R}^n\mid d(x, y) < \varepsilon\}.\]
\(N_{\varepsilon}(x)\) is also called the open ball with radius \(\varepsilon\) centered at \(x\).

Example
  • In \(\mathbb{R}\), \(N_{\varepsilon}(x) = (x - \varepsilon, x + \varepsilon)\) is an open interval.
  • In \(\mathbb{R}^2\), let \(a = (a_1, a_2)\). Then, \(N_{\varepsilon}(a) = \{(x_1,x_2) \in \mathbb{R}^2 | (x_1 - a_1)^2 + (x_2 - a_2)^2 < \varepsilon^2\}\) is the interior of the circle with radius \(\varepsilon\) centered at \(a\).
  • In \(\mathbb{R}^3\), an \(\varepsilon\)-neighbor is the interior of a sphere.


Comments

Post a Comment

Popular posts from this blog

Birth process

-
Image
  Defining the birth process  Consider a colony of bacteria that never dies. We study the following process known as  the birth process , also known as  the Yule process . The colony starts with \(n_0\) cells at time \(t = 0\). Assume that the probability that any individual cell divides in the time interval \((t, t + \delta t)\) is proportional to \(\delta t\) for small \(\delta t\). Further assume that each cell division is independent of others. Let \(\lambda\) be the  birth rate.  The probability of a cell division for a population of \(n\) cells during \(\delta t\) is \(\lambda n \delta t\). We assume that the probability that two or more births take place in the time interval \(\delta t\) is \(o(\delta t)\). That is, it can be ignored. Consequently, the probability that no cell divides during \(\delta t\) is \(1 - \lambda n \delta t - o(\delta t)\). Note that this process is an example of the Markov chain with states \({n_0}, {n_0 + 1}, {n_0 + 2}...
Read more

Fourier series and the heat equation

-
Image
Joseph Fourier introduced the Fourier series to solve the heat equation in the 1810s. In this post, we show how the Fourier transform arises naturally in a simplified version of the heat equation. Suppose we have the unit circle \(S\) made of a metal wire. Pick an arbitrary point \(A\) on the circle. Any point \(P\) on the circle is identified by the distance \(x\) from \(A\) to \(P\) along the circle in the counter-clockwise direction (i.e., \(x\) is the angle of the section between \(A\) and \(P\) in radian). Let \(u(t,x)\) represent the temperature at position \(x\) and time \(t\). The temperature distribution at \(t = 0\) is given by \(u(0, x) = f(x)\). Assuming no radiation of heat out of the metal wire, \(u(t,x)\) for \(t > 0\) and \(0\leq x \leq 2\pi\) is determined by the following partial differential equation  (PDE) called the  heat equation : \[\gamma\frac{\partial u}{\partial t} = \kappa\frac{\partial^2 u}{\partial x^2}\] and the initial condition \[u(0,x) = f(x...
Read more

Series: Introduction

-
Image
Given a sequence \(\{a_n\}\), the expression \[\sum_{n=0}^{\infty}a_n = a_0 + a_1 + a_2 + \cdots\] is called a series (or infinite series ). This expression may or may not have value. At this point, it is purely formal. Note that the order of addition matters : We first add \(a_0\) and \(a_1\), to the result of which we add \(a_2\), to the result of which we add \(a_3\), and so on (Not something like we first add \(a_{101}\) and \(a_{58}\), then add \(a_{333051}\), and so on). We will see, however, that for a special class of series (the positive term series), the order of addition does not matter if the series converges. Example . The sum of a geometric progression \(\{ar^n\}\), that is, \(\sum_{n=0}^{\infty}ar^n\) is called a geometric series . It is understood that \(r^0 = 1\) including the case when \(r = 0\). □ Given a series \(\sum_{n=0}^{\infty}a_n\) and a number \(n\geq 0\), the sum \[\sum_{k=0}^{n}a_k = a_0 + a_1 + \cdots + a_n\] is called the \(n\)-th partial sum . We m...
Read more