Matrix operations

We introduce various operations on matrices. A matrix can be multiplied by a scalar. Two matrices can be added or multiplied. A matrix can be transposed. Writing down these operations explicitly can be tedious, so we introduce a systematic notation for representing matrices and their elements.

We want to denote each entry of a matrix systematically.

Suppose we have a $2 \times 3$ matrix

$M = (\begin{matrix} 1 & 1 & 2 \\ - 1 & 7 & 2 \end{matrix}) .$

This may be written as

$M = (\begin{matrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \end{matrix})$

where $m_{11} = 1,$ $m_{21} = - 1$ , etc. But if we have a $100 \times 100$ matrix, this way of writing is very tedious, to say the least. In mathematics, it is crucial to be ``appropriately lazy''. So here's what we usually do:

$M = (m_{i j})$

where it is understood that the indices $i$ and $j$ run through some appropriate ranges (in this particular example, $i = 1, 2$ and $j = 1, 2, 3$ .) This is a very tidy way. Furthermore, when $M$ is a matrix, we also write $(M)_{i j}$ to indicate its $(i, j)$ -element ( $i$ -th row, $j$ -th column). Thus, if we have $M = (m_{i j})$ , we have $(M)_{i j} = m_{i j}$ .

Definition (Scalar-matrix multiplication)

Let $A = (a_{i j})$ be an $n \times m$ matrix, and $λ \in R$ be a constant. Then the scalar multiplication of $A$ by $λ$ is defined by

$λ A = (λ a_{i j}) .$

That is,

$λ A = (\begin{matrix} λ a_{11} & λ a_{12} & \dots & λ a_{1 m} \\ λ a_{21} & λ_{22} & \dots & λ a_{2 m} \\ ⋮ & ⋱ & ⋮ \\ λ a_{n 1} & λ a_{n 2} & \dots & λ a_{n m} \end{matrix}) .$

Another way to write this is

$(λ A)_{i j} = λ a_{i j} .$

Example.

$3 (\begin{matrix} 1 & 1 & 2 \\ - 1 & 7 & 2 \end{matrix}) = (\begin{matrix} 3 & 3 & 6 \\ - 3 & 21 & 6 \end{matrix}) .$ □

Definition (Matrix addition)

Let $A = (a_{i j})$ and $B = (b_{i j})$ be both $n \times m$ matrices. We define a new $n \times m$ matrix $A + B$ by

$A + B = (a_{i j} + b_{i j}) .$

Another way to write this is

$(A + B)_{i j} = (A)_{i j} + (B)_{i j} .$

Remark. To add two matrices, their sizes must be exactly equal. □

Example. $(\begin{matrix} 1 & 1 & 2 \\ - 1 & 7 & 2 \end{matrix}) + (\begin{matrix} 3 & 2 & 1 \\ 4 & 5 & 6 \end{matrix}) = (\begin{matrix} 4 & 3 & 3 \\ 3 & 12 & 8 \end{matrix}) .$ □

Definition (Matrix multiplication)

Let $A = (a_{i j})$ be an $n \times m$ matrix and $B = (b_{i j})$ be an $m \times p$ matrix. We define a new $n \times p$ matrix $A B$ by

$A B = (\sum_{k = 1}^{m} a_{i k} b_{k j})$

Another way to write this is

$(A B)_{i j} = \sum_{k = 1}^{m} (A)_{i k} (B)_{k j} .$

Remark.

To multiply two matrices, the number of columns of the first matrix and the number of rows of the second matrix must be equal.
Even if $A B$ is defined, $B A$ may not be defined because the sizes may not match.
In general, matrix multiplication is not commutative. That is, $A B = B A$ does not hold in general.

□

Example. $(\begin{matrix} 1 & 1 & 2 \\ - 1 & 7 & 2 \end{matrix}) (\begin{matrix} 1 & 0 \\ 0 & 1 \\ 2 & 3 \end{matrix}) = (\begin{matrix} 5 & 7 \\ 3 & 13 \end{matrix}) .$

□

Example.

$(\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}) (\begin{matrix} - 2 & 3 \\ - 1 & 4 \end{matrix}) = (\begin{matrix} - 4 & 11 \\ - 10 & 25 \end{matrix}) .$

$(\begin{matrix} - 2 & 3 \\ - 1 & 4 \end{matrix}) (\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}) = (\begin{matrix} 7 & 8 \\ 11 & 14 \end{matrix}) .$

Thus,

$(\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}) (\begin{matrix} - 2 & 3 \\ - 1 & 4 \end{matrix}) \neq (\begin{matrix} - 2 & 3 \\ - 1 & 4 \end{matrix}) (\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}) .$

□

Definition (Transpose)

Let $A = (a_{i j})$ be an $n \times m$ matrix. The transpose of $A$ , denoted $A^{T}$ is the $m \times n$ matrix whose entry in the $i$ -th row and $j$ -th column is $a_{j i}$ . In other words,

$(A^{T})_{i j} = (A)_{j i} .$

``The $(i, j)$ -element of $A^{T}$ is the $(j, i)$ -element of $A$ .''

Example. ${(\begin{matrix} 1 & 1 & 2 \\ - 1 & 7 & 2 \end{matrix})}^{T} = (\begin{matrix} 1 & - 1 \\ 1 & 7 \\ 2 & 2 \end{matrix}) .$ □

Lemma

Let $A$ be a matrix. Then,

$(A^{T})^{T} = A .$

That is, the transpose of the transpose of $A$ is $A$ itself.

Proof.

$((A^{T})^{T})_{i j} = (A^{T})_{j i} = (A)_{i j} .$ ■

$n$ -dimensional row vectors $a = (a_{1}, \dots, a_{n})$ and $b = (b_{1}, \dots, b_{n})$ may be regarded as $1 \times n$ matrices. Consequently, the transpose $b^{T}$ , a column vector, may be regarded as an $n \times 1$ matrix. We can matrix-multiply $a$ and $b^{T}$ to have a $1 \times 1$ ``matrix'', that is,

$a b^{T} = (a_{1} b_{1} + a_{2} b_{2} + \dots + a_{n} b_{n}) .$

If we identify $1 \times 1$ matrices with scalars, this result is nothing but the scalar product. Thus we have

$a b^{T} = ⟨ a, b ⟩ .$

This observation can be generalized as follows. Suppose the rows of the $n \times m$ matrix $A$ are the $m$ -dimensional row vectors $a_{1}, a_{2}, \dots, a_{n}$ , that is,

$A = (a_{i j}) = (\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n} \end{matrix}) .$

Similarly, suppose the columns of the $m \times p$ matrix $B$ are the $m$ -dimensional column vectors $b_{1}^{T}, b_{2}^{T}, \dots, b_{p}^{T}$ , that is,

$B = (b_{i j}) = (\begin{matrix} b_{1}^{T} & b_{2}^{T} & \dots & b_{p}^{T} \end{matrix}) .$

Since $b_{i}^{T}$ is a column vector, its transpose $b_{i}$ is a row vector. We have

$\begin{array}{rcl} A B & = & (\begin{array}{c} a_{1} \\ a_{2} \\ ⋮ \\ a_{n} \end{array}) (\begin{array}{c} b_{1}^{T} & b_{2}^{T} & \dots & b_{p}^{T} \end{array}) \\ = & (\begin{array}{c} ⟨ a_{1}, b_{1} ⟩ & ⟨ a_{1}, b_{2} ⟩ & \dots & ⟨ a_{1}, b_{p} ⟩ \\ ⟨ a_{2}, b_{1} ⟩ & ⟨ a_{2}, b_{2} ⟩ & \dots & ⟨ a_{2}, b_{p} ⟩ \\ ⋮ & ⋱ & ⋮ \\ ⟨ a_{n}, b_{1} ⟩ & ⟨ a_{n}, b_{2} ⟩ & \dots & ⟨ a_{n}, b_{p} ⟩ \end{array}) \\ = & (⟨ a_{i}, b_{j} ⟩) . \end{array}$

Thus, every element of a matrix product $A B$ is a scalar product between a row vector and a column vector.

Search This Blog

Brunei Math Club