Extreme values of multivariate functions

A local maximum (or minimum) value of a function is the maximum (or minimum) value of the function in the neighbor of a point. More formally,

Definition (Local maximum, local minimum)

Let $f(x,y)$ be a function on an open region $U\subset {\mathbb{R}}^2$ and $P=(a,b)\in U$.

1. $f(a,b)$ is said to be a local maximum value of the function $f(x,y)$ if there exists $\delta >0$ such that, for all $(x,y)\in U$, if $(x,y)\in N_{\delta }(P)\cap U$ and $(x,y)\ne (a,b)$, then $f(x,y)<f(a,b)$.
2. $f(a,b)$ is said to be a local minimum value of the function $f(x,y)$ if there exists $\delta >0$ such that, for all $(x,y)\in U$, if $(x,y)\in N_{\delta }(P)\cap U$ and $(x,y)\ne (a,b)$, then $f(x,y)>f(a,b)$.

Local maximum and minimum values are collectively called extreme values.

Theorem (Necessary condition for extreme values)

Let $f(x,y)$ be a function on an open region $U$, and $(a,b)\in U$. Suppose that $f_x(a,b)$ and $f_y(a,b)$ exist. If $f(a,b)$ is an extreme value, then $f_x(a,b)=f_y(a,b)=0$.

Proof. If $f(x,y)$ has an extreme value at $(a,b)$, then the univariate function $f(x,b)$ of $x$ has an extreme value at $x=a$. By the corresponding theorem for univariate functions, $f_x(a,b)=0$. Similarly, $f_y(a,b)=0$. ■

The converse of this theorem is not necessarily true.

Example. Consider $f(x,y)=x^2-y^2$ (See Fig:NoExtreme below). We can see that $f_x(0,0)=f_y(0,0)=0$. However, the graph of $f(x,0)=x^2$ is convex whereas that of $f(0,y)=-y^2$ is concave. This means that, if we move from the origin along the $x$-axis, the value of $f(x,y)$ increases, but if we move from the origin along the $y$-axis, the value decreases. Therefore, $f(0,0)$a  is neither a a local maximum nor a local minimum. □

Figrue Fig:NoExtreme. The graph of $z=x^2-y^2$. This function has neither a local maximum nor a local minimum at $(x,y)=(0,0)$. (See the above example.)

In the case of univariate functions, we can determine whether a function takes a local maximum or local minimum value at a given point by the sign of the second differential coefficient at that point. We have a corresponding theorem for the case of bivariate functions, but the situation is a bit more complicated. We remark the following theorem without proof.

Theorem (Criteria of extreme values)

Let $f(x,y)$ be a function of class $C^2$ on an open region $U$. Suppose that $(a,b)\in U$ and $f_x(a,b)=f_y(a,b)=0$. Let us define the determinant $D$ by
$$D=f_{xx}(a,b)f_{yy}(a,b)-[f_{xy}(a,b){]}^2.$$

1. If $D>0$,

1. if $f_{xx}(a,b)>0$, then $f(x,y)$ has a local minimum value at $(x,y)=(a,b)$;
2. if $f_{xx}(a,b)<0$, then $f(x,y)$ has a local maximum value at $(x,y)=(a,b)$.

1. If $D<0$, $f(x,y)$ does not have a local extremum at $(x,y)=(a,b)$.

Proof. Omitted (beyond the scope of this module). ■

Remark. Note that the "determinant" $D$ in the above theorem is indeed the determinant of a matrix:$$D=\left|\begin{array}{cc}{f}_{xx}(a,b)& f_{xy}(a,b)\\ f_{yx}(a,b)& f_{yy}(a,b)\end{array}\right|.$$ This matrix is called the Hessian of the function. You may guess the Hessian of a general multivariate function $f(x_1,x_2,\cdots ,x_n)$. □

Remark. This theorem is only applicable to bivariate (two-variable) functions. For general multivariate functions, such as $f(x_1,x_2,\cdots ,x_n)$, the corresponding theorem is even more complicated (but it exists). That is, if all the eigenvalues of the Hessian are positive (or negative) at a given point, then the function takes a local minimum (or maximum) value at the point. (Note that the Hessian is always symmetric if the function is of class $C^2$. Therefore, all the eigenvalues are guaranteed to be real.) □

Example. Let us find the extreme values of the function $f(x,y)=x^3+y^3-3(x+y)$ on ${\mathbb{R}}^2$ (See the figure below).

The graph of $z=x^3+y^3-3(x+y)$.

Solving 

$$\begin{array}{rcl}{f}_x(x,y)& =& 3(x^2-1)=0,\\ f_y(x,y)& =& 3(y^2-1)=0,\end{array}$$

we have

$$(x,y)=(1,1),(-1,-1),(-1,1),(1,-1).$$

From

$$\begin{array}{rcl}{f}_{xx}(x,y)& =& 6x,\\ f_{xy}(x,y)& =& 0,\\ f_{yy}(x,y)& =& 6y,\end{array}$$

we have 

$$D=f_{xx}(x,y)f_{yy}(x,y)-[f_{xy}(x,y){]}^2=36xy.$$

• When $(x,y)=(1,1)$, we have $D=36>0$ and $f_{xx}(1,1)=6>0$ so that $f(1,1)=-4$ is a local minimum.
• When $(x,y)=(-1,-1)$, we have $D=36>0$ and $f_{xx}(-1,-1)=-6<0$ so that $f(-1,-1)=4$ is a local maximum.
• When $(x,y)=(-1,1)$ or $(1,-1)$, we have $D=-36<0$ so $f(x,y)$ does not take extreme values.

□