I'm given the matrix $X = xx^{T}\in\mathbb{R}^{d \ x \ d}, x\in\mathbb{R}^{d}$. Does somebody know how to compute $\lambda_{max}(X)$ or $\lambda_{min}(X)$? I only want to know these two eigenvalues, the others are not really important.
I seem to be stuck.
I'm thankful for any answer.
$\endgroup$ 23 Answers
$\begingroup$Assuming $x$ is non-zero, $X$ has rank $1$, so almost all the eigenvalues are $0$. As for the single non-zero eigenvalue, consider what $Xx$ becomes.
$\endgroup$ 5 $\begingroup$We assume
$x \ne 0, \tag 1$
lest
$X = xx^T = 0, \tag 2$
and the problem is trivial. For
$x \ne 0, \tag 3$
we have
$Xx = (xx^T)x = x(x^Tx) = (x^Tx)x, \tag 4$
and we see that
$x^Tx > 0 \tag 5$
is an eigenvalue of $X = xx^T$ with associated eigenvector $x$.
Now if
$0 \ne y \in \Bbb R^d \tag 6$
is such that
$x^Ty = 0, \tag 7$
then
$Xy = (xx^T)y = x(x^Ty) = (0)y = 0, \tag 8$
i.e. $0$ is an eigenvalue of $X$ with eigenvector $y$. The mapping
$x^T(\cdot): \Bbb R^d \to \Bbb R, \; y \to x^Ty \tag 9$
is a linear functional on $\Bbb R^d$ and as such
$\dim \ker x^T(\cdot) = d - 1; \tag{10}$
thus the $0$-eigenspace of $X$, which is $\ker x^T(\cdot)$, is of dimension $d - 1$. Having exhausted the number of available dimensions of $\Bbb R^d$, we conclude that $x^Tx$ is an eigenvalue of multiplicity $1$, whilst the eigenvalue $0$ is of multiplicity $d - 1$; there are no other eigenvalues or eigenvectors of $X$.
$\endgroup$ $\begingroup$I can't add a comment to @Arthurs's answer because I don't have enough reputation, but I wanted to add my 2 cents anyway :)
Say you had another vector $\mathbf{y}$, with the same dimensions as $\mathbf{x}$. In that case $\mathbf{x^{T}y}=y_{proj_{x}}$, where $y_{proj_{x}}$ is the dot product between $\mathbf{x}$ and $\mathbf{y}$, or in other words the modulus of the projection of $\mathbf{y}$ on $\mathbf{x}$.
Similarly, the outer-product of $\mathbf{x}$ gives you your $\mathbf{X}$ matrix: $\mathbf{X}=\mathbf{xx^{T}}$. If you take a moment to look at it, $\mathbf{Xy}=\mathbf{xx^{T}y}=\mathbf{x(x^{T}y)}=\mathbf{x}y_{proj_{x}}$
Which is the projection of $\mathbf{y}$ on $\mathbf{x}$, so through this intuition you can say that the only eigenvector is the projection direction $\mathbf{x}$
Edit: Just realized that in order for it to be a projection, $\mathbf{x}$ should be a unit vector, otherwise it scales the projection by $\left\|\mathbf{x}\right\|^2$, so that would be the eigenvalue: $\left\|\mathbf{x}\right\|^2$
$\endgroup$ 0
