Any vector in ${\mathbb{C}}^n$ can be expressed as $x=Bb+y$ for some $y\in {\mathbb{C}}^n$. We show that $||A(Bb+y)-b|{|}_2^2$ is minimized when $y=0$.

$$\begin{array}{rl}||A(Bb+y)-b|{|}_2^2& =[(AB-I)b+Ay{]}^{\ast }[(AB-I)b+Ay]\\ & =||(AB-I)b|{|}_2^2+||Ay|{|}_2^2+||Ay|{|}_2^2+y^{\ast }{A}^{\ast }(AB-I)b+b^{\ast }(AB-I{)}^{\ast }Ay\\ (\ast )& =||(AB-I)b|{|}_2^2+||Ay|{|}_2^2\end{array}$$

$(\ast )$ we see that $[A^{\ast }(AB-I){]}^{\ast }=(AB-I)A=ABA-A=0$ since $B$ is a 1, 2 generalized inverse (ie, $AB$ is hermitian and $ABA=A$).

ie, the expression depends only on the term $||Ay|{|}_2^2$, which is minimized precisely when $||Ay||=0$ such as when $y=0$ for example. (the first term of the expression is constant).

• thus the solutions to the least squares problem is the set $\{Bb+y:y\in \text{Null}(A)\}$

(Uniqueness first). Suppose $B,C\in M_{n,m}$ are both 1-2-3-4 generalized inverses for $A$. NTS $B=C$.

• Claim 1: $AB=AC$
• $AB=ACAB$ since $A=ACA$ (1 generalized inverse)
• $AC$ and $AB$ are both hermitian. so we have
• $AB=C^{\ast }{A}^{\ast }{B}^{\ast }{A}^{\ast }=C^{\ast }(ABA{)}^{\ast }=C^{\ast }{A}^{\ast }$ since $ABA=A$
• But $AC$ is hermitian, so we get $AB=AC$.
• Claim 2: $BA=CA$ argued analogously to the first claim
• $BA=BACA$ since $A=ACA$
• Then $BA=BACA=A^{\ast }{B}^{\ast }{A}^{\ast }{C}^{\ast }$ since both $BA$ and $CA$ are hermitian
• $BA=(ABA{)}^{\ast }{C}^{\ast }=A^{\ast }{C}^{\ast }=CA$ since $CA$ is hermitian
  Consider $CAB$.
• By claim 2, we have $BAB=CAB$
• By claim 1, we have $CAB=CAC$.
• And by property 3 of the generalized inverses, we get $B=BAB=CAB=CAC=C$.

(Existence)
Let us first consider a special case. Suppose $\mathrm{\Sigma }$ is $M_{m,n}$ "diagonal". Ie, $i\ne j,{\mathrm{\Sigma }}_{ij}=0$. Define ${\mathrm{\Sigma }}^{{}^{†}}\in M_{n,m}$ "diagonal". And for all $i$, we have $({\mathrm{\Sigma }}^{{}^{†}}{)}_{ii}=\frac{1}{{\mathrm{\Sigma }}_{ii}}$ if ${\mathrm{\Sigma }}_{ii}\ne 0$ and $0$ otherwise. Then ${\mathrm{\Sigma }}^{{}^{†}}$ is a 1-2-3-4 generalized inverse for $\mathrm{\Sigma }$. We can check this easily

• clearly $\mathrm{\Sigma }{\mathrm{\Sigma }}^{{}^{†}}$ is hermitian, same with ${\mathrm{\Sigma }}^{{}^{†}}\mathrm{\Sigma }$.
• $\mathrm{\Sigma }{\mathrm{\Sigma }}^{{}^{†}}\mathrm{\Sigma }=\mathrm{\Sigma }$
• ${\mathrm{\Sigma }}^{{}^{†}}\mathrm{\Sigma }{\mathrm{\Sigma }}^{{}^{†}}={\mathrm{\Sigma }}^{{}^{†}}$

Now, what happens if we have a matrix $Y\in M_{m,n}$ with a 1-2-3-4 generalized inverse $Z\in M_{n,m}$? Let $U\in M_m$ be unitary, $V\in M_n$ also unitary. Then $UY{V}^{\ast }$ has a 1-2-3-4 generalized inverse $VZ{U}^{\ast }$. We can show this easily:

• $[UY{V}^{\ast }][VZ{U}^{\ast }][UY{V}^{\ast }]=UYZY{V}^{\ast }$. Then since $Z$ is 1-2-3-4 generalized inverse we get $UYZY{V}^{\ast }=UY{V}^{\ast }$
• $[UY{V}^{\ast }][VZ{U}^{\ast }]=UYZ{U}^{\ast }$ is hermitian since $YZ$ is hermitian
• (And the other two conditions are shown exactly analogously)

Say $A=U\mathrm{\Sigma }{V}^{\ast }$ is an SVD of $A$. Then the moore-penrose inverse of $A$ is $A^{{}^{†}}=V{\mathrm{\Sigma }}^{{}^{†}}{U}^{\ast }$ by the two above facts.

• if $A$ is real, then the SVD is real and so the pseudoinverse is also real.

Recall that the solutions of the least squares problem are exactly $A^{†}b+y:y\in \text{Null}(A)$. Let $A=U\mathrm{\Sigma }{V}^{\ast }$ be an SVD and the rank of $A$ is $k$.

• $\text{Null}(A)=\text{span}\{v_{k+1},v_{k+2},\dots ,v_n\}$
• $\text{Range}(A^{†})=\text{span}\{v_0,v_1,\dots ,v_k\}$ since $A^{†}=V{\mathrm{\Sigma }}^{†}{U}^{\ast }$ is (almost) an SVD. Almost because the sigmas in ${\mathrm{\Sigma }}^{†}$ are not necessarily non-decreasing. (recall that for SVD we assume that the singular values are ordered)

Thus for all $y\in \text{Null}(A)$ we have $y\perp A^{†}b$ since we can see that $\text{range}(A^{†})\perp \text{Null}(A)$ from the above. Thus for any $y\in \text{Null}(A)$, we have

$$||A^{†}b+y|{|}_2^2=[A^{†}b+y{]}^{\ast }[A^{†}b+y]=||A^{†}b|{|}_2^2+||y|{|}_2^2+0+0⟹$$

• the minimum occurs precisely when $y=0$!