normalized gaussian random matrix preserves geometry in expectation

[[concept]]

[!themes] Topics

Evaluation Error: SyntaxError: Unexpected token '>'

at DataviewInlineApi.eval (plugin:dataview:19027:21)

at evalInContext (plugin:dataview:19028:7)

at asyncEvalInContext (plugin:dataview:19038:32)

at DataviewJSRenderer.render (plugin:dataview:19064:19)

at DataviewJSRenderer.onload (plugin:dataview:18606:14)

at DataviewJSRenderer.load (app://obsidian.md/app.js:1:1182416)

at DataviewApi.executeJs (plugin:dataview:19607:18)

at DataviewCompiler.eval (plugin:digitalgarden:10763:23)

at Generator.next (<anonymous>)

at fulfilled (plugin:digitalgarden:77:24)

Important

Let $G \sim N (0, 1)^{\otimes d \times m}$ and $Y \in R^{m \times n}$ . Now, define

\hat{G} := \frac{1}{\sqrt{d}} G, Law (\hat{G}) = N {(0, \frac{1}{d})}^{\otimes d \times m}

Then

\begin{align} \mathbb{E}[\,\lvert \lvert \hat{G}y \rvert \rvert {#2} \,] &= \lvert \lvert y \rvert \rvert {#2} \\ \mathbb{E}[\,\langle \hat{G}y_{1}, \hat{G}y_{2} \rangle \,] &= \langle y_{1}, y_{2} \rangle \\ \mathbb{E}\left[ \,(\hat{G}Y)^{\intercal}(\hat{G}Y)\, \right] &= Y^{\intercal}Y \end{align}

ie, in expectation, $\hat{G}$ preserves the lengths, angles, and Gram matrices. That is, it preserves all the geometry of $R^{m}$

Line of Reasoning

Note that for any column of $Y$ , say $y = y_{i}$ , we have

Expected Magnitude

$\begin{align} \mathbb{E}[\lvert \lvert Gy \rvert \rvert {#2} ] &= \mathrm{Tr}(\text{Cov}(Gy) ) \\ &= d \lvert \lvert y \rvert \rvert {#2}$

\end{align}$$

and for any $y_{1}, y_{2}$ columns of $Y$

Expected Direction

$\begin{aligned} E [⟨ G y_{1}, G y_{2} ⟩] & = Tr (Cov (G y_{1}, G y_{2})) \\ = d ⟨ y_{1}, y_{2} ⟩ \end{aligned}$

And, looking at the Gram matrix,

Expected Gram Matrix

$E [(G Y)^{⊺} (G Y)] = d Y^{⊺} Y$

Caveats

Note

Note that this holds for any $d$ , including $d ≪ m$ and $d = 1$ . This is because we are taking expectations.

If $y$ is selected first before we realize $\hat{G}$ , then we hope that $| | \hat{G} y | |$ will concentrate about $| | y | |$ . Our calculations for the joint show that

\text{Law}(\hat{G}y) = {\cal N}\left( 0, \frac{1}{d}\lvert \lvert y \rvert \rvert {#2} I_{d} \right)

So (with rescaling, obviously), our concentration inequality for magnitude of standard gaussian random vector holds and we should see the desired behavior.

y

does not depend on

\hat{G}

Assuming we ensure $\hat{G} ⊥ y$ , the probability of actually preserving geometry depends on $d$ .

Example

If $d = 1$ , then $\hat{G} = g^{⊺}$ for some $g \in N (0, I_{d})$ . We still have

\mathbb{E}[\lvert \lvert \hat{G}y_{i} \rvert \rvert {#2} ] = \mathbb{E}[\langle g,y_{i} \rangle {#2} ] =\lvert \lvert y_{i} \rvert \rvert {#2}

But clearly $⟨ g, y_{i} ⟩^{2} \approx | | y_{i} | |^{2}$ cannot hold simultaneously for many $y_{i}$ regardless of if we choose them before realizing $\hat{G}$ .

Exercise

Construct some examples of this case

y

depends on

\hat{G}

...

If $d ≪ m$ , then once $\hat{G}$ is realized, we can pick $y \in Null (\hat{G})$ so that $0 = | | \hat{G} y | | ≪ | | y | |$ .

In this case, $\hat{G}$ will not preserve the geometry of arbitrary vectors $y$ .
In particular, we can find vectors $y$ that depend on $\hat{G}$ such that the geometry is badly distorted.

Review

#flashcards/math/rmt

Normalized gaussian random matrices preserve geometry (in expectation), BUT

Assuming we ensure $\hat{G} ⊥ y$ , the probability of {1||preserving geometry} depends on {2||the "output" dimension $d$ }.

For $\hat{G} \sim N {(0, \frac{1}{d})}^{\otimes d \times m}$ and $y \in R^{m}$ , what is the (co)variance matrix of $\hat{G} y$ ?
-?-

\frac{1}{d} \lvert \lvert y \rvert \rvert {#2} I_{d}$$ <!--SR:!2025-09-18,2,230--> If $\hat{G} \sim {\cal N}\left( 0, \frac{1}{d} \right)^{\otimes d\times m}$, why do we need $y\in \mathbb{R}^m$ to be independent of $\hat{G}$ to preserve geometry if $d<m$? -?- Otherwise, we can choose $y \in \text{Null}(\hat{G})$ and then the norm is not preserved. <!--SR:!2025-09-18,3,250--> # References >[!references] ## See Also - ### Mentions > [!mentions]+ > | File | Last Modified | > | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------- | > |normalized gaussian random matrix preserves geometry in expectation| 2025-09-13 | > |Random Matrix Lecture 02| 2025-09-13 | > { .block-language-dataview} <span class="dv-modified"><span>Created 2025-09-13 ֍ Last Modified 2025-09-18</span></span>