statistical risk minimization problem

Data

subject:: Data Science Methods for Large Scale Graphs parent:: Graph Signals and Graph Signal Processing theme:: math notes

Statistical Risk Minimization Problem

Suppose $x,y$ are related by some (known) statistical model $p(x,y)$, and we want to estimate $p(x,y)$ using a model $\tilde{y}=f(x)$, where $f$ is a member of our hypothesis class $\mathcal{F}$.

Let $\ell (y,\hat{y})$ be our loss function. Then the statistical risk minimization problem is to minimize the expected loss over distribution $p(x,y)$: $f^{\ast }=\arg {\min }_{f\in \mathcal{F}}{\mathbb{E}}_{p(x,y)}\{\ell (y,f(x))\}$ The optimal estimator $f^{\ast }$ is the function $f\in \mathcal{F}$ with minimal expected cost over all possible functions $f$.

Note

Typically, we are interested in either

• Predicting $y$ from $x$ with the convolutional distribution $y\sim p(y|x)$.
• ex: stochastic outputs: VAEs, diffusion, etc
• Predicting $y$ from $x$ with the conditional expectation $y=\mathbb{E}(y|x)$
• ex: deterministic outputs, classical regime/supervised learning

Mentions

TABLE
FROM [[]]
 
FLATTEN choice(contains(artist, this.file.link), 1, "") + choice(contains(author, this.file.link), 1, "") + choice(contains(director, this.file.link), 1, "") + choice(contains(source, this.file.link), 1, "") as direct_source
 
WHERE !direct_source