statistical risk minimization problem

Data
Statistical Risk Minimization Problem

Suppose x,y are related by some (known) statistical model p(x,y), and we want to estimate p(x,y) using a model y~=f(x), where f is a member of our hypothesis class F.

Let (y,y^) be our loss function. Then the statistical risk minimization problem is to minimize the expected loss over distribution p(x,y):

f=argminfFEp(x,y){(y,f(x))}

The optimal estimator f is the function fF with minimal expected cost over all possible functions f.

Note

Typically, we are interested in either

  • Predicting y from x with the convolutional distribution yp(y|x).
  • ex: stochastic outputs: VAEs, diffusion, etc
  • Predicting y from x with the conditional expectation y=E(y|x)
  • ex: deterministic outputs, classical regime/supervised learning

Mentions

File
2025-02-03 graphs lecture 4
2025-02-05 graphs lecture 5