leaky ReLU

[[concept]]

Leaky ReLU

The leaky ReLU is a pointwise nonlinearity that is given as:

ρ (x) = {\begin{cases} x, x \geq 0 \\ α x, x < 0 \end{cases}

File
graph attention model
2025-02-19 graphs lecture 9