LeakyReLU

α\alpha1
f(x)={αxx<0xotherwisef(x) = \begin{cases} \alpha x & x < 0 \\ x & \text{otherwise}\end{cases} 

α\alpha1
f′(x)={αx<01x>0f^{\prime}(x) = \begin{cases} \alpha & x < 0 \\ 1 & x > 0\end{cases}