torch.nn.LeakyReLU

原型

CLASS torch.nn.LeakyReLU(negative_slope=0.01, inplace=False)

参数

• negative_slope (float) – 控制负值斜率，默认为 1e-2
• inplace (bool) – in-place 操作，默认为 False

定义

$$\text{LeakyReLU}(x)=\max (0,x)+\text{negative\_slope}\ast \min (0,x)$$

or

$$\text{LeakyReLU}(x)=\{\begin{array}{ll}x,& \text{ if }x\ge 0\\ \text{negative\_slope}\times x,& \text{ otherwise }\end{array}$$

图

代码

import torch
import torch.nn as nnm = nn.LeakyReLU(0.1)
input = torch.randn(2)
output = m(input)print("input: ", input)    # input:  tensor([-1.5754,  0.6229])
print("output: ", output)  # output:  tensor([-0.1575,  0.6229])

【参考】

LeakyReLU — PyTorch 1.13 documentation