softmax 函数
https://blog.csdn.net/m0_37769093/article/details/107732606
softmax 函数如下所示：
y i = exp ( x i ) ∑ j = 1 n exp ( x j ) y_{i} = \frac{\exp(x_{i})}{\sum_{j=1}^{n}{\exp(x_j)}} yi=∑j=1nexp(xj)exp(xi)
softmax求导如下：
i = j i = j i=j 的情况：
∂ y i ∂ x i = exp ( x i ) ∑ j = 1 n exp ( x j ) − ( exp ( x i ) ) 2 ( ∑ j = 1 n exp ( x j ) ) 2 \frac{\partial y_{i}}{\partial x_{i}} = \frac{\exp(x_{i})}{\sum_{j=1}^{n}{\exp(x_j)}} - \frac{(\exp(x_{i}))^2}{(\sum_{j=1}^{n}{\exp(x_j)})^2} ∂xi∂yi=∑j=1nexp(xj)exp(xi)−(∑j=1nexp(xj))2(exp(xi))2
∂ y i ∂ x i = y i − ( y i ) 2 \frac{\partial y_{i}}{\partial x_{i}} = y_{i} - (y_{i})^2 ∂xi∂yi=yi−(yi)2
i ≠ j i \neq j i=j 的情况：
∂ y i ∂ x j = − ( exp ( x i ) × exp ( x j ) ) ( ∑ j = 1 n exp ( x j ) ) 2 \frac{\partial y_{i}}{\partial x_{j}} = - \frac{(\exp(x_{i})\times\exp(x_{j}))}{(\sum_{j=1}^{n}{\exp(x_j)})^2} ∂xj∂yi=−(∑j=1nexp(xj))2(exp(xi)×exp(xj))
∂ y i ∂ x j = − y i y j \frac{\partial y_{i}}{\partial x_{j}} = - y_{i}y_{j} ∂xj∂yi=−yiyj