- P-ISSN1738-6764
- E-ISSN2093-7504
- KCI
ISSN : 1738-6764
Statistical Analyses of Cross-Entropy Error Function with Probabilistic Target Encoding for Training Neural Networks
Abstract
Training neural networks with softmax outputs requires assigning target values to output nodes. Due to its simplicity, we often use one-hot encoding, which adopts “one” or “zero” as the desired output values. However, when training neural networks to minimize the cross-entropy error function between the desired and actual output node values, overfitting of neural networks to training samples becomes a significant issue. A probabilistic target encoding has been proposed to mitigate the overfitting. In this paper, we derive the optimal solutions for output nodes that minimize the cross-entropy error function using the probabilistic target encoding. In the extreme case of the probabilistic target encoding, the analysis corresponds to the cross-entropy error function with one-hot encoding. The statistical analyses conducted to derive the optimal solutions provide considerable insights, including the interval of target values for the Bayes classifier.
- keywords
- Cross-entropy Loss Function, Probabilistic Target Encoding, Optimal Solution of Output Values, Softmax Function, Neural Networks
- 다운로드 수
- 조회수
- 0KCI 피인용수
- 0WOS 피인용수