- P-ISSN1738-6764
- E-ISSN2093-7504
- KCI
ISSN : 1738-6764
Deriving a New Divergence Measure from Extended Cross-Entropy Error Function
Deriving a New Divergence Measure from Extended Cross-Entropy Error Function
Wakuya, Hiroshi (Graduate School of Science and Engineering Saga University)
Park, Sun-Gyu (Division of Architecture Mokwon University)
Noh, Hwang-Woo (Department of Visual Design Hanbat National University)
Yoo, Jae-Soo (School of Information and Communication Engineering Chungbuk National University)
Min, Byung-Won (Division of Information Communication Engineering Mokwon University)
Oh, Yong-Sun (Division of Information Communication Engineering Mokwon University)
Abstract
Relative entropy is a divergence measure between two probability density functions of a random variable. Assuming that the random variable has only two alphabets, the relative entropy becomes a cross-entropy error function that can accelerate training convergence of multi-layer perceptron neural networks. Also, the n-th order extension of cross-entropy (nCE) error function exhibits an improved performance in viewpoints of learning convergence and generalization capability. In this paper, we derive a new divergence measure between two probability density functions from the nCE error function. And the new divergence measure is compared with the relative entropy through the use of three-dimensional plots.
- keywords
- Cross-Entropy, The n-th Order Extension of Cross-Entropy, Divergence Measure, Information Theory, Neural Networks
- 다운로드 수
- 조회수
- 0KCI 피인용수
- 0WOS 피인용수