一类目标函数的逆向构造
INVERSE CONSTRUCTING OF A SET OF OBJECTIVE FUNCTIONS
-
摘要: 面向解决真实世界问题的神经应用需求,本文提出了一种构造目标函数的逆向方法,即将目标函数的构造任务转化为误差信号的设计。应用这一方法,我们构造出了一类的目标函数,它不仅可以解除均方误差(MSE)函数的假饱和状态,从而缩短了网络的训练时间,而且能够克服相对熵函数带来的过度适应性问题,从而提高了网络的泛化能力。Abstract: To meet the requirements with large-scale neural networks for real-world applications, an inverse way of constructing objective functions was proposed in this paper, which translates the task of constructing objective functions into the design of error signals. Followed this way, a set of objective functions has been given as examples to eliminate the false saturation in Mean Squared Error (MSE) and overspecialization in Cross Entropy (CE). The verification of its power was also made by the comparison with MSE and CE in the tasks of estimating the scaled likelihood for the Hidden Markov Models' states in the Hybrid HMM/ANN models, and showed consistent advantages with the theoretical expectations.
-
Richard M D, Lippmann R P. Neural network classifiers estimate Bayesian a posteriori probabilities[J].Neural Computation.1991, 3:461-483[2]Gish H. A probabilistic approach to the understanding and training of neural network classifiers,Proc. IEEE Intl Conf. on ASSP, New Mexico, USA: 1990, 1361-1364.[3]Haykin S. Neural Netwoks: Comprehensive Foundation. New York: Macmillan College Publishing Company, 1994, 696. ［4］Karayiannis N B. Accelerating the training of feedforward neural networks using generalized Hebbian rules for initializing the internal representations. IEEE Trans. on Neural Networks,1996, 7(2): 419-425.[4]Morgan N, Bourlard H. Continuous speech recognition: An introduction to the hybrid HMM/connectionist approach. IEEE Signal Processing Magazine, 1995, 12(3): 25-42.[5]杜利民,候自强.自动语音识别中的人工神经网络方法.物理学进展,1996,(9).[6]焦李成.神经网络系统理论,西安:西安电子科技大学出版社,1992,26-41. -
计量
- 文章访问数: 2200
- HTML全文浏览量: 75
- PDF下载量: 520
- 被引次数: 0
下载:
