基于多流三音素DBN模型的音视频语音识别和音素切分

吕国云; 蒋冬梅; 樊养余; 赵荣椿; H.Sahli; W.Verhelst

doi:10.3724/SP.J.1146.2007.01216

基于多流三音素DBN模型的音视频语音识别和音素切分

doi: 10.3724/SP.J.1146.2007.01216

基金项目:

中国博士后科学基金和中国科技部与比利时弗拉芒大区科技合作项目([2004]487)资助课题

计量
- 文章访问数: 4057
- HTML全文浏览量: 112
- PDF下载量: 1045
- 被引次数: 0
出版历程
- 收稿日期: 2007-07-23
- 修回日期: 2008-12-11
- 刊出日期: 2009-02-19

DBN Model Based Multi-stream Asynchrony Triphone for Audio-Visual Speech Recognition and Phone Segmentation

摘要

摘要: 为实现音视频语音识别和同时对音频视频流进行准确的音素切分，该文提出一个新的多流异步三音素动态贝叶斯网络(MM-ADBN-TRI)模型，在词级别上描述了音频视频流的异步性，音频流和视频流都采用了词-三音素-状态-观测向量的层次结构，识别基元是三音素，描述了连续语音中的协同发音现象。实验结果表明：该模型在音视频语音识别和对音频视频流的音素切分方面，以及在确定音视频流的异步关系上，都具备较好的性能。
- 语音识别;动态贝叶斯网络;音素切分;音视频
Abstract: In this paper, a novel Multi-stream Multi-states Asynchronous Dynamic Bayesian Network based context-dependent TRIphone (MM-ADBN-TRI) model is proposed for audio-visual speech recognition and phone segmentation. The model looses the asynchrony of audio and visual stream to the word level. Both in audio stream and in visual stream, word-triphone-state topology structure is used. Essentially, MM-ADBN-TRI model is a triphone model whose recognition basic units are triphones, which captures the variations in real continuous speech spectra more accurately. Recognition and segmentation experiments are done on continuous digit audio-visual speech database, and results show that: MM-ADBN-TRI model obtains the best overall performance in word accuracy and phone segmentation results with time boundaries, and more reasonable asynchrony between audio and visual speech.

HTML全文

参考文献(1)

Potamianos G and Neti C, et al.. Recent advances in theautomatic recognition of audiovisual speech[J].Proc. IEEE.2003, 91(9):1306-1326[2]王志明, 蔡莲红, 艾海舟. 基于数据驱动方法的汉语文本-可视语音合成. 软件学报, 2005, 16(6): 1054-1063.Wang Z M, Cai L H, and Ai H Z. Text-to-visual speech inChinese based on data-driven approach. Journal of Software,2005, 16(6): 1054-1063.[3]Nefian A, Liang L, and Pi X, et al.. Dynamic Bayesiannetworks for audio-visual speech recognition[J].EURASIP,Journal on Applied Signal Processing.2002, 2002(11):1274-1288[4]Ravyse Ilse, Jiang D M, and Jiang X Y, et al.. DBN basedmodels for audio-visual speech analysis and recognition. 2006Pacific-Rim Conference on Multimedia (PCM 2006),Hangzhou, China, Nov, 2006: 19-30.[5]L Guoyun, Jiang Dongmei, and Sahli H, et al.. A novel DBNmodel for large vocabulary continuous speech recognition andphone segmentation. International Conference on ArtificialIntelligence and Pattern Recognition (AIPR-07), Orlando,Florida, USA, July 2007: 397-402.[6]Bilmes J and Bartels C. Graphical model architectures forspeech recognition. IEEE Signal Processing Magazine, 2005,22(5): 89-100.[7]L Guoyun, Jiang Dongmei, and Zhao Rongchun, et al..Multi-stream asynchrony Dynamic Bayesian Network modelfor audio-visual continuous speech recognition. 14thInternational Conference on systems, Signals and ImageProcessing (IWSSIP 2007), Maribor, Slovenia, June, 2007,1: 437-440.[8]Young S J, Odell J, and Woodland P C. Tree-based statetying for high accuracy acoustic modeling. In ProceedingsARPA Workshop on Human Language Technology,Plainsboro, New Jersey, USA, 1994: 307-312.[9]Zhou Yi, Gu Lie, and Zhang Hongjiang. Bayesian tangentshape model: Estimating shape and pose parameters viabayesian inference. The IEEE Conference on ComputerVision and Pattern Recognition, Wisconsin, USA, June, 2003,1: 109-116.[10]Jiang D M, Xie L, and Zhao R C, et al.. Acoustic visememodeling for speech driven animation: a case study. In Proc.1st IEEE Benelux Workshop on Model based Processing andcoding of Audio (MPCA-2002), Leuven, Belgium, November,2002, 1: 49-52.

施引文献

资源附件(0)

访问统计