A Review of Sign Language Production Technology Based on the Presentation of Sign Language Content
-
摘要: 手语是保证听障人群与健听人群之间正常交流的主要语言形式，因此研发一种能将语音或文本翻译成手语的系统至关重要，手语生成(SLP)技术是此系统的核心技术之一。然而，当前手语生成技术仍面临多通道特征协同建模困难、时序一致性难以保持、高质量手语数据稀缺、主客观评价指标存在差异以及实时生成与边缘部署挑战等瓶颈，制约了技术的实际应用。该文综述了近些年来手语生成所用数据集、评价指标及手语生成技术的发展情况。首先，该文先详细梳理了用于手语生成技术的数据集及其未来发展方向。其次，总结了手语生成方法的评价指标。然后，详细介绍了手语生成的核心架构和方法，并按照呈现形式将其技术研究方向分为基于运动图生成、基于条件视频/图像生成、基于数字人生成三类，重点分析了各类方法在解决上述瓶颈问题中的优势与局限。最后，梳理了手语生成技术的发展脉络及未来展望。Abstract:
Objective Sign language is the primary language form ensuring normal communication between hearing-impaired individuals and hearing individuals. According to World Health Organization statistics, the global population with hearing loss is large and continues to grow, which makes a bidirectional sign language–spoken language translation system highly necessary. Sign Language Recognition (SLR) and Sign Language Production (SLP) are the two core technologies of such a system. SLP aims to convert spoken language information in text or speech form into grammatically well-formed, natural, and semantically equivalent sign language action sequences, and then to generate readable sign language videos through stitching, rendering, or avatar-driven techniques. Although SLP has attracted increasing attention, current technologies still face critical bottlenecks, including difficulties in multi-channel feature co-modeling, challenges in maintaining temporal consistency, scarcity of high-quality sign language data, discrepancies between subjective and objective evaluation metrics, and constraints on real-time generation and edge deployment. Existing reviews have summarized representative methods and Transformer-based progress, but they have not sufficiently analyzed multilingual adaptation, avatar diversity, linguistic coverage, and cross-method fusion. Therefore, this review systematically examines datasets, evaluation metrics, core architectures, presentation-based generation methods, and development trends of SLP, with particular attention to the presentation forms of generated sign language content and the “uncanny valley” effect in avatar-based generation. The review aims to provide a structured reference for improving the naturalness, intelligibility, controllability, and deployability of SLP systems. Methods This review is organized around the complete technical pipeline of SLP, including datasets, evaluation metrics, model architectures, presentation-based generation methods, linguistic coverage analysis, and cross-method fusion. First, continuous sign language datasets applicable to SLP are reviewed, including CLS-Daily, RWTH-PHOENIX-2014T, How2Sign, Prompt2Sign, SignAvatars, LSE-Sign, and GSL ( Table 1 ). Their language coverage, sample scale, data modalities, sample types, number of signers, and construction years are compared. The development directions of sign language datasets are further summarized from four perspectives: continuous expansion of data scale, more comprehensive language coverage, transition from single-modal to multi-modal fusion, and increasing realism and naturalness of scenarios and expressions. Second, objective and subjective evaluation metrics are summarized. Objective metrics include Mean Per Joint Position Error (MPJPE), Dynamic Time Warping (DTW), back-translation-based semantic metrics such as BLEU and ROUGE, and Fréchet Inception Distance (FID). Subjective metrics include Mean Opinion Score (MOS), the classical four-dimensional scoring system covering semantic expression, structure and spatial usage, fluency, and Non-Manual Marker (NMM) accuracy, and the seven-dimensional mixed evaluation framework that further introduces expressiveness, mental load, and acceptance. Third, representative deep learning architectures are reviewed, including Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Generative Adversarial Network (GAN), Variational Autoencoder (VAE), and Transformer, as well as their variants and hybrid models. On this basis, SLP methods are categorized according to the presentation form of the final sign language video or image into motion graph-based generation, conditional video/image-based generation, and digital avatar-based generation. Representative models, datasets, reported results, and major limitations are compared (Table 2 ,Table 3 ). Moreover, a linguistic-dimension analysis is conducted to evaluate the coverage of the three method categories over handshape, location, movement, orientation, and NMM (Table 5 ). Finally, cross-method fusion paths and future development directions are discussed.Results and Discussions The review shows that the three presentation-based SLP paradigms have distinct advantages and limitations ( Table 4 ). Motion graph-based methods construct directed graphs from motion capture data and generate new motion sequences by graph traversal, interpolation, and transition modeling. They have natural advantages in motion continuity and physical plausibility, and can be combined with Neural Machine Translation (NMT) and image translation models for continuous text-to-pose translation. However, their performance is highly dependent on the scale, diversity, and annotation quality of motion capture data. The graph structure also suffers from limited scalability and high search complexity, which restricts motion richness, cross-language transfer, and large-scale application. Conditional video/image-based methods adopt encoder–decoder architectures and generative models such as CNN, RNN, VAE, GAN, Transformer, and diffusion models to generate sign language videos or images conditioned on text, speech, pose, or visual guidance. They can produce highly realistic videos and support end-to-end generation and multi-modal input. Representative studies have reported competitive results on datasets such as RWTH-PHOENIX-2014T, How2Sign, CLS-Daily, and Turkish Sign Language (Table 2 ). Nevertheless, temporal consistency and long-sequence stability remain insufficient, and problems such as action drift, error accumulation, occlusion sensitivity, and fine-grained finger motion distortion are still prominent. Digital avatar-based methods use parameterized human models, especially the SMPL family, to control body, hand, and facial movements in a unified parameter space. SMPL, SMPL+H, SMPL-X, STAR, and SMPLest-X provide different trade-offs among modeling capability, computational efficiency, and deployment cost (Table 3 ). Avatar-based methods offer precise pose control, standardized baselines, and strong reusability, but cross-modal temporal synchronization between facial expressions and hand movements is still inadequate. In addition, unrealistic skin texture, facial micro-expressions, eye dynamics, and motion synchronization may induce the “uncanny valley” effect, reducing the trust and acceptance of hearing-impaired users. Recent progress in neural rendering, 3D Gaussian splatting, and diffusion models, such as GaussianAvatars and DiffPoseTalk, has improved facial detail and expression controllability, but challenges remain in real-time performance, cross-identity generalization, and semantic alignment with sign language NMMs.From the linguistic perspective, the three method categories show significant differences in covering the five core parameters of sign language (Table 5 ). Motion graph-based methods are limited by database coverage in handshape generation and have weak absolute position control and orientation controllability. Conditional video/image-based methods can generate realistic hand textures and large-scale body movements, but they lack explicit 3D spatial modeling and are weak in fine-grained finger motion, palm orientation, and temporal alignment of facial expressions with manual signs. Avatar-based methods achieve relatively high handshape precision and precise spatial control through parameterized models, but spatial grammar modeling, orientation control, joint face–hand optimization, and NMM semantic modeling remain insufficient. Overall, no single technical route can simultaneously satisfy the requirements of temporal consistency, visual realism, structural controllability, linguistic completeness, and real-time deployment. Cross-method fusion has therefore become an important development direction. Three fusion paths are identified: the retrieval–generation path, which introduces motion graph retrieval as a prior or initialization for deep generative models; the parametric structure–neural rendering path, which combines SMPL-X-based geometric constraints with diffusion models or 3D Gaussian splatting; and the unified discrete representation–large model path, which maps continuous motions into a shared discrete codebook and uses large language models for end-to-end semantic-to-motion modeling. These paths can integrate complementary advantages, but they also introduce new problems such as inter-module error propagation, inconsistent training objectives, and increased system complexity. Therefore, unified joint optimization objectives and end-to-end training mechanisms are essential for achieving genuine fusion gains rather than simple stacking. Conclusions and Prospects SLP is a key technology for building an accessible communication bridge between hearing-impaired and hearing populations. Although rapid progress has been made in datasets, generative models, avatar modeling, and evaluation methods, large-scale practical deployment still faces multiple challenges. Future research should advance in five dimensions. First, latent-space modeling and generation paradigm innovation should be strengthened. The combination of Vector Quantized Variational Autoencoder (VQ-VAE) and Transformer can construct compact motion codebooks and reduce the complexity of long-sequence modeling, while diffusion models and flow models are expected to improve training stability, sample diversity, and generation quality. Hybrid architectures that combine the real-time capability and interpretability of VAEs with the high-quality generation of diffusion models deserve further exploration. Second, sign language dataset construction should shift from simply pursuing sample scale to coordinated development of scale, quality, scenario coverage, and annotation. Fine-grained annotations covering frame-level 3D poses, expression parameters, handshape categories, and temporal alignment should be established, and real-world corpora covering daily conversation, public services, education, and healthcare should be supplemented. Cross-lingual data sharing and pre-training mechanisms should also be explored to support low-resource sign languages and dialects. Third, neural rendering and personalized avatars should be further developed. Neural Radiance Fields and 3D Gaussian splatting can improve skin texture, dynamic lighting, and hair details, while text-driven diffusion models can support personalized appearance generation. Hybrid representations that combine parameterized models with neural rendering are expected to preserve physical plausibility and driving controllability while improving visual realism and reducing the uncanny valley effect. Fourth, a multi-dimensional evaluation system combining objective and subjective metrics should be established. In addition to geometric and temporal metrics such as MPJPE and DTW, structured linguistic evaluation dimensions should be introduced, including handshape classification accuracy, temporal alignment accuracy of NMMs, and spatial location consistency. Standardized subjective evaluation protocols and multi-dimensional questionnaires should be developed, and comprehension accuracy should be treated as a core metric. Fifth, end-to-end system integration and scenario-based deployment should be promoted. SLP should be deeply integrated with speech recognition, machine translation, and large language models to build real-time “speech/text-to-sign language” translation systems. Cloud–edge collaborative computing, knowledge distillation, neural architecture search, dynamic inference, and quantization compression can support low-latency inference on mobile and embedded devices. Scenario adaptation in education, medical consultation, judicial services, public transportation, and public service halls will be an important breakthrough for moving SLP from laboratory prototypes to large-scale public services and daily assistance. -
表 2 视频生成方法
生成
方法模型方法 数据集 结果 比较 运动图 Motion Graph + NMT + pix2pixHD[78] Phoenix-2014T BLEU-4=15.26 图规模与检索效率
制约了动作丰富度条件图像/视频 U-Net+cGAN[78,79] Turkish-Sign BLEU-4 = 16.7；
FID = 52.3对遮挡与细粒度手指
运动敏感，精度不高Transformer+LDM[16] How2Sign BLEU-4 = 19.8;
FID = 39.1Transformer-CVAE+GAN[17] CLS-Daily BLEU-4 = 18.3；
DTW-MJE = 7.1°PSAT-RS(金字塔半自回归 Transformer + 富语义嵌入)[77] PHOENIX14T、CSL BLEU4=10.01、ROUGE=28.73 数字人 Rule-based+IK+Smooth-Skeleton[80] 自采3D关键帧库 DTW-MJE = 7.4 动作精度和自然度较高，
易出现恐怖谷效应CLIP-条件 Transformer-CVAE[93] ASL3DWord (WLASL 子集) DTW-MJE = 6.8°；FID = 4.12 SignVAE[21] SignAvatars FID=4.359(ASL)；0.884(HamNoSys)；0.756(Word-300)；1.335(Phoenix-2014T) GNN + diffusion + CLIP[95] How2Sign BLEU-4=13.12;
DTW-MJE =7.83°;
FID=1.52
下载: 导出CSV
表 4 三类手语生成方法比较
生成方法 生成质量 时序一致性 训练成本 推理速度 可迁移性 核心优势与主要局限 基于运动图 中等。依赖动作捕捉数据库质量，动作丰富度受图规模限制 过渡姿势的运动趋势如速度和加速度连贯，运动片段在物理层面连续 较低。无需深度学习训练，但需投入大量资源构建高质量动作捕捉数据库 受图搜索复杂度制约。随节点数和边数增加，分支定界搜索时间显著上升 较差。严重依赖特定动作捕捉数据，跨语种、跨场景迁移需
重建图结构生成的序列合理可控但
数据获取困难，图结构可扩展性较差，动作丰富
度受限基于条件视频/图像生成 高。GAN可生成照片级真实感视频；Diffusion模型进一步提升视觉质量与
多样性GAN长序列稳定性差，存在动作漂移；Transformer和Diffusion有所改善但仍存在误差累积 较高。需要大规模标注数据集和较高GPU算力；Diffusion模型训练要求更高 GAN推理较快，适合实时应用；Diffusion模型需多步去噪，推理速度较慢 中等。Transformer架构跨语种适配性较好，但仍需目标语种数据
微调生成的视频视觉质量高且多为端到端生成，过程简洁，并且适应多模态输入；但时序一致性难以保持，且长序列生成不稳定 基于
数字人较高。SMPL-X可实现姿态的精细化控制。结合神经渲染可实现高保真皮肤纹理 中等。分通道独立建模导致面部－手部时序相位差，跨模态同步精度不足 中等。需3D人体模型和姿态参数标注，但可利用预训练SMPL模型降低从头训练成本 参数化模型推理较快；但高斯溅射/NeRF渲染计算量大，实时性受限 较好。SMPL系列为标准化基线，跨项目复用性强，生态成熟 姿态控制精确，可驱动个性化形象且标准化基线生态成熟；但面部－手部同步精度不足，外观细节建模难度高且存在“恐怖谷效应”
下载: 导出CSV
表 5 三类生成方法在语言学参数维度上的覆盖度比较
语言学参数 基于运动图 基于条件视频/图像 基于数字人 手型 受限于数据库覆盖 手指细节精度不足 参数化可控，精度较高 位置 轨迹检索，绝对位置控制较弱 缺乏三维空间显式建模 精确可控，空间语法建模弱 运动 物理连续性好 大范围运动表现好，细粒度差 物理合理，抖动噪声存在问题 方向 难以独立控制 丢失三维方向信息 间接控制，精度不足 非手控标记 几乎不涉及 可生成但时序对齐差 可生成但联合优化不成熟
下载: 导出CSV
-
[1] World Health Organization. World report on hearing[EB/OL]. https://www.who.int/publications/i/item/9789240020481, 2021. [2] 中国残疾人联合会. 2010年末全国残疾人总数及各类、不同残疾等级人数[EB/OL]. https://www.cdpf.org.cn/zwgk/zccx/cjrgk/15e9ac67d7124f3fb4a23b7e2ac739aa.htm, 2021.China Disabled Persons' Federation. Total number of persons with disabilities in China at the end of 2010 and the number of persons with different types and grades of disabilities[EB/OL]. https://www.cdpf.org.cn/zwgk/zccx/cjrgk/15e9ac67d7124f3fb4a23b7e2ac739aa.htm, 2021. (查阅网上资料,未找到本条文献英文信息,请确认). [3] 陶唐飞, 刘天宇. 基于手语表达内容与表达特征的手语识别技术综述[J]. 电子与信息学报, 2023, 45(10): 3439–3457. doi: 10.11999/JEIT221051.TAO Tangfei and LIU Tianyu. A survey of sign language recognition technology based on sign language expression content and expression characteristics[J]. Journal of Electronics & Information Technology, 2023, 45(10): 3439–3457. doi: 10.11999/JEIT221051. [4] RASTGOO R, KIANI K, ESCALERA S, et al. Sign language production: A review[C]. Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Nashville, USA, 2021: 3446–3456. doi: 10.1109/CVPRW53098.2021.00384. [5] YAZDANI S, VAN GENABITH J, and ESPAÑA-BONET C. Continual learning in multilingual sign language translation[C]. Proceedings of 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Albuquerque, USA, 2025: 10923–10938. doi: 10.18653/v1/2025.naacl-long.546. [6] 朱洋, 曾庆宁, 赵学军. 双通道解码的端到端连续语音识别[J]. 桂林电子科技大学学报, 2024, 44(2): 167–173. doi: 10.16725/j.1673-808X.2023223.ZHU Yang, ZENG Qingning, and ZHAO Xuejun. End-to-end continuous speech recognition with dual-channel decoding[J]. Journal of Guilin University of Electronic Technology, 2024, 44(2): 167–173. doi: 10.16725/j.1673-808X.2023223. [7] STOLL S, CAMGOZ N C, HADFIELD S, et al. Text2Sign: Towards sign language production using neural machine translation and generative adversarial networks[J]. International Journal of Computer Vision, 2020, 128(4): 891–908. doi: 10.1007/s11263-019-01281-2. [8] LOPER M, MAHMOOD N, ROMERO J, et al. SMPL: A skinned multi-person linear model[J]. Seminal Graphics Papers: Pushing the Boundaries, 2023, 2: 88. doi: 10.1145/3596711.3596800. [9] MADADI M, BERTICHE H, and ESCALERA S. SMPLR: Deep learning based SMPL reverse for 3D human pose and shape recovery[J]. Pattern Recognition, 2020, 106: 107472. doi: 10.1016/j.patcog.2020.107472. [10] 张淑军, 张群, 李辉. 基于深度学习的手语识别综述[J]. 电子与信息学报, 2020, 42(4): 1021–1032. doi: 10.11999/JEIT190416.ZHANG Shujun, ZHANG Qun, and LI Hui. Review of sign language recognition based on deep learning[J]. Journal of Electronics & Information Technology, 2020, 42(4): 1021–1032. doi: 10.11999/JEIT190416. [11] STOLL S, CAMGÖZ N C, HADFIELD S, et al. Sign language production using neural machine translation and generative adversarial networks[C]. Proceedings of the British Machine Vision Conference (BMVC), Newcastle, UK, 2018: 304. [12] 孔乐毅, 张金艺, 楼亮亮. 有限样本条件下欠规范手语识别容错特征扩充[J]. 计算机研究与发展, 2022, 59(3): 683–693. doi: 10.7544/issn1000-1239.20200789.KONG Leyi, ZHANG Jinyi, and LOU Liangliang. Tolerance feature extension of substandard sign language recognition with finite samples[J]. Journal of Computer Research and Development, 2022, 59(3): 683–693. doi: 10.7544/issn1000-1239.20200789. [13] CAMGOZ C N, KOLLER O, HADFIELD S, et al. Sign language transformers: Joint end-to-end sign language recognition and translation[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 10020–10030. doi: 10.1109/CVPR42600.2020.01004. [14] SAUNDERS B, CAMGOZ N C, and BOWDEN R. Progressive transformers for end-to-end sign language production[C]. Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK, 2020: 687–705. doi: 10.1007/978-3-030-58621-8_40. [15] 郭丹, 唐申庚, 洪日昌, 等. 手语识别、翻译与生成综述[J]. 计算机科学, 2021, 48(3): 60–70. doi: 10.11896/jsjkx.210100227.GUO Dan, TANG Shengeng, HONG Richang, et al. Review of sign language recognition, translation and generation[J]. Computer Science, 2021, 48(3): 60–70. doi: 10.11896/jsjkx.210100227. [16] RASTGOO R, KIANI K, ESCALERA S, et al. A survey on recent advances in Sign Language Production[J]. Expert Systems with Applications, 2024, 243: 122846. doi: 10.1016/j.eswa.2023.122846. [17] ZHOU Hao, ZHOU Wengang, QI Weizhen, et al. Improving sign language translation with monolingual data by sign back-translation[C]. Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition: CVPR 2021, Nashville, USA, 2021: 1316–1325. doi: 10.1109/CVPR46437.2021.00137. [18] CAMGOZ N C, HADFIELD S, KOLLER O, et al. Neural sign language translation[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 7784–7793. doi: 10.1109/CVPR.2018.00812. [19] DUARTE A, PALASKAR S, VENTURA L, et al. How2Sign: A large-scale multimodal dataset for continuous American sign language[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 2734–2743. doi: 10.1109/CVPR46437.2021.00276. [20] FANG Sen, CHEN Chen, WANG Lei, et al. SignLLM: Sign language production large language models[C]. Proceedings of the IEEE/CVF International Conference on Computer Vision, Honolulu, USA, 2025: 6681–6693. doi: 10.1109/ICCVW69036.2025.00690. [21] YU Zhengdi, HUANG Shaoli, CHENG Yongkang, et al. SignAvatars: A large-scale 3D sign language holistic motion dataset and benchmark[C]. Proceedings of the 18th European Conference on Computer Vision - ECCV 2024, Milan, Italy, 2024: 1–19. doi: 10.1007/978-3-031-72652-1_1. [22] GUTIERREZ-SIGUT E, COSTELLO B, BAUS C, et al. LSE-Sign: A lexical database for Spanish sign language[J]. Behavior Research Methods, 2016, 48(1): 123–137. doi: 10.3758/s13428-014-0560-1. [23] ADALOGLOU N, CHATZIS T, PAPASTRATIS I, et al. A comprehensive study on deep learning-based methods for sign language recognition[J]. IEEE Transactions on Multimedia, 2022, 24: 1750–1762. doi: 10.1109/TMM.2021.3070438. [24] BRAGG D, KOLLER O, BELLARD M, et al. Sign language recognition, generation, and translation: An interdisciplinary perspective[C]. Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility, Pittsburgh, USA, 2019: 16–31. doi: 10.1145/3308561.3353774. [25] SAKOE H and CHIBA S. Dynamic programming algorithm optimization for spoken word recognition[J]. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1978, 26(1): 43–49. doi: 10.1109/TASSP.1978.1163055. [26] PAPINENI K, ROUKOS S, WARD T, et al. BLEU: A method for automatic evaluation of machine translation[C]. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, USA, 2002: 311–318. doi: 10.3115/1073083.1073135. [27] LIN C Y. ROUGE: A package for automatic evaluation of summaries[C]. Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL) - Text Summarization Branches Out, Barcelona, Spain, 2004: 74–81. [28] HEUSEL M, RAMSAUER H, UNTERTHINER T, et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium[C]. Proceedings of the 31st Conference on Neural Information Processing Systems (NeurIPS), Long Beach, USA, 2017: 6629–6640. [29] International Telecommunication Union. Methods for Subjective Determination of Transmission Quality[M]. Geneva: International Telecommunication Union, 1996. (查阅网上资料, 未找到本条文献页码信息, 请补充). [30] YUAN Zhenxun, WANG Haodong, ZHAO Kangwen, et al. Signing, not just spelling: A syntactically-aware sign language generation system with large language models[J]. Multimedia Systems, 2026, 32(5): 327. doi: 10.1007/s00530-026-02316-8. [31] LIU Yang, KANG Hui, HE Yurun, et al. Beyond technical metrics: Understanding the gap between AI performance and deaf user experience in Chinese natural sign language generation[C]. Proceedings of 2026 CHI Conference on Human Factors in Computing Systems, Barcelona, Spain, 2026: 1578. doi: 10.1145/3772318.3791429. [32] KRICHEN M. Convolutional neural networks: A survey[J]. Computers, 2023, 12(8): 151. doi: 10.3390/computers12080151. [33] 武东辉, 王金凤, 仇森, 等. 基于EWBiLSTM-ATT的数据手套手语识别[J]. 计算机工程, 2025, 51(8): 107–119. doi: 10.19678/j.issn.1000-3428.0070202.WU Donghui, WANG Jinfeng, QIU Sen, et al. Sign language recognition using data gloves based on EWBiLSTM-ATT[J]. Computer Engineering, 2025, 51(8): 107–119. doi: 10.19678/j.issn.1000-3428.0070202. [34] ABDELGHFAR H A, AHMED A M, ALANI A A, et al. QSLRS-CNN: Qur'anic sign language recognition system based on convolutional neural networks[J]. The Imaging Science Journal, 2024, 72(2): 254–266. doi: 10.1080/13682199.2023.2202576. [35] KRALJEVIĆ L, RUSSO M, PAUKOVIĆ M, et al. A dynamic gesture recognition interface for smart home control based on Croatian sign language[J]. Applied Sciences, 2020, 10(7): 2300. doi: 10.3390/app10072300. [36] VAHDANI E, JING Longlong, HUENERFAUTH M, et al. Multi-modal multi-channel American sign language recognition[J]. International Journal of Artificial Intelligence and Robotics Research, 2024, 1(1): 2450001. doi: 10.1142/S2972335324500017. [37] 钱惠敏, 陈实, 皇甫晓瑛. 基于双流-非局部时空残差卷积神经网络的人体行为识别[J]. 电子与信息学报, 2024, 46(3): 1100–1108. doi: 10.11999/JEIT230168.QIAN Huimin, CHEN Shi, and HUANGFU Xiaoying. Human activities recognition based on two-stream NonLocal spatial temporal residual convolution neural network[J]. Journal of Electronics & Information Technology, 2024, 46(3): 1100–1108. doi: 10.11999/JEIT230168. [38] NARAYAN S, MAZUMDAR A P, and VIPPARTHI S K. SBI-DHGR: Skeleton-based intelligent dynamic hand gestures recognition[J]. Expert Systems with Applications, 2023, 232: 120735. doi: 10.1016/j.eswa.2023.120735. [39] XIONG Baoping, CHEN Wensheng, NIU Yinxi, et al. A global and local feature fused CNN architecture for the sEMG-based hand gesture recognition[J]. Computers in Biology and Medicine, 2023, 166: 107497. doi: 10.1016/j.compbiomed.2023.107497. [40] AROOJ S, ALTAF S, AHMAD S, et al. Enhancing sign language recognition using CNN and SIFT: A case study on Pakistan sign language[J]. Journal of King Saud University - Computer and Information Sciences, 2024, 36(2): 101934. doi: 10.1016/j.jksuci.2024.101934. [41] 杨淑莹, 田迪, 郭杨杨, 等. 端到端手语翻译研究[J]. 系统科学学报, 2024, 32(1): 71–77.YANG Shuying, TIAN Di, GUO Yangyang, et al. Research on end-to-end sign language translation[J]. Journal of Systems Science, 2024, 32(1): 71–77. [42] XIA Kun, LU Weiwei, FAN Hongliang, et al. A sign language recognition system applied to deaf-mute medical consultation[J]. Sensors, 2022, 22(23): 9107. doi: 10.3390/s22239107. [43] WANG Wei, BO Xiangkun, LI Weilu, et al. Triboelectric bending sensors for AI‐enabled sign language recognition[J]. Advanced Science, 2025, 12(8): 2408384. doi: 10.1002/advs.202408384. [44] CHO K, VAN MERRIËNBOER B, GÜLÇEHRE Ç, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]. Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 2014: 1724–1734. doi: 10.3115/v1/D14-1179. [45] XIAO Qinkun, QIN Minying, and YIN Yuting. Skeleton-based Chinese sign language recognition and generation for bidirectional communication between deaf and hearing people[J]. Neural Networks, 2020, 125: 41–55. doi: 10.1016/j.neunet.2020.01.030. [46] HWANG E J, KIM J H, and PARK J C. Non-autoregressive sign language production with gaussian space[C]. Proceedings of the 32nd British Machine Vision Conference, 2021: 197. (查阅网上资料, 未找到本条文献出版地信息, 请确认). [47] GRAVES A, FERNÁNDEZ S, GOMEZ F, et al. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks[C]. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, USA, 2006: 369–376. doi: 10.1145/1143844.1143891. [48] SAUNDERS B, CAMGOZ N C, and BOWDEN R. Signing at scale: Learning to co-articulate signs for large-scale photo-realistic sign language production[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 5131–5141. doi: 10.1109/CVPR52688.2022.00508. [49] SAUNDERS B, BOWDEN R, and CAMGÖZ N C. Adversarial training for multi-channel sign language production[C]. Proceedings of the 31st British Machine Vision Conference, 2020. doi: 10.5244/C.34.63. (查阅网上资料,未找到本条文献出版地信息,请确认). [50] KRISHNA S, UKEY J, and J D B. GAN based Indian sign language synthesis[C]. Proceedings of the Twelfth Indian Conference on Computer Vision, Graphics and Image Processing, Jodhpur, India, 2021: 42. doi: 10.1145/3490035.3490301. [51] PAPASTRATIS I, DIMITROPOULOS K, and DARAS P. Continuous sign language recognition through a context-aware generative adversarial network[J]. Sensors, 2021, 21(7): 2437. doi: 10.3390/S21072437. [52] XIE Pan, PENG Taiying, DU Yao, et al. Sign language production with latent motion transformer[C]. Proceedings of 2024 IEEE/CVF Winter Conference on Applications of Computer Vision: WACV 2024, Waikoloa, USA, 2024: 3012–3022. doi: 10.1109/WACV57701.2024.00300. [53] MA Xiaohan, JIN Rize, and CHUNG T S. Multi-channel spatio-temporal transformer for sign language production[C]. Proceedings of 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, Turin, Italy, 2024: 11699–11712. doi: 10.63317/4b2rkf4368ih. [54] FANG Sen, SUI Chunyu, ZHOU Yanghao, et al. SignDiff: Diffusion model for American sign language production[C]. Proceedings of 2025 IEEE 19th International Conference on Automatic Face and Gesture Recognition (FG), Tampa/Clearwater, USA, 2025: 1–11. doi: 10.1109/FG61629.2025.11099440. [55] TANG Shengeng, XUE Feng, WU Jingjing, et al. Gloss-driven conditional diffusion models for sign language production[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2025, 21(4): 105. doi: 10.1145/3663572. [56] BALTATZIS V, POTAMIAS R A, VERVERAS E, et al. Neural sign actors: A diffusion model for 3D sign language production from text[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2024: 1985–1995. doi: 10.1109/CVPR52733.2024.00194. [57] LI Jing, KANG Di, PEI Wenjie, et al. Audio2Gestures: Generating diverse gestures from speech audio with conditional variational autoencoders[C]. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, Canada, 2021: 11273–11282. doi: 10.1109/ICCV48922.2021.01110. [58] HWANG E J, LEE H, and PARK J C. A gloss-free sign language production with discrete representation[C]. Proceedings of 2024 18th International Conference on Automatic Face and Gesture Recognition (FG), Istanbul, Turkiye, 2024: 1–6. doi: 10.1109/FG59268.2024.10581980. [59] LAKHAL M I and BOWDEN R. Diversity-aware sign language production through a pose encoding variational autoencoder[C]. Proceedings of the IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG), Istanbul, Turkiye, 2024: 1–10. doi: 10.1109/FG59268.2024.10581951. [60] DONG Lu, WANG Xiao, and Nwogu I. Word-conditioned 3D American sign language motion generation[C]. Proceedings of the Association for Computational Linguistics: EMNLP 2024, Miami, USA, 2024: 9993–9999. doi: 10.18653/v1/2024.findings-emnlp.584. [61] ZHAO Rui, ZHANG Liang, FU Biao, et al. Conditional variational autoencoder for sign language translation with cross-modal alignment[C]. Proceedings of the 38th AAAI Conference on Artificial Intelligence, Vancouver, Canada, 2024: 19643–19651. doi: 10.1609/aaai.v38i17.29937. [62] AO Tenglong, GAO Qingzhe, LOU Yuke, et al. Rhythmic gesticulator: Rhythm-aware co-speech gesture synthesis with hierarchical neural embeddings[J]. ACM Transactions on Graphics (TOG), 2022, 41(6): 209. doi: 10.1145/3550454.3555435. [63] YIN Zhizhuo, Tsui Y H, and HUI Pan. M3G: Multi-granular gesture generator for audio-driven full-body human motion synthesis[C]. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025: 1–10. doi: 10.48550/arXiv.2505.08293. (查阅网上资料,未找到本条文献信息,请确认). [64] LI Xinjie, CHEN Ziyi, YU Xinlu, et al. Co-speech gesture video generation with implicit motion-audio entanglement[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, USA, 2025: 11384–11394. doi: 10.1109/CVPR52734.2025.01063. [65] 邵舒羽, 杜垚, 范晓丽. 基于Transformer和多模态对齐的非自回归手语翻译技术研究[J]. 电子与信息学报, 2024, 46(7): 2932–2941. doi: 10.11999/JEIT230801.SHAO Shuyu, DU Yao, and FAN Xiaoli. Non-autoregressive sign language translation technology based on transformer and multimodal alignment[J]. Journal of Electronics & Information Technology, 2024, 46(7): 2932–2941. doi: 10.11999/JEIT230801. [66] SAUNDERS B, CAMGOZ N C, and BOWDEN R. Continuous 3D multi-channel sign language production via progressive transformers and mixture density networks[J]. International Journal of Computer Vision, 2021, 129(7): 2113–2135. doi: 10.1007/s11263-021-01457-9. [67] ARIB S H, AKTER R, RAHMAN S, et al. SignFormer-GCN: Continuous sign language translation using spatio-temporal graph convolutional networks[J]. PLoS One, 2025, 20(2): e0316298. doi: 10.1371/journal.pone.0316298. [68] 韩宗旺, 杨涵, 吴世青, 等. 时空自适应图卷积与Transformer结合的动作识别网络[J]. 电子与信息学报, 2024, 46(6): 2587–2595. doi: 10.11999/JEIT230551.HAN Zongwang, YANG Han, WU Shiqing, et al. Action recognition network combining spatio-temporal adaptive graph convolution and transformer[J]. Journal of Electronics & Information Technology, 2024, 46(6): 2587–2595. doi: 10.11999/JEIT230551. [69] DAMDOO R and KUMAR P. SignEdgeLVM transformer model for enhanced sign language translation on edge devices[J]. Discover Computing, 2025, 28(1): 15. doi: 10.1007/s10791-025-09509-1. [70] TAŞYÜREK S M, KIZILTEPE T, and KELES H Y. Disentangle and regularize: Sign language production with articulator-based disentanglement and channel-aware regularization[C]. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Tucson, USA, 2026: 8458–8467. doi: 10.1109/WACV61042.2026.00816. [71] KOVAR L, GLEICHER M, and PIGHIN F. Motion graphs[J]. ACM Transactions on Graphics, 2002, 21(3): 473–482. doi: 10.1145/566654.566605. [72] ARIKAN O and FORSYTH D A. Interactive motion generation from examples[J]. ACM Transactions on Graphics (TOG), 2002, 21(3): 483–490. doi: 10.1145/566654.566606. [73] LEE J and SHIN S Y. A hierarchical approach to interactive motion editing for human-like figures[C]. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 1999: 39–48. doi: 10.1145/311535.311539. (查阅网上资料,未找到本条文献出版地信息,请确认). [74] CHEN Qifeng and KOLTUN V. Photographic image synthesis with cascaded refinement networks[C]. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 2017: 1520–1529. doi: 10.1109/ICCV.2017.168. [75] GREGOR K, DANIHELKA I, GRAVES A, et al. DRAW: A recurrent neural network for image generation[C]. Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 2015: 1462–1471. [76] SHARMA P and FILHOL M. Sign language synthesis using pose priors[C]. Proceedings of the 9th International Conference on Movement and Computing (MOCO'24), Utrecht, Netherlands, 2024: 15. doi: 10.1145/3658852.3659080. [77] CUI Zhenchao, CHEN Ziang, LI Zhaoxin, et al. A pyramid semi-autoregressive transformer with rich semantics for sign language production[J]. Sensors, 2022, 22(24): 9606. doi: 10.3390/s22249606. [78] OZKAN F, TEKIN H K, and KELES H Y. Turkish sign language video generation from pose sequences using a conditional GAN model[C]. Proceedings of 2024 9th International Conference on Computer Science and Engineering (UBMK), Antalya, Turkiye, 2024: 1–6. doi: 10.1109/UBMK63289.2024.10773583. [79] ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 5967–5976. doi: 10.1109/CVPR.2017.632. [80] GIBET S, LEFEBVRE-ALBARET F, HAMON L, et al. Interactive editing in French sign language dedicated to virtual signers: Requirements and challenges[J]. Universal Access in the Information Society, 2016, 15(4): 525–539. doi: 10.1007/s10209-015-0411-6. [81] TRAN D, BOURDEV L, FERGUS R, et al. Learning spatiotemporal features with 3D convolutional networks[C]. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 4489–4497. doi: 10.1109/ICCV.2015.510. [82] ZHOU Yizhou, SUN Xiaoyan, ZHA Zhengjun, et al. MiCT: Mixed 3D/2D convolutional tube for human action recognition[C]. Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 449–458. doi: 10.1109/CVPR.2018.00054. [83] ROMERO J, TZIONAS D, and BLACK M J. Embodied hands: Modeling and capturing hands and bodies together[J]. ACM Transactions on Graphics, 2017, 36(6): 245. doi: 10.1145/3130800.3130883. [84] PAVLAKOS G, CHOUTAS V, GHORBANI N, et al. Expressive body capture: 3D hands, face, and body from a single image[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 10967–10977. doi: 10.1109/CVPR.2019.01123. [85] OSMAN A A A, BOLKART T, and BLACK M J. STAR: Sparse trained articulated human body regressor[C]. Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK, 2020: 598–613. doi: 10.1007/978-3-030-58539-6_36. [86] YIN Wanqi, CAI Zhongang, WANG Ruisi, et al. SMPLest-X: Ultimate scaling for expressive human pose and shape estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026, 48(2): 1778–1794. doi: 10.1109/TPAMI.2025.3618174. [87] WALSH H, SAUNDERS B, and BOWDEN R. Sign stitching: A novel approach to sign language production[C]. Proceedings of the 35th British Machine Vision Conference, Glasgow, UK, 2024. [88] SHEN Kaiyue, GUO Chen, KAUFMANN M, et al. X-Avatar: Expressive human avatars[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 16911–16921. doi: 10.1109/CVPR52729.2023.01622. [89] DONG Lu, CHAUDHARY L, XU Fei, et al. SignAvatar: Sign language 3D motion reconstruction and generation[C]. Proceedings of 2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG), Istanbul, Turkiye, 2024: 1–10. doi: 10.1109/FG59268.2024.10581934. [90] ZUO Ronglai, WEI Fangyun, CHEN Zenggui, et al. A simple baseline for spoken language to sign language translation with 3D avatars[C]. Proceedings of the 18th European Conference on Computer Vision, Milan, Italy, 2024: 36–54. doi: 10.1007/978-3-031-72967-6_3. [91] IVASHECHKIN M, MENDEZ O, and BOWDEN R. SignSplat: Rendering sign language via Gaussian splatting[J]. arXiv preprint arXiv: 2505.02108, 2025. doi: 10.48550/arXiv.2505.02108. [92] KRISHNAMURTHY S, BHAT V, and JAIN A. DiffSign: AI-assisted generation of customizable sign language videos with enhanced realism[C]. Proceedings of the European Conference on Computer Vision, Milan, Italy, 2024: 84–99. doi: 10.1007/978-3-031-92591-7_6. [93] YU Ronghao, LIU Yun, BAI Xiyue, et al. 3DSignDiff: Towards 3D sign language gesture generation[C]. Proceedings of the ICASSP 2025–2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Hyderabad, India, 2025: 1–5. doi: 10.1109/ICASSP49660.2025.10890302. [94] CAO Yukang, CAO Yanpei, HAN Kai, et al. DreamAvatar: Text-and-shape guided 3D human avatar generation via diffusion models[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2024: 958–968. doi: 10.1109/CVPR52733.2024.00097. [95] HONG Fangzhou, ZHANG Mingyuan, PAN Liang, et al. AvatarCLIP: Zero-shot text-driven generation and animation of 3D avatars[J]. ACM Transactions on Graphics (TOG), 2022, 41(4): 161. doi: 10.1145/3528223.3530094. [96] LIAO Tingting, YI Hongwei, XIU Yuliang, et al. Tada! Text to animatable digital avatars[C]. Proceedings of 2024 International Conference on 3D Vision (3DV), Davos, Switzerland, 2024: 1508–1519. doi: 10.1109/3DV62453.2024.00150. [97] OTHMAN A. Sign Language Processing: From Gesture to Meaning[M]. Cham: Springer, 2024: 978–973. doi: 10.1007/978-3-031-68763-1. [98] LUPIÓN M, NAVARRO-LÁZARO J, GONZÁLEZ-RUIZ V, et al. Enhancing smart home accessibility for the deaf through transformer-based sign language production and avatars[M]. BRAVO J, NUGENT C, and CLELAND I. Proceedings of the International Conference on Ubiquitous Computing and Ambient Intelligence (UCAmI 2024). Cham: Springer, 2024: 264–274. doi: 10.1007/978-3-031-77571-0_26. [99] MORI M, MACDORMAN K, and KAGEKI N. The uncanny valley [from the field][J]. IEEE Robotics & Automation Magazine, 2012, 19(2): 98–100. doi: 10.1109/MRA.2012.2192811. [100] 周琪, 雷月秋. 好奇还是恐惧?——社交媒体用户与虚拟数字人的社交互动研究[J]. 新闻大学, 2025(7): 33–47, 121–122. doi: 10.20050/j.cnki.xwdx.2025.07.003.ZHOU Qi and LEI Yueqiu. Curious or fearful?—Exploring the social interaction with human avatars based on a serial-mediation model[J]. Journalism Research, 2025(7): 33–47, 121–122. doi: 10.20050/j.cnki.xwdx.2025.07.003. [101] 神雨丹. 基于恐怖谷效应的数字虚拟角色设计研究[J]. 心理学进展, 2022, 12(11): 3935–3943. doi: 10.12677/AP.2022.1211476.SHEN Yudan. Research on digital virtual character design based on the uncanny valley effect[J]. Advances in Psychology, 2022, 12(11): 3935–3943. doi: 10.12677/AP.2022.1211476. [102] QIAN Shenhan, KIRSCHSTEIN T, SCHONEVELD L, et al. GaussianAvatars: Photorealistic head avatars with rigged 3D Gaussians[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2024: 20299–20309. doi: 10.1109/CVPR52733.2024.01919. [103] SUN Zhiyao, LV Tian, YE Sheng, et al. DiffPoseTalk: Speech-driven stylistic 3D facial animation and head pose generation via diffusion models[J]. ACM Transactions on Graphics, 2024, 43(4): 46. doi: 10.1145/3658221. -
下载:
图(7) / 表(5)
计量
- 文章访问数: 18
- HTML全文浏览量: 3
- PDF下载量: 2
- 被引次数: 0
