面向三维集成MoE加速器的TSV动态数据映射与协同优化方法

杨佳霖; 夏晨婕; 吴慧明; 李宁远; 宋远; 刘波

doi:10.11999/JEIT260565

面向三维集成MoE加速器的TSV动态数据映射与协同优化方法

doi: 10.11999/JEIT260565 cstr: 32379.14.JEIT260565

东南大学集成电路学院南京 210096

基金项目: 国家自然科学基金(62574041)

详细信息

作者简介:
杨佳霖：男，博士生，研究方向为三维集成热管理和信号完整性

夏晨婕：女，硕士生，研究方向为三维集成TSV布局优化和容错编码l

吴慧明：男，硕士生，研究方向为三维集成的容错编码电路及传输可靠性

李宁远：男，博士生，研究方向为三维集成热管理与数据流调度

宋远：女，硕士生，研究方向为面向三维集成的大语言模型加速器设计

刘波：男，副教授，研究方向为智能计算芯片设计、三维集成等

通讯作者:
刘波　liubo_cnasic@seu.edu.cn

中图分类号: TN47
计量
- 文章访问数: 26
- HTML全文浏览量: 5
- PDF下载量: 4
- 被引次数: 0
出版历程
- 收稿日期: 2026-05-06
- 修回日期: 2026-08-28
- 录用日期: 2026-08-28
- 网络出版日期: 2026-09-03

Dynamic Data Mapping and Co-Optimization Method for TSVs in 3D-Integrated MoE Accelerators

School of Integrated Circuits, Southeast University, Nanjing 210096, China

Funds: The National Natural Science Foundation of China (62574041)

摘要

摘要: 随着混合专家(MoE)的广泛应用，基于硅通孔(TSV)的三维集成电路(3D IC)成为突破诸多瓶颈的关键技术。然而，高密度TSV阵列在高吞吐量、高并发传输场景下面临严峻的信号完整性与可靠性问题，且MoE推理任务具有动态变化的通信模式，静态TSV分配难以适配突发性大批量数据传输。针对上述问题，该文面向大规模TSV阵列的高优化复杂度提出簇规模划分方案，提升优化效率和灵活性。此外，围绕MoE专用加速器高并发数据传输场景下不同阶段数据流特征差异，提出了基于容错机制的计算阶段和负载感知的动态数据映射(PLDM)优化策略，根据不同计算阶段的特点动态调整数据映射，并依据簇级误码率和簇负载状态动态分配数据。TSV簇传输仿真实验表明，相比静态映射，动态映射使得平均带宽利用率提升约12.6%，簇负载均衡度提升2.2倍。该文随后在三层堆叠系统上部署MoE推理并进行系统验证。在系统级错误注入情况下，本方案可以将模型推理困惑度损失控制在0.02以内，带宽利用率提升约15%，簇负载均衡性提升约3.4倍。
- 三维集成电路 /
- 混合专家 /
- 硅通孔 /
- 动态数据映射 /
- 负载均衡
Abstract: Objective The rapid progress of large-scale intelligent computing, especially Mixture-of-Experts (MoE), has positioned Three-Dimensional Integrated Circuit (3D IC) based on Through-Silicon Via (TSV) as a key solution to memory-wall bottlenecks via high bandwidth and density. As a core 3D IC technology, TSVs enable vertical inter-chip connections, reducing path length, parasitic delays, power, and boosting data rates. MoE-specific accelerators, characterized by high data density and strong fault tolerance, introduce new challenges and opportunities for TSV layout. These include aggravated signal integrity and reliability issues in dense arrays, and the inadequacy of static TSV allocation for dynamic, bursty MoE traffic. Conversely, their inherent fault tolerance permits optimization design spaces for employing fault-tolerance mechanisms. This paper exploits MoE dataflow characteristics and hardware fault tolerance to devise a data mapping strategy for high-density TSV arrays based on fault-tolerance mechanisms, targeting improved performance and reliability. Methods This paper investigates cluster partitioning schemes and data mapping strategies to enhance the reliability of TSV data transmission. To address the high complexity of global optimization in large-scale TSV arrays, a cluster size partitioning scheme is proposed. By structurally partitioning a large-scale TSV array into several small-scale TSV clusters, the global optimization problem is decomposed into local, scalable subproblems, thereby improving optimization efficiency and flexibility while ensuring optimization moderation. Through comprehensive consideration of multiple metrics and simulation-based evaluation, the cluster size is finally determined to be 6×6. In response to the varying dataflow characteristics and load distribution across different computational stages, this paper proposes a Phase- and Load-Aware Dynamic Data Mapping (PLDM) strategy. The strategy pre-partitions the TSV array into multiple fixed-size clusters and classifies them into critical clusters and general clusters based on metrics such as coupling strength, bandwidth, and latency. At runtime, the PLDM strategy dynamically adjusts data mapping according to the characteristics of different computational stages. Furthermore, this paper achieves a co-optimization design of PLDM with the encoding circuit. The load monitoring module and the error monitoring module share certain data buffers and control status registers, enabling hardware resource reuse. Meanwhile, the error monitoring results provide real-time feedback on the reliability level of each TSV cluster, based on which the mapping controller preferentially allocates data transmission to clusters with lighter loads and lower bit error rates. This approach realizes resource sharing and load balancing, thereby improving data transmission reliability and link utilization efficiency for high-density TSV arrays. Results and Discussions This paper analyzes the bandwidth utilization and load balancing performance of three mapping schemes: random, static, and dynamic. The results show that both the dynamic and random mapping schemes achieve average bandwidth utilization close to the theoretical maximum. However, the random mapping scheme maps approximately 37.52% of critical data into general clusters with relatively high bit error rates, thereby increasing unreliability. Compared with static mapping, the dynamic mapping scheme improves average bandwidth utilization from 0.7982 to 0.8984, a relative increase of about 12.6%, reduces inter-cluster load fluctuation by 54.6%, and correspondingly improves load balancing by a factor of 2.2 (Fig. 4). Compared with random mapping, the dynamic mapping scheme reduces inter-cluster load fluctuation by about 8.6%, and reduces the latency of critical data and non-critical data by 36.6% and 34.7%, respectively (Table 2). To further evaluate the optimization effects of the proposed PLDM strategy on metrics such as load balancing and bandwidth utilization, four comparative schemes are configured: (1) Baseline scheme; (2) Static mapping scheme; (3) Sparse TSV layout using TSV-Aware Adaptive Fault-Tolerant Coding (TSV-AFTC) and PLDM; (4) High-density TSV layout based on scheme (3). Taking the Qwen3-30B-A3B model as an example, the normalized loads of 16 clusters in the Multi-Head Attention (MHA) and Feed-Forward Network (FFN) stages are compared across the four schemes. The results indicate that the proposed dynamic data mapping scheme achieves balanced load distribution across clusters in both the MHA and FFN stages, ranging from 0.48 to 0.52, while ensuring that all critical data are mapped to critical clusters. The high-density TSV scheme further reduces the load per cluster to approximately 0.34–0.37, demonstrating that dynamic mapping can effectively suppress stage-wise hot spots and improve load balancing (Fig. 7). Subsequently, system-level fault injection is applied to the transmitted data to simulate data reliability under extreme conditions for different schemes. The results show that for the proposed scheme (sparse), the degradation in perplexity (PPL) compared to the ideal case is controlled within 0.02, while the average bandwidth utilization is improved by approximately 15% and cluster load balancing is enhanced by a factor of 3.4. Under the high-density scheme, the PPL increase is controlled within 0.05, the average bandwidth utilization reaches about 71.7%, and the cluster load balancing is improved by a factor of 2 (Table 4). Conclusions This paper investigates TSV data mapping for 3D MoE accelerators and proposes a PLDM strategy based on TSV-AFTC, which allocates data from different computational stages to reliable and lightly loaded TSV clusters according to cluster-level bit error rates and load conditions. Through circuit co-design, approximately 12% of hardware resources can be saved. Compared with static mapping, the proposed scheme improves average bandwidth utilization by about 12.6% and enhances cluster load balancing by a factor of 2.2. Under system-level fault injection, the scheme limits the degradation of model inference perplexity to within 0.02, while achieving approximately 15% improvement in bandwidth utilization and a 3.4× enhancement in cluster load balancing.
- Three-Dimensional Integrated Circuit (3D IC) /
- Mixture-of-Experts (MoE) /
- Through-Silicon Via (TSV) /
- Dynamic data mapping /
- Load balance

HTML全文

图 1 三维集成加速器和MoE架构图

下载: 全尺寸图片幻灯片

图 2 $ m\times n $簇边界KOZ示意图

下载: 全尺寸图片幻灯片

图 3 数据映射策略框架图

下载: 全尺寸图片幻灯片

图 4 不同映射策略平均带宽利用率和负载不均衡度对比

下载: 全尺寸图片幻灯片

图 5 轻量误码监测与数据映射协同设计电路框架图

下载: 全尺寸图片幻灯片

图 6 三层堆叠的三维集成MoE加速器架构图

下载: 全尺寸图片幻灯片

图 7 Qwen3-30B-A3B下MHA和FFN阶段的簇负载分布

下载: 全尺寸图片幻灯片

表 1 不同簇规模下的关键评估指标

簇规模	中心TSV耦合电容(fF)	簇平均耦合电容(fF)	最差峰值串扰噪声(mV)	TSV单bit能耗(fJ)	单TSV平均延迟(ps)	总面积(mm²)
8	24.96	22.33	41.6	22.854	3.879	0.162
16	27.94	24.08	131.1	23.291	3.953	0.16
32	28.06	25.49	160.6	23.645	4.014	0.153
36	28.11	25.92	221.5	23.673	4.029	0.148
64	28.14	26.5	404	23.897	4.056	0.135
128	28.16	26.71	459.7	24.131	4.059	0.12

下载: 导出CSV

表 2 三种映射策略其余性能指标对比表

映射策略	关键数据延迟	非关键数据延迟	关键-通用比例(%)
随机映射	4.51	4.5	37.52
静态映射	1.28	21	0
动态映射	2.86	2.94	0

下载: 导出CSV

表 3 TSV阵列设计参数

设计参数	数值
TSV总数量	576
簇规模	6×6
簇数量	16
TSV直径(μm)	3.5
TSV氧化物厚度(μm)	0.12
TSV高度(μm)	50
TSV间距(μm)	10

下载: 导出CSV

表 4 不同方案性能对比

方案	基线方案			静态映射			本方案(稀疏)			本方案(高密度)
方案	Qw	Mix	DS	Qw	Mix	DS	Qw	Mix	DS	Qw	Mix	DS
困惑度(PPL)	7.68	7.39	6.06	7.21	6.97	5.95	6.95	6.68	5.32	6.98	6.69	5.34
平均带宽利用率(%)	52	50	48	45	43	40	68	65	62	75	72	68
簇负载均衡(归一化)	0.038	0.041	0.044	0.375	0.387	0.401	0.011	0.012	0.014	0.019	0.021	0.024

下载: 导出CSV

参考文献(21)

[1]	景川芳, 朱晓荣. 混合专家驱动的大规模异构本地电力通信网资源分配与调度算法[J]. 电子与信息学报, 2026, 48(5): 2122–2131. doi: 10.11999/JEIT251176. JING Chuanfang and ZHU Xiaorong. Routing and resource scheduling algorithm driven by mixture of experts in large-scale heterogeneous local power communication network[J]. Journal of Electronics & Information Technology, 2026, 48(5): 2122–2131. doi: 10.11999/JEIT251176.
[2]	HUANG Haochen, ZHONG Shuzhang, ZHANG Zhe, et al. HD-MoE: Hybrid and dynamic parallelism for mixture-of-expert LLMs with 3D near-memory processing[C]. Proceedings of 2025 IEEE/ACM International Conference on Computer Aided Design, Munich, Germany, 2025: 1–9. doi: 10.1109/ICCAD66269.2025.11240984.
[3]	张芊帆, 何茜, 田雨, 等. 3D IC封装技术中硅通孔研究进展综述[J]. 电子与信息学报, 2025, 47(9): 3057–3069. doi: 10.11999/JEIT250377. ZHANG Qianfan, HE Xi, TIAN Yu, et al. Review of research progress on TSV technology in 3D IC packaging[J]. Journal of Electronics & Information Technology, 2025, 47(9): 3057–3069. doi: 10.11999/JEIT250377.
[4]	HSU M K, CHANG Yaowen, and BALABANOV V. TSV-aware analytical placement for 3D IC designs[C]. Proceedings of the 48th Design Automation Conference, San Diego, USA, 2011: 664–669. doi: 10.1145/2024724.2024875.
[5]	AHMED M A and CHRZANOWSKA-JESKE M. TSV capacitance aware 3-D floorplanning[C]. Proceedings of 2013 IEEE International 3D Systems Integration Conference, San Francisco, USA, 2013: 1–6. doi: 10.1109/3DIC.2013.6702358.
[6]	XU Qi, CHEN Song, XU Xiaodong, et al. Clustered fault tolerance TSV planning for 3-D integrated circuits[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2017, 36(8): 1287–1300. doi: 10.1109/TCAD.2017.2681080.
[7]	XU Qi, SUN Wenhao, CHEN Song, et al. Cellular structure-based fault-tolerance TSV configuration in 3D-IC[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2022, 41(5): 1196–1208. doi: 10.1109/TCAD.2021.3084920.
[8]	LI Guoliang, SHAN Guangbao, MENG Baoping, et al. Intelligent multifield collaborative optimization method for TSV array with performance constraints[J]. IEEE Transactions on Electron Devices, 2023, 70(9): 4772–4778. doi: 10.1109/TED.2023.3296686.
[9]	KUMAR R and KHATRI S P. Crosstalk avoidance codes for 3D VLSI[C]. Proceedings of 2013 Design, Automation & Test in Europe Conference & Exhibition, Grenoble, France, 2013: 1673–1678. doi: 10.7873/DATE.2013.338.
[10]	CUI Xiaole, WEI Chen, FENG Xu, et al. Mosaic-3C1S: A low overhead crosstalk suppression scheme for rectangular TSV array[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2023, 42(5): 1380–1392. doi: 10.1109/TCAD.2021.3103823.
[11]	SONG T, LIU Chang, PENG Yarui, et al. Full-chip multiple TSV-to-TSV coupling extraction and optimization in 3D ICs[C]. Proceedings of the 50th Annual Design Automation Conference, Austin, USA, 2013: 180. doi: 10.1145/2463209.2488956.
[12]	KIM J, CHO J, PAK J S, et al. I/O power estimation and analysis of high-speed channels in through-silicon via (TSV)-based 3D IC[C]. Proceedings of the 19th Topical Meeting on Electrical Performance of Electronic Packaging and Systems, Austin, USA, 2010: 41–44. doi: 10.1109/EPEPS.2010.5642539.
[13]	BAMBERG L, NAJAFI A, and GARCÍA-ORTIZ A. Edge effect aware crosstalk avoidance technique for 3D integration[C]. Proceedings of the 27th International Symposium on Power and Timing Modeling, Optimization and Simulation, Thessaloniki, Greece, 2017: 1–8. doi: 10.1109/PATMOS.2017.8106994.
[14]	XIA Chenjie, WU Huiming, LIU Dengke, et al. TSV-aware hybrid fault-tolerant coding for data-aware 3D neural network accelerators[C]. Proceedings of 2025 IEEE International Conference on Integrated Circuits, Technologies and Applications, Macao, China, 2025: 226–227. doi: 10.1109/ICTA68203.2025.11329602.
[15]	WANG Zheng, JIN Boxiao, CHANG Yuming, et al. Model tells you where to merge: Adaptive KV cache merging for LLMs on long-context tasks. The Thirteenth International Conference on Learning Representations. Singapore: OpenReview. net, 2025. 1–19.
[16]	YANG An, LI Anfeng, YANG Baosong, et al. Qwen3 technical report[EB/OL]. https://arxiv.org/abs/2505.09388, 2025. (查阅网上资料,不确定本条文献类型及格式是否正确,请确认).
[17]	王泽昊, 朱振华, 谢童欣, 等. 混合专家大语言模型的系统与架构优化技术综述[J]. 电子与信息学报, 2025, 47(11): 4055–4078. doi: 10.11999/JEIT250407. WANG Zehao, ZHU Zhenhua, XIE Tongxin, et al. A survey on system and architecture optimization techniques for mixture-of-experts large language models[J]. Journal of Electronics & Information Technology, 2025, 47(11): 4055–4078. doi: 10.11999/JEIT250407.
[18]	REDOLFI A, VELENIS D, THANGARAJU S, et al. Implementation of an industry compliant, 5×50μm, via-middle TSV technology on 300mm wafers[C]. Proceedings of the 61st Electronic Components and Technology Conference, Lake Buena Vista, USA, 2011: 1384–1388. doi: 10.1109/ECTC.2011.5898692.
[19]	JIAO Binbin, QIAO Jingping, JIA Shiqi, et al. Low stress TSV arrays for high-density interconnection[J]. Engineering, 2024, 38: 201–208. doi: 10.1016/j.eng.2023.11.023.
[20]	JIANG A Q, SABLAYROLLES A, ROUX A, et al. Mixtral of experts[EB/OL]. https://arxiv.org/abs/2401.04088, 2024. (查阅网上资料,不确定本条文献类型及格式是否正确,请确认).
[21]	DeepSeek-AI. DeepSeek-V2: A strong, economical, and efficient mixture-of-experts language model[EB/OL]. https://arxiv.org/abs/2405.04434, 2024. (查阅网上资料,不确定本条文献类型及格式是否正确,请确认).