Dynamic Data Mapping and Co-Optimization Method for TSVs in 3D-Integrated MoE Accelerators

YANG Jialin; XIA Chenjie; WU Huiming; LI Ningyuan; SONG Yuan; LIU Bo

doi:10.11999/JEIT260565

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

YANG Jialin, XIA Chenjie, WU Huiming, LI Ningyuan, SONG Yuan, LIU Bo. Dynamic Data Mapping and Co-Optimization Method for TSVs in 3D-Integrated MoE Accelerators[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260565

Citation:

YANG Jialin, XIA Chenjie, WU Huiming, LI Ningyuan, SONG Yuan, LIU Bo. Dynamic Data Mapping and Co-Optimization Method for TSVs in 3D-Integrated MoE Accelerators[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260565

Citation:

PDF( 2124 KB)

Dynamic Data Mapping and Co-Optimization Method for TSVs in 3D-Integrated MoE Accelerators

doi: 10.11999/JEIT260565 cstr: 32379.14.JEIT260565

School of Integrated Circuits, Southeast University, Nanjing 210096, China

Funds: The National Natural Science Foundation of China (62574041)

Received Date: 2026-05-06
Accepted Date: 2026-08-28
Rev Recd Date: 2026-08-28

Available Online: 2026-09-03

Abstract

Abstract

Objective The rapid progress of large-scale intelligent computing, especially Mixture-of-Experts (MoE), has positioned Three-Dimensional Integrated Circuit (3D IC) based on Through-Silicon Via (TSV) as a key solution to memory-wall bottlenecks via high bandwidth and density. As a core 3D IC technology, TSVs enable vertical inter-chip connections, reducing path length, parasitic delays, power, and boosting data rates. MoE-specific accelerators, characterized by high data density and strong fault tolerance, introduce new challenges and opportunities for TSV layout. These include aggravated signal integrity and reliability issues in dense arrays, and the inadequacy of static TSV allocation for dynamic, bursty MoE traffic. Conversely, their inherent fault tolerance permits optimization design spaces for employing fault-tolerance mechanisms. This paper exploits MoE dataflow characteristics and hardware fault tolerance to devise a data mapping strategy for high-density TSV arrays based on fault-tolerance mechanisms, targeting improved performance and reliability. Methods This paper investigates cluster partitioning schemes and data mapping strategies to enhance the reliability of TSV data transmission. To address the high complexity of global optimization in large-scale TSV arrays, a cluster size partitioning scheme is proposed. By structurally partitioning a large-scale TSV array into several small-scale TSV clusters, the global optimization problem is decomposed into local, scalable subproblems, thereby improving optimization efficiency and flexibility while ensuring optimization moderation. Through comprehensive consideration of multiple metrics and simulation-based evaluation, the cluster size is finally determined to be 6×6. In response to the varying dataflow characteristics and load distribution across different computational stages, this paper proposes a Phase- and Load-Aware Dynamic Data Mapping (PLDM) strategy. The strategy pre-partitions the TSV array into multiple fixed-size clusters and classifies them into critical clusters and general clusters based on metrics such as coupling strength, bandwidth, and latency. At runtime, the PLDM strategy dynamically adjusts data mapping according to the characteristics of different computational stages. Furthermore, this paper achieves a co-optimization design of PLDM with the encoding circuit. The load monitoring module and the error monitoring module share certain data buffers and control status registers, enabling hardware resource reuse. Meanwhile, the error monitoring results provide real-time feedback on the reliability level of each TSV cluster, based on which the mapping controller preferentially allocates data transmission to clusters with lighter loads and lower bit error rates. This approach realizes resource sharing and load balancing, thereby improving data transmission reliability and link utilization efficiency for high-density TSV arrays. Results and Discussions This paper analyzes the bandwidth utilization and load balancing performance of three mapping schemes: random, static, and dynamic. The results show that both the dynamic and random mapping schemes achieve average bandwidth utilization close to the theoretical maximum. However, the random mapping scheme maps approximately 37.52% of critical data into general clusters with relatively high bit error rates, thereby increasing unreliability. Compared with static mapping, the dynamic mapping scheme improves average bandwidth utilization from 0.7982 to 0.8984, a relative increase of about 12.6%, reduces inter-cluster load fluctuation by 54.6%, and correspondingly improves load balancing by a factor of 2.2 (Fig. 4). Compared with random mapping, the dynamic mapping scheme reduces inter-cluster load fluctuation by about 8.6%, and reduces the latency of critical data and non-critical data by 36.6% and 34.7%, respectively (Table 2). To further evaluate the optimization effects of the proposed PLDM strategy on metrics such as load balancing and bandwidth utilization, four comparative schemes are configured: (1) Baseline scheme; (2) Static mapping scheme; (3) Sparse TSV layout using TSV-Aware Adaptive Fault-Tolerant Coding (TSV-AFTC) and PLDM; (4) High-density TSV layout based on scheme (3). Taking the Qwen3-30B-A3B model as an example, the normalized loads of 16 clusters in the Multi-Head Attention (MHA) and Feed-Forward Network (FFN) stages are compared across the four schemes. The results indicate that the proposed dynamic data mapping scheme achieves balanced load distribution across clusters in both the MHA and FFN stages, ranging from 0.48 to 0.52, while ensuring that all critical data are mapped to critical clusters. The high-density TSV scheme further reduces the load per cluster to approximately 0.34–0.37, demonstrating that dynamic mapping can effectively suppress stage-wise hot spots and improve load balancing (Fig. 7). Subsequently, system-level fault injection is applied to the transmitted data to simulate data reliability under extreme conditions for different schemes. The results show that for the proposed scheme (sparse), the degradation in perplexity (PPL) compared to the ideal case is controlled within 0.02, while the average bandwidth utilization is improved by approximately 15% and cluster load balancing is enhanced by a factor of 3.4. Under the high-density scheme, the PPL increase is controlled within 0.05, the average bandwidth utilization reaches about 71.7%, and the cluster load balancing is improved by a factor of 2 (Table 4). Conclusions This paper investigates TSV data mapping for 3D MoE accelerators and proposes a PLDM strategy based on TSV-AFTC, which allocates data from different computational stages to reliable and lightly loaded TSV clusters according to cluster-level bit error rates and load conditions. Through circuit co-design, approximately 12% of hardware resources can be saved. Compared with static mapping, the proposed scheme improves average bandwidth utilization by about 12.6% and enhances cluster load balancing by a factor of 2.2. Under system-level fault injection, the scheme limits the degradation of model inference perplexity to within 0.02, while achieving approximately 15% improvement in bandwidth utilization and a 3.4× enhancement in cluster load balancing.
- Three-Dimensional Integrated Circuit (3D IC),
- Mixture-of-Experts (MoE),
- Through-Silicon Via (TSV),
- Dynamic data mapping,
- Load balance

FullText(HTML)

References(21)

References

[1]	景川芳, 朱晓荣. 混合专家驱动的大规模异构本地电力通信网资源分配与调度算法[J]. 电子与信息学报, 2026, 48(5): 2122–2131. doi: 10.11999/JEIT251176. JING Chuanfang and ZHU Xiaorong. Routing and resource scheduling algorithm driven by mixture of experts in large-scale heterogeneous local power communication network[J]. Journal of Electronics & Information Technology, 2026, 48(5): 2122–2131. doi: 10.11999/JEIT251176.
[2]	HUANG Haochen, ZHONG Shuzhang, ZHANG Zhe, et al. HD-MoE: Hybrid and dynamic parallelism for mixture-of-expert LLMs with 3D near-memory processing[C]. Proceedings of 2025 IEEE/ACM International Conference on Computer Aided Design, Munich, Germany, 2025: 1–9. doi: 10.1109/ICCAD66269.2025.11240984.
[3]	张芊帆, 何茜, 田雨, 等. 3D IC封装技术中硅通孔研究进展综述[J]. 电子与信息学报, 2025, 47(9): 3057–3069. doi: 10.11999/JEIT250377. ZHANG Qianfan, HE Xi, TIAN Yu, et al. Review of research progress on TSV technology in 3D IC packaging[J]. Journal of Electronics & Information Technology, 2025, 47(9): 3057–3069. doi: 10.11999/JEIT250377.
[4]	HSU M K, CHANG Yaowen, and BALABANOV V. TSV-aware analytical placement for 3D IC designs[C]. Proceedings of the 48th Design Automation Conference, San Diego, USA, 2011: 664–669. doi: 10.1145/2024724.2024875.
[5]	AHMED M A and CHRZANOWSKA-JESKE M. TSV capacitance aware 3-D floorplanning[C]. Proceedings of 2013 IEEE International 3D Systems Integration Conference, San Francisco, USA, 2013: 1–6. doi: 10.1109/3DIC.2013.6702358.
[6]	XU Qi, CHEN Song, XU Xiaodong, et al. Clustered fault tolerance TSV planning for 3-D integrated circuits[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2017, 36(8): 1287–1300. doi: 10.1109/TCAD.2017.2681080.
[7]	XU Qi, SUN Wenhao, CHEN Song, et al. Cellular structure-based fault-tolerance TSV configuration in 3D-IC[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2022, 41(5): 1196–1208. doi: 10.1109/TCAD.2021.3084920.
[8]	LI Guoliang, SHAN Guangbao, MENG Baoping, et al. Intelligent multifield collaborative optimization method for TSV array with performance constraints[J]. IEEE Transactions on Electron Devices, 2023, 70(9): 4772–4778. doi: 10.1109/TED.2023.3296686.
[9]	KUMAR R and KHATRI S P. Crosstalk avoidance codes for 3D VLSI[C]. Proceedings of 2013 Design, Automation & Test in Europe Conference & Exhibition, Grenoble, France, 2013: 1673–1678. doi: 10.7873/DATE.2013.338.
[10]	CUI Xiaole, WEI Chen, FENG Xu, et al. Mosaic-3C1S: A low overhead crosstalk suppression scheme for rectangular TSV array[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2023, 42(5): 1380–1392. doi: 10.1109/TCAD.2021.3103823.
[11]	SONG T, LIU Chang, PENG Yarui, et al. Full-chip multiple TSV-to-TSV coupling extraction and optimization in 3D ICs[C]. Proceedings of the 50th Annual Design Automation Conference, Austin, USA, 2013: 180. doi: 10.1145/2463209.2488956.
[12]	KIM J, CHO J, PAK J S, et al. I/O power estimation and analysis of high-speed channels in through-silicon via (TSV)-based 3D IC[C]. Proceedings of the 19th Topical Meeting on Electrical Performance of Electronic Packaging and Systems, Austin, USA, 2010: 41–44. doi: 10.1109/EPEPS.2010.5642539.
[13]	BAMBERG L, NAJAFI A, and GARCÍA-ORTIZ A. Edge effect aware crosstalk avoidance technique for 3D integration[C]. Proceedings of the 27th International Symposium on Power and Timing Modeling, Optimization and Simulation, Thessaloniki, Greece, 2017: 1–8. doi: 10.1109/PATMOS.2017.8106994.
[14]	XIA Chenjie, WU Huiming, LIU Dengke, et al. TSV-aware hybrid fault-tolerant coding for data-aware 3D neural network accelerators[C]. Proceedings of 2025 IEEE International Conference on Integrated Circuits, Technologies and Applications, Macao, China, 2025: 226–227. doi: 10.1109/ICTA68203.2025.11329602.
[15]	WANG Zheng, JIN Boxiao, CHANG Yuming, et al. Model tells you where to merge: Adaptive KV cache merging for LLMs on long-context tasks. The Thirteenth International Conference on Learning Representations. Singapore: OpenReview. net, 2025. 1–19.
[16]	YANG An, LI Anfeng, YANG Baosong, et al. Qwen3 technical report[EB/OL]. https://arxiv.org/abs/2505.09388, 2025. (查阅网上资料,不确定本条文献类型及格式是否正确,请确认).
[17]	王泽昊, 朱振华, 谢童欣, 等. 混合专家大语言模型的系统与架构优化技术综述[J]. 电子与信息学报, 2025, 47(11): 4055–4078. doi: 10.11999/JEIT250407. WANG Zehao, ZHU Zhenhua, XIE Tongxin, et al. A survey on system and architecture optimization techniques for mixture-of-experts large language models[J]. Journal of Electronics & Information Technology, 2025, 47(11): 4055–4078. doi: 10.11999/JEIT250407.
[18]	REDOLFI A, VELENIS D, THANGARAJU S, et al. Implementation of an industry compliant, 5×50μm, via-middle TSV technology on 300mm wafers[C]. Proceedings of the 61st Electronic Components and Technology Conference, Lake Buena Vista, USA, 2011: 1384–1388. doi: 10.1109/ECTC.2011.5898692.
[19]	JIAO Binbin, QIAO Jingping, JIA Shiqi, et al. Low stress TSV arrays for high-density interconnection[J]. Engineering, 2024, 38: 201–208. doi: 10.1016/j.eng.2023.11.023.
[20]	JIANG A Q, SABLAYROLLES A, ROUX A, et al. Mixtral of experts[EB/OL]. https://arxiv.org/abs/2401.04088, 2024. (查阅网上资料,不确定本条文献类型及格式是否正确,请确认).
[21]	DeepSeek-AI. DeepSeek-V2: A strong, economical, and efficient mixture-of-experts language model[EB/OL]. https://arxiv.org/abs/2405.04434, 2024. (查阅网上资料,不确定本条文献类型及格式是否正确,请确认).