Optimized Design of Non-Transparent Bridge for Heterogeneous Interconnects in Hyper-converged Infrastructure

ZHENG Rui; SHEN Jianliang; LV Ping; DONG Chunlei; SHAO Yu; ZHU Zhengbin

doi:10.11999/JEIT250272

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

ZHENG Rui, SHEN Jianliang, LV Ping, DONG Chunlei, SHAO Yu, ZHU Zhengbin. Optimized Design of Non-Transparent Bridge for Heterogeneous Interconnects in Hyper-converged Infrastructure[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250272

Citation:

ZHENG Rui, SHEN Jianliang, LV Ping, DONG Chunlei, SHAO Yu, ZHU Zhengbin. Optimized Design of Non-Transparent Bridge for Heterogeneous Interconnects in Hyper-converged Infrastructure[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250272

Citation:

ZHENG Rui, SHEN Jianliang, LV Ping, DONG Chunlei, SHAO Yu, ZHU Zhengbin. Optimized Design of Non-Transparent Bridge for Heterogeneous Interconnects in Hyper-converged Infrastructure[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250272

PDF( 5814 KB)

Optimized Design of Non-Transparent Bridge for Heterogeneous Interconnects in Hyper-converged Infrastructure

doi: 10.11999/JEIT250272 cstr: 32379.14.JEIT250272

1.
Institute of Information Technology, Information Engineering University, Zhengzhou 450001, China
2.
Armed Police Command College, Tianjin 300250, China

Funds: The National Key Research and Development Program of China (2022YFB2901000)

Received Date: 2025-04-14
Rev Recd Date: 2025-08-01

Available Online: 2025-08-11

Abstract

Abstract

Objective The integration of heterogeneous computing resource clusters into modern Hyper-Converged Infrastructure (HCI) systems imposes stricter performance requirements in latency, bandwidth, throughput, and cross-domain transmission stability. Traditional HCI systems primarily rely on the Ethernet TCP/IP protocol, which exhibits inherent limitations, including low bandwidth efficiency, high latency, and limited throughput. Existing PCIe Switch products typically employ Non-Transparent Bridges (NTBs) for conventional dual-system connections or intra-server communication; however, they do not meet the performance demands of heterogeneous cross-domain transmission within HCI environments. To address this limitation, a novel Dual-Mode Non-Transparent Bridge Architecture (D-MNTBA) is proposed to support dual transmission modes. D-MNTBA combines a fast transmission mode via a bypass mechanism with a stable transmission mode derived from the Traditional Data Path Architecture (TDPA), thereby aligning with the data characteristics and cross-domain streaming demands of HCI systems. Hardware-level enhancements in address and ID translation schemes enable D-MNTBA to support more complex mappings while minimizing translation latency. These improvements increase system stability and effectively support the cross-domain transmission of heterogeneous data in HCI systems. Methods To overcome the limitations of traditional single-pass architectures and the bypass optimizations of the TDPA, the proposed D-MNTBA incorporates both a fast transmission path and a stable transmission path. This dual-mode design enables the NTB to leverage the data characteristics of HCI systems for telegram-based streaming, thereby reducing dependence on intermediate protocols and data format conversions. The stable transmission mode ensures reliable message delivery, while the fast transmission mode—enhanced through hardware-level optimizations in address and ID translation—supports high-real-time cross-domain communication. This combination improves overall transmission performance by reducing both latency and system overhead. To meet the low-latency demands of the bypass transmission path, the architecture implements hardware-level enhancements to the address and ID conversion modules. The address translation module is expanded with a larger lookup table, allowing for more complex and flexible mapping schemes. This enhancement enables efficient utilization of non-contiguous and fragmented address spaces without compromising performance. Simultaneously, the ID conversion module is optimized through multiple conversion strategies and streamlined logic, significantly reducing the time required for ID translation. Results and Discussions Address translation in the proposed D-MNTBA is validated through emulation within a constructed HCI environment. The simulation log for indirect address translation shows no errors or deadlocks, and successful hits are observed on BAR2/3. During dual-host disk access, packet header addresses and payload content remain consistent, with no packet loss detected (Fig. 14), indicating that indirect address translation is accurately executed under D-MNTBA. ID conversion performance is evaluated by comparing the proposed architecture with the TDPA implemented in the PEX8748 chip. The switch based on D-MNTBA exhibits significantly shorter ID conversion times. A maximum reduction of approximately 34.9% is recorded, with an ID conversion time of 71 ns for a 512-byte payload (Fig. 15). These findings suggest that the ID function mapping method adopted in D-MNTBA effectively reduces conversion latency and enhances system performance. Throughput stability is assessed under sustained heavy traffic with payloads ranging from 256 to 2048 bytes. The maximum throughputs of D-MNTBA, the Ethernet card, and PEX8748 are measured at 1.36 GB/s, 0.97 GB/s, and 0.9 GB/s, respectively (Fig. 16). Compared to PEX8748 and the Ethernet architecture, D-MNTBA improves throughput by approximately 51.1% and 40.2%, respectively, and shows the slowest degradation trend, reflecting superior stability in heterogeneous cross-domain transmission. Bandwidth comparison reveals that D-MNTBA outperforms TDPA and the Ethernet card, with bandwidth improvements of approximately 27.1% and 19.0%, respectively (Fig. 17). These results highlight the significant enhancement in cross-domain transmission performance achieved by the proposed architecture in heterogeneous environments. Conclusions This study proposes a Dual-Mode D-MNTBA to address the challenges of heterogeneous interconnection in HCI systems. By integrating a fast transmission path enabled by a bypass architecture with the stable transmission path of the TDPA, D-MNTBA accommodates the specific data characteristics of cross-domain transmission in heterogeneous environments and enables efficient message routing. D-MNTBA enhances transmission stability while improving system-wide performance, offering robust support for high-real-time cross-domain transmission in HCI. It also reduces latency and overhead, thereby improving overall transmission efficiency. Compared with existing transmission schemes, D-MNTBA achieves notable gains in performance, making it a suitable solution for the demands of heterogeneous domain interconnects in HCI systems. However, the architectural enhancements, particularly the bypass design and associated optimizations, increase logic resource utilization and power consumption. Future work should focus on refining hardware design, layout, and wiring strategies to reduce logic complexity and resource consumption without compromising performance.
- Hyper-Converged Infrastructure (HCI),
- Heterogeneous interconnection,
- PCIe Non-Transparent Bridge (NTB),
- Cross-domain transmission,
- Performance optimization

FullText(HTML)

References(24)

References

[1]	张桉齐, 张利彬. 基于跨机内存映射的高速互联机制[J]. 指挥控制与仿真, 2025, 47(3): 109–115. doi: 10.3969/j.issn.1673-3819.2025.03.015. ZHANG Anqi and ZHANG Libin. A high-speed interconnection mechanism based on multi-hosts memory mapping[J]. Command Control & Simulation, 2025, 47(3): 109–115. doi: 10.3969/j.issn.1673-3819.2025.03.015.
[2]	王浩, 王勇, 冯长磊, 等. Chiplet互联技术综述[J/OL]. 计算机研究与发展, 1–13. https://link.cnki.net/urlid/11.1777.TP.20250304.1704.002, 2025. WANG Hao, WANG Yong, FENG Changlei, et al. Review of Chiplet interconnection technology[J/OL]. Journal of Computer Research and Development, 1–13. http://kns.cnki.net/kcms/detail/11.1777.TP.2040.20304.1704.002.html, 2025.
[3]	DASARRAJU V K. Implementing and managing hyper-converged infrastructure with VMware vSAN and azure stack HCI[J]. Journal of Artificial Intelligence, Machine Learning and Data Science, 2024, 2(2): 666–670. doi: 10.51219/JAIMLD/reddy-davu/170.
[4]	YANG Yang, JIA Junjun, and LV Hua. Resources optimization research of service oriented architecture based on hyper converged infrastructure cloud computing platform[M]. CHEN Chihua, SCAPELLATO A, BARBIERO A, et al. Applied Mathematics, Modeling and Computer Simulation. Amsterdam: IOS Press, 2023: 78–84. doi: 10.3233/ATDE230942.
[5]	SILVA-ATENCIO G and UMAÑA-RAMÍREZ M. The evolution and trends of hyperconvergence in the telecommunications sector: A competitive intelligence review[J]. Dyna, 2023, 90(227): 126–132. doi: 10.15446/dyna.v90n227.107360.
[6]	USHAKOV Y, USHAKOVA M, and LEGASHEV L. Research of a virtual infrastructure network with hybrid software-defined switching[J]. Engineering Proceedings, 2023, 33(1): 52. doi: 10.3390/engproc2023033052.
[7]	谭政源, 钟佳卿, 陈娟. 等. AI+HPC: “智能+”驱动下的超算系统软件及应用技术发展综述[J]. 计算机科学, 2025, 52(5): 1–10. doi: 10.11896/jsjkx.241100177. TAN Zhengyuan, ZHONG Jiaqing, CHEN Juan, et al. AI+HPC: An overview of supercomputing system software and application technology development driven by “AI+”[J]. Computer Science, 2025, 52(5): 1–10. doi: 10.11896/jsjkx.241100177.
[8]	TSUNG C K, LIU J C, LIU S H, et al. Performance analysis in HyperFlex and vSAN hyper convergence platforms for online course consideration[J]. IEEE Access, 2022, 10: 124464–124474. doi: 10.1109/ACCESS.2022.3224435.
[9]	NISA N, KHAN A S, AHMAD Z, et al. TPAAD: Two‐phase authentication system for denial of service attack detection and mitigation using machine learning in software‐defined network[J]. International Journal of Network Management, 2024, 34(3): e2258. doi: 10.1002/nem.2258.
[10]	MA Xiaohan, WANG Ying, WANG Yuji, et al. Survey on chiplets: Interface, interconnect and integration methodology[J]. CCF Transactions on High Performance Computing, 2022, 4(1): 43–52. doi: 10.1007/s42514-022-00093-0.
[11]	NOAMAN M, KHAN M S, ABRAR M F, et al. Challenges in integration of heterogeneous internet of things[J]. Scientific Programming, 2022, 2022(1): 8626882. doi: 10.1155/2022/8626882.
[12]	LAU J H. Recent advances and trends in multiple system and heterogeneous integration with TSV-less interposers[J]. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2022, 12(8): 1271–1281. doi: 10.1109/TCPMT.2022.3194374.
[13]	YIN Jinyong, WANG Honghui, and XU Zhenpeng. A reconfigurable rack-scale interconnect architecture based on PCIe fabric[C]. 2022 IEEE 2nd International Conference on Data Science and Computer Application (ICDSCA), Dalian, China, 2022: 306–310. doi: 10.1109/ICDSCA56264.2022.9988411.
[14]	HOSSEINABADY M, ZAINOL M A B, and NUNEZ-YANEZ J. Heterogeneous FPGA+GPU embedded systems: Challenges and opportunities[EB/OL]. https://arxiv.org/abs/1901.06331, 2019.
[15]	ZHAO Xujie and ZHI Xiaoli. A PCIe-based inter-processor messaging architecture[C]. 2024 4th International Conference on Electronic Information Engineering and Computer (EIECT), IEEE, Shenzhen, China, 2024: 548–553. doi: 10.1109/EIECT64462.2024.10866443.
[16]	HUANG Yibo, HUANG Yukai, YAN Ming, et al. An ultra-low latency and compatible PCIe interconnect for rack-scale communication[C]. The 18th International Conference on emerging Networking EXperiments and Technologies, Roma, Italy, 2022: 232–244. doi: 10.1145/3555050.3569128.
[17]	刘佳兴, 胡怀湘. 非透明桥在国产化平台上的应用[J]. 信息技术, 2020, 44(11): 103–107. doi: 10.13274/j.cnki.hdzj.2020.11.020. LIU Jiaxing and HU Huaixiang. Application of non-transparent bridge on localized platform[J]. Information Technology, 2020, 44(11): 103–107. doi: 10.13274/j.cnki.hdzj.2020.11.020.
[18]	徐健, 张建泉, 张健. 基于PCIE非透明桥的嵌入式异构平台设计[J]. 微电子学与计算机, 2018, 35(1): 26–30. doi: 10.19304/j.cnki.issn1000-7180.2018.01.006. XU Jian, ZHANG Jianquan and ZHANG Jian. An application of PCIE non-transparent bridge on a heterogeneous platform[J]. Microelectronics & Computer, 2018, 35(1): 26–30. doi: 10.19304/j.cnki.issn1000-7180.2018.01.006.
[19]	李沛杰, 刘勤让, 陈艇, 等. 异构集成互连接口研究综述[J]. 集成电路与嵌入式系统, 2024, 24(2): 31–40. LI Peijie, LIU Qinrang, CHEN Ting, et al. Research on the heterogeneous integrated interconnect interface[J]. Integrated Circuits and Embedded Systems, 2024, 24(2): 31–40.
[20]	HOU Wentao, ZHANG Jie, WANG Zeke, et al. Understanding routable PCIe performance for composable infrastructures[C]. The 21st USENIX Symposium on Networked Systems Design and Implementation. Santa Clara, USA, 2024: 271–312.
[21]	LIM S H and CHA K. Shared memory model over a switchless PCIe NTB interconnect network[J]. Journal of Information Processing Systems, 2022, 18(1): 159–172. doi: 10.3745/JIPS.01.0085.
[22]	任晓旭, 谭靖超, 邓辉, 等. 基于端边云超融合的算力网络架构[J]. 计算机应用, 2022, 42(S1): 195–200. doi: 10.11772/j.issn.1001-9081.2021081491. REN Xiaoxu, TAN Jingchao, DENG Hui, et al. Arithmetic network architecture based on end-side cloud hyperconvergence[J]. Journal of Computer Application, 2022, 42(S1): 195–200. doi: 10.11772/j.issn.1001-9081.2021081491.
[23]	NABLI H, BEN DJEMAA R, and AMOUS BEN AMOR I. Description, discovery, and recommendation of Cloud services: A survey[J]. Service Oriented Computing and Applications, 2022, 16(3): 147–166. doi: 10.1007/s11761-022-00343-7.
[24]	卢宇彤, 陈志广. 基于超算的多模式计算融合支撑系统[J]. 中山大学学报(自然科学版中英文), 2024, 63(6): 150–160. doi: 10.13471/j.cnki.acta.snus.ZR20240293. LU Yutong and CHEN Zhiguang. The convergent computing based on supercomputer[J]. Acta Scientiarum Naturalium Universitatis Sunyatseni, 2024, 63(6): 150–160. doi: 10.13471/j.cnki.acta.snus.ZR20240293.