A Structure-preserving Semantic Transmission Method for Low-bandwidth Networks
-
摘要: 针对极低带宽条件下传统信源编码在低码率场景中易出现“数字悬崖”效应，且现有语义通信方法难以同时兼顾结构保真、色彩一致性与低时延的问题，该文提出一种面向低带宽网络的结构保真语义传输方法。该方法构建语义-结构-外观多模态解耦表示框架，通过引入对象感知的语义引导编码机制，实现关键区域驱动的自适应码率分配；同时设计确定性双流生成重建模型，以结构约束与语义调制协同引导图像重建，从而降低推理复杂度与时延。实验在Set14, DIV2K数据集及基于通用软件无线电平台(USRP)的LoRa实际物理链路环境下开展，结果表明，在约0.3 bpp的极低码率条件下，该方法可实现低于60 ms的端到端推理时延，并在主体结构一致性与视觉自然度方面优于传统编码方法。研究结果表明，所提方法在受限带宽条件下能够有效提升传输效率与重建质量，适用于高可靠低时延的边缘视觉通信场景。Abstract:
Objective Low-bandwidth visual transmission is essential for edge intelligence applications such as disaster inspection, underwater exploration, and remote assistance. These scenarios require visual communication systems to provide a low bit rate, high structural fidelity, stable color reconstruction, and low inference latency. However, conventional image coding methods suffer severe quality degradation at extremely low bit rates because of the digital cliff effect and compression artifacts. Semantic communication offers a promising solution by transmitting task-relevant representations rather than pixel-level information. However, existing methods still struggle to balance transmission efficiency, reconstruction fidelity, and decoding complexity. Therefore, a structure-preserving semantic transmission method termed Edge-Link is proposed for low-bandwidth networks. Methods Edge-Link adopts a multimodal decoupled representation framework that separates image information into three complementary streams: a semantic stream, a structural stream, and a low-frequency appearance stream. The semantic stream is extracted using a frozen Contrastive Language-Image Pre-training (CLIP) encoder to provide global semantic guidance. The structural stream is derived from Canny edge information to explicitly preserve object boundaries. A low-resolution color map is used as the low-frequency appearance stream to preserve the global color distribution and illumination characteristics with minimal transmission overhead. Furthermore, a Fast Segment Anything Model (FastSAM)-based semantic gateway is used to distinguish foreground objects from background regions. An object-aware bit-rate allocation strategy is then applied to prioritize semantically important regions under bandwidth constraints. At the receiver, a deterministic dual-stream generative network based on SPatially ADaptive normalization (SPADE) is used for image reconstruction. Structural information provides spatial constraints, whereas semantic features guide image generation. This design avoids the high latency associated with iterative diffusion sampling. A LoRa communication prototype based on GNU Radio and a Universal Software Radio Peripheral (USRP) is also implemented to evaluate transmission feasibility and robustness under physical wireless-channel conditions. Results and Discussions Experiments are conducted on Set14 and DIV2K to evaluate transmission efficiency, reconstruction quality, perceptual naturalness, and latency. For 512 × 512 images, the proposed method achieves an average payload of 10.96 KB, corresponding to approximately 0.31 bits per pixel (bpp), and the end-to-end inference latency remains below 60 ms. Over a LoRa narrowband link of approximately 8 kbps, the airtime latency per frame is approximately 11 s, confirming the feasibility of the method under bandwidth-limited conditions ( Table 1 ). The ablation study shows that the object-aware bit-rate allocation strategy improves the trade-off between bit rate and reconstruction quality. On DIV2K, the average payload is reduced from 126.09 KB to 93.55 KB while better reconstruction quality is maintained in semantically important regions (Table 2 ). Compared with JPEG, the proposed method also provides better structure preservation and perceptual quality at low bit rates. For image 0843 with a payload of 15.54 KB, the overall Structural Similarity Index Measure (SSIM) increases from 0.783 to 0.963, whereas the Learned Perceptual Image Patch Similarity (LPIPS) for the subject region of interest (Subject-ROI) decreases from 0.561 to 0.307 (Table 4 ). Under the same bit-rate constraint, the proposed method further outperforms VQGAN and ControlNet+Canny. It achieves a Peak Signal-to-Noise Ratio (PSNR) of 23.738, an SSIM of 0.622, an LPIPS of 0.223, and a Natural Image Quality Evaluator (NIQE) value of 4.570. These results indicate a better balance between reconstruction fidelity and perceptual quality in low-bandwidth semantic image reconstruction (Table 5 ).Conclusions A structure-preserving semantic transmission framework for low-bandwidth networks is presented. By combining multimodal decoupled representation, object-aware bit-rate allocation, and deterministic dual-stream generative reconstruction, the framework balances transmission efficiency, structural fidelity, perceptual quality, and inference latency. The results show that the proposed method is suitable for high-reliability edge visual communication scenarios that require accurate object contours, stable color appearance, and efficient inference. Validation using the LoRa prototype further supports its practical applicability to bandwidth-constrained edge networks. -
表 1 系统端到端性能评估结果
指标 实测值 备注 平均载荷 10.96 kB Set14平均结果 算法推理时延 <60 ms RTX 3090(编码&解码) 空口传输时延 ≈11 s LoRa
SF=7(约8 kbit/(s·Hz))平均带宽效率 0.31 bpp 10.96 kB/5122 
下载: 导出CSV
表 2 DIV2K 数据集对比结果
策略 平均载荷(KB) 全局SSIM LPIPS 备注 Global 126.09 0.6350 0.3328 无 Object-Aware 93.55 0.6385 全图：0.380主体：0.223 带宽节约26%
下载: 导出CSV
表 3 Edge-Link与传统编码方法在典型图像上的对比结果
图像 载荷(KB) 方法 SSIM LPIPS NIQE Face 3.63 Ours 0.633 0.393 8.77 Face 3.63 JPEG 0.442 0.659 21.67 Face 3.63 WebP 0.709 0.373 7.81 Foreman 7.64 Ours 0.828 0.223 5.72 Foreman 7.64 JPEG 0.802 0.334 12.97 Foreman 7.64 WebP 0.925 0.083 6.51 Bridge 8.60 Ours 0.388 0.505 4.26 Bridge 8.60 JPEG 0.596 0.348 7.47 Bridge 8.60 WebP 0.650 0.365 4.90 Average 10.96 Ours 0.716 0.236 5.52 Average 10.96 JPEG 0.696 0.304 8.794 注：NIQE指标越低代表图像越自然。本方法只在背景信息较多的图片中数据表现优于WebP，对于平均数据而言 ，本方法并不具有优势，故未展示。
下载: 导出CSV
表 4 部分DIV2K 高分辨率图像局部细节重建质量对比
图片 载荷(KB) 方法 整体LPIPS Subject-ROI
LPIPS整体SSIM 整体NIQE Subject-ROI
NIQE备注 0843 15.54 本文 0.059 0.307 0.963 8.13 9.56 极简场景 15.54 JPEG 0.112 0.561 0.783 11.68 10.94 极简场景 15.54 WebP 0.093 0.114 0.970 8.79 7.86 极简场景 0877 36.71 本文 0.201 0.226 0.874 6.33 8.30 丰富纹理 36.71 JPEG 0.402 0.416 0.791 16.19 21.95 丰富纹理 36.71 WebP 0.065 0.139 0.924 6.97 9.31 丰富纹理 平均 ～93 本文 0.371 0.223 0.622 4.57 7.80 平均数据 ～93 JPEG 0.222 0.305 0.739 5.530 10.122 平均数据 注：Dataset Avg.中的 Subject-ROI LPIPS为全集主体区域测量均值，用于评估语义前景区域的重建。对于WebP，本方法在平均数据上并不具有优势，故未展示。
下载: 导出CSV
表 5 不同视觉重建方法的性能比较
方法 PSNR SSIM LPIPS NIQE 本文 23.738 0.622 0.223 4.570 VQGAN 21.965 0.588 0.220 6.026 ControlNet+Canny 9.618 0.189 0.731 10.071
下载: 导出CSV
表 6 多模态解耦消融实验结果(Set14数据集)
实验配置 整体SSIM 整体LPIPS 主体SSIM 主体LPIPS 完整模型 0.572 0.358 0.716 0.236 去除结构流 0.510 0.522 0.671 0.343 去除语义流 0.537 0.420 0.693 0.278 去除外观流 0.389 0.485 0.464 0.425
下载: 导出CSV
-
[1] JERNBERG C, SANDIN J, ZIEMKE T, et al. The effect of latency, speed and task on remote operation of vehicles[J]. Transportation Research Interdisciplinary Perspectives, 2024, 26: 101152. doi: 10.1016/j.trip.2024.101152. [2] BOURTSOULATZE E, KURKA D B, and GÜNDÜZ D. Deep joint source-channel coding for wireless image transmission[J]. IEEE Transactions on Cognitive Communications and Networking, 2019, 5(3): 567–579. doi: 10.1109/TCCN.2019.2919300. [3] DONG Chao, DENG Yubin, LOY C C, et al. Compression artifacts reduction by a deep convolutional network[C]. The 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 2015: 576–584. doi: 10.1109/ICCV.2015.73. [4] LIU Yating, WANG Xiaojie, NING Zhaolong, et al. A survey on semantic communications: Technologies, solutions, applications and challenges[J]. Digital Communications and Networks, 2024, 10(3): 528–545. doi: 10.1016/j.dcan.2023.05.010. [5] CHAI Jingxuan, XIAO Yong, and SHI Guangming. On the rate-distortion-complexity tradeoff for semantic communication[J]. IEEE Internet of Things Journal, 2026, 13(14): 31768–31781. doi: 10.1109/JIOT.2026.3689652. [6] 陈阳, 马欢, 姬智, 等. 面向图像恢复任务的语义通信网络能耗优化[J]. 电子与信息学报, 2026, 48(1): 183–190. doi: 10.11999/JEIT250915.CHEN Yang, MA Huan, JI Zhi, et al. Optimization of energy consumption in semantic communication networks for image recovery tasks[J]. Journal of Electronics & Information Technology, 2026, 48(1): 183–190. doi: 10.11999/JEIT250915. [7] AITHAL S K, MAINI P, LIPTON Z, et al. Understanding hallucinations in diffusion models through mode interpolation[C]. The 38th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2024: 134614–134644. doi: 10.52202/079017-4278. [8] JIA Zhaoyang, LI Jiahao, LI Bin, et al. Generative latent coding for ultra-low bitrate image compression[C]. The 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, USA, 2024: 26088–26098. doi: 10.1109/CVPR52733.2024.02465. [9] PEZONE F, MUSA O, CAIRE G, et al. Semantic-preserving image coding based on conditional diffusion models[C]. ICASSP 2024–2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Korea, 2024: 13501–13505. doi: 10.1109/ICASSP48485.2024.10447279. [10] ZHANG Chaoning, CHO J, PUSPITASARI F D, et al. A survey on Segment Anything Model (SAM): Vision foundation model meets prompt engineering[EB/OL]. arXiv preprint arXiv: 2306.06211. https://arxiv.org/abs/2306.06211, 2023. [11] KINGMA D P and WELLING M. Auto-encoding variational Bayes[EB/OL]. arXiv preprint arXiv: 1312.6114. https://arxiv.org/abs/1312.6114, 2013. [12] BURGESS C P, HIGGINS I, PAL A, et al. Understanding disentangling in β-VAE[EB/OL]. arXiv preprint arXiv: 1804.03599. https://arxiv.org/abs/1804.03599?context=cs.LG#1, 2018. [13] 罗一畅, 齐析屿, 张博锐, 等. 分割一切模型的轻量化研究综述[J]. 电子与信息学报, 2026, 48(2): 713–731. doi: 10.11999/JEIT250894.LUO Yichang, QI Xiyu, ZHANG Borui, et al. A survey of lightweight techniques for segment anything model[J]. Journal of Electronics & Information Technology, 2026, 48(2): 713–731. doi: 10.11999/JEIT250894. [14] TAPPAREL J, AFISIADIS O, MAYORAZ P, et al. An open-source LoRa physical layer prototype on GNU radio[C]. The 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Atlanta, USA, 2020: 1–5. doi: 10.1109/SPAWC48557.2020.9154273. [15] SUN Xiaorui, LIU Jun, SHEN Hengtao, et al. On efficient variants of Segment Anything Model: A survey[J]. International Journal of Computer Vision, 2025, 133(10): 7406–7436. doi: 10.1007/s11263-025-02539-8. [16] RADFORD A, KIM J W, HALLACY C, et al. Learning transferable visual models from natural language supervision[C]. The 38th International Conference on Machine Learning, 2021: 8748–8763. [17] PARK T, LIU Mingyu, WANG Tingchun, et al. Semantic image synthesis with spatially-adaptive normalization[C]. The 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, USA, 2019: 2332–2341. doi: 10.1109/CVPR.2019.00244. [18] BLAU Y and MICHAELI T. The perception-distortion tradeoff[C]. The 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6228–6237. doi: 10.1109/CVPR.2018.00652. [19] JIANG Wei, ZHAI Yongqi, LI Hangyu, et al. Learned image compression with ROI-weighted distortion and bit allocation[EB/OL]. CoRR. https://arxiv.org/html/2401.08154v2, 2024. [20] MAO Qi, YANG Tinghan, ZHANG Yinuo, et al. Extreme image compression using fine-tuned VQGANs[C]. The 2024 Data Compression Conference (DCC), Snowbird, USA, 2024: 203–212. doi: 10.1109/DCC58796.2024.00028. [21] CHEN Weilong, XU Wenxuan, CHEN Haoran, et al. Semantic communication based on large language model for underwater image transmission[J]. IEEE Transactions on Mobile Computing, 2026, 25(2): 2060–2075. doi: 10.1109/TMC.2025.3607717. -
下载:
图(10) / 表(7)
计量
- 文章访问数: 117
- HTML全文浏览量: 24
- PDF下载量: 11
- 被引次数: 0
