Joint Local Linear Embedding and Deep Reinforcement Learning for RIS-MISO Downlink Sum-Rate Optimization

SUN Jun; YANG Junlong; YANG Qingqing; HU Mingzhi; WU Ziyi

doi:10.11999/JEIT241083

Volume 47 Issue 7

Jul. 2025

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 > 47(7): 2117-2126

SUN Jun, YANG Junlong, YANG Qingqing, HU Mingzhi, WU Ziyi. Joint Local Linear Embedding and Deep Reinforcement Learning for RIS-MISO Downlink Sum-Rate Optimization[J]. Journal of Electronics & Information Technology, 2025, 47(7): 2117-2126. doi: 10.11999/JEIT241083

Citation:

SUN Jun, YANG Junlong, YANG Qingqing, HU Mingzhi, WU Ziyi. Joint Local Linear Embedding and Deep Reinforcement Learning for RIS-MISO Downlink Sum-Rate Optimization[J]. Journal of Electronics & Information Technology, 2025, 47(7): 2117-2126. doi: 10.11999/JEIT241083

Citation:

PDF( 3269 KB)

Joint Local Linear Embedding and Deep Reinforcement Learning for RIS-MISO Downlink Sum-Rate Optimization

doi: 10.11999/JEIT241083 cstr: 32379.14.JEIT241083

Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China

Funds: The National Natural Science Foundation of China (62461030), Yunnan Fundamental Research Projects (202401AT070355)

Received Date: 2024-12-09
Rev Recd Date: 2025-05-29

Available Online: 2025-06-13

Publish Date: 2025-07-22

Abstract

Abstract

Objective Reconfigurable Intelligent Surfaces (RISs) enhance signal transmission efficiency for large-scale user networks by adaptively controlling signal propagation paths. In RIS-assisted Multiple Input Single Output (MISO) systems, Deep Reinforcement Learning (DRL) is widely employed to jointly optimize Base Station (BS) beamforming and RIS phase shifts. However, the channel state space expands quadratically with the number of users, leading to increased training overhead and reduced algorithm efficiency. To address this challenge, the LLE-SAC algorithm is proposed, in which Local Linear Embedding (LLE) is integrated for dimensionality reduction with the Soft Actor-Critic (SAC) algorithm for policy optimization. This joint framework aims to improve system throughput and training efficiency by reducing the complexity of the channel state representation, thereby enabling the construction of a scalable and intelligent communication system for RIS-assisted MISO in multi-user scenarios. Methods The LLE-SAC algorithm models the wireless environment as a cascaded channel comprising the links between the BS, RIS, and user equipment. To reduce the dimensionality of the high-dimensional channel state, the algorithm searches for the optimal number of neighboring nodes and low-dimensional features based on the principle of minimizing reconstruction error. These parameters are selected through a randomized search strategy to ensure minimal information loss during dimensionality reduction. The LLE algorithm is then applied using the identified optimal parameters to map the original high-dimensional state into a low-dimensional representation. Parameter selection in LLE is constrained to preserve the local geometric structure of the nonlinear channel data and achieve efficient dimensionality reduction. The resulting low-dimensional state, combined with the BS transmission power and user equipment receive power, forms the input state space for the SAC algorithm. Within the SAC framework, the state space comprises the reduced-dimension representation of the cascaded channel and the BS beamforming and RIS phase shift matrix from the previous time step. The action space consists of the current BS beamforming vectors and RIS phase shifts. The reward function is defined as the sum rate of the RIS-assisted MISO system, guiding the agent to iteratively optimize its beamforming strategy. By leveraging both channel state abstraction and historical control parameters, the agent dynamically selects actions that maximize the system sum rate under complex multi-user conditions. Results and Discussions The LLE-SAC algorithm reduces the dimensionality of the high-dimensional cascaded channel state. It then computes the BS beamforming vectors and RIS phase shifts based on the resulting low-dimensional representation to maximize the sum rate of the RIS-assisted MISO system. Simulation results demonstrate that LLE-SAC effectively identifies the optimal number of neighboring nodes and low-dimensional features to minimize reconstruction error (Fig. 6, Fig. 7). For a system with 30 users, the minimum reconstruction error reaches 0.061 when the number of neighboring nodes is set to 2 and the dimensionality is reduced to 15, compressing the state space from 7092 to 960. In terms of training overhead (Fig. 8), the LLE-SAC algorithm reduces training time by 18.3% and computational resource usage by 64.8% relative to the conventional SAC algorithm when the user count reaches 40. This efficiency gain increases with user scale, further reducing training overhead in large-scale scenarios. Under high transmission power (Fig. 9), the LLE-SAC algorithm achieves a higher sum rate than both the alternating optimization and semi-definite relaxation algorithms, while maintaining comparable performance to SAC. The algorithm also scales effectively with the number of transmit antennas, achieving increased sum rates and reduced inter-user interference, further confirming its effectiveness. Moreover, in ten independent runs using different random seeds (Fig. 10), the LLE-SAC algorithm consistently yields optimal sum rate performance, demonstrating both robustness and stability. Conclusions The proposed method addresses the challenge of high-dimensional channel states, which significantly increase the training overhead in RIS-assisted MISO systems, by integrating the LLE algorithm with the SAC framework. This integration enables effective dimensionality reduction of the cascaded channel state, thereby lowering training costs while maintaining sum rate performance. The simulation results demonstrate three key findings. First, when the number of users reaches 40, the LLE-SAC algorithm reduces training time by 18.3% and computational resource consumption by 64.8% compared to the SAC algorithm. Second, under increasing transmission power, the proposed method achieves superior sum rate performance relative to conventional optimization methods and performs comparably to SAC. Third, across different antenna configurations, the LLE-SAC algorithm yields improved sum rates with increasing transmission power, demonstrating its robustness and scalability. Future work will explore the application of the LLE-SAC algorithm in edge computing environments with large-scale user access.
- Reconfigurable Intelligent Surface (RIS),
- Local Linear Embedding (LLE),
- Deep Reinforcement Learning (DRL),
- Sum-rate,
- Training overhead

FullText(HTML)

References(23)

References

[1]	BASAR E, ALEXANDROPOULOS G C, LIU Yuanwei, et al. Reconfigurable intelligent surfaces for 6G: Emerging hardware architectures, applications, and open challenges[J]. IEEE Vehicular Technology Magazine, 2024, 19(3): 27–47. doi: 10.1109/MVT.2024.3415570.
[2]	BILOTTI F, BARBUTO M, HAMZAVI-ZARGHANI Z, et al. Reconfigurable intelligent surfaces as the key-enabling technology for smart electromagnetic environments[J]. Advances in Physics: X, 2024, 9(1): 2299543. doi: 10.1080/23746149.2023.2299543.
[3]	FENG Yijun, HU Qi, QU Kai, et al. Reconfigurable intelligent surfaces: Design, implementation, and practical demonstration[J]. Electromagnetic Science, 2023, 1(2): 0020111. doi: 10.23919/emsci.2022.0011.
[4]	GUO Huayan, LIANG Yingchang, CHEN Jie, et al. Weighted sum-rate maximization for intelligent reflecting surface enhanced wireless networks[C]. 2019 IEEE Global Communications Conference, Waikoloa, USA, 2019: 1–6. doi: 10.1109/GLOBECOM38437.2019.9013288.
[5]	田心记, 孟浩然, 李兴旺, 等. 双STAR-RIS辅助下行NOMA系统中最大化和速率的方法[J]. 电子与信息学报, 2024, 46(9): 3537–3543. doi: 10.11999/JEIT240007. TIAN Xinji, MENG Haoran, LI Xingwang, et al. Method of maximizing sum rate for dual STAR-RIS assisted downlink NOMA systems[J]. Journal of Electronics & Information Technology, 2024, 46(9): 3537–3543. doi: 10.11999/JEIT240007.
[6]	ZHU Guangxu, LYU Zhonghao, JIAO Xiang, et al. Pushing AI to wireless network edge: An overview on integrated sensing, communication, and computation towards 6G[J]. Science China Information Sciences, 2023, 66(3): 130301. doi: 10.1007/s11432-022-3652-2.
[7]	LEE H, LEE B, YANG H, et al. Towards 6G hyper-connectivity: Vision, challenges, and key enabling technologies[J]. Journal of Communications and Networks, 2023, 25(3): 344–354. doi: 10.23919/JCN.2023.000006.
[8]	ZHONG Ruikang, LIU Yuanwei, MU Xidong, et al. AI empowered RIS-assisted NOMA networks: Deep learning or reinforcement learning?[J]. IEEE Journal on Selected Areas in Communications, 2022, 40(1): 182–196. doi: 10.1109/JSAC.2021.3126068.
[9]	HUANG Hongji, SONG Yiwei, YANG Jie, et al. Deep-learning-based millimeter-wave massive MIMO for hybrid precoding[J]. IEEE Transactions on Vehicular Technology, 2019, 68(3): 3027–3032. doi: 10.1109/TVT.2019.2893928.
[10]	HUANG Hao, XIA Wenchao, XIONG Jian, et al. Unsupervised learning-based fast beamforming design for downlink MIMO[J]. IEEE Access, 2019, 7: 7599–7605. doi: 10.1109/ACCESS.2018.2887308.
[11]	陈真, 杜晓宇, 唐杰, 等. 基于深度强化学习的RIS辅助通感融合网络: 挑战与机遇[J]. 电子与信息学报, 2024, 46(9): 3467–3473. doi: 10.11999/JEIT240086. CHEN Zhen, DU Xiaoyu, TANG Jie, et al. DRL-based RIS-assisted ISAC network: Challenges and opportunities[J]. Journal of Electronics & Information Technology, 2024, 46(9): 3467–3473. doi: 10.11999/JEIT240086.
[12]	CHEN Peng, LI Xiao, MATTHAIOU M, et al. DRL-based RIS phase shift design for OFDM communication systems[J]. IEEE Wireless Communications Letters, 2023, 12(4): 733–737. doi: 10.1109/LWC.2023.3242449.
[13]	HUANG Chongwen, MO Ronghong, and YUEN C. Reconfigurable intelligent surface assisted multiuser MISO systems exploiting deep reinforcement learning[J]. IEEE Journal on Selected Areas in Communications, 2020, 38(8): 1839–1850. doi: 10.1109/JSAC.2020.3000835.
[14]	ZHANG Ruichen, XIONG Ke, LU Yang, et al. Energy efficiency maximization in RIS-assisted SWIPT networks with RSMA: A PPO-based approach[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(5): 1413–1430. doi: 10.1109/JSAC.2023.3240707.
[15]	HUANG Chongwen, YANG Zhaohui, ALEXANDROPOULOS G C, et al. Multi-hop RIS-empowered terahertz communications: A DRL-based hybrid beamforming design[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(6): 1663–1677. doi: 10.1109/JSAC.2021.3071836.
[16]	SAGLAM B, GURGUNOGLU D, and KOZAT S S. Deep reinforcement learning based joint downlink beamforming and RIS configuration in RIS-aided MU-MISO systems under hardware impairments and imperfect CSI[C]. 2023 IEEE International Conference on Communications Workshops, Rome, Italy, 2023: 66–72. doi: 10.1109/ICCWorkshops57953.2023.10283517.
[17]	IZENMAN A J. Introduction to manifold learning[J]. WIREs: Computational Statistics, 2012, 4(5): 439–446. doi: 10.1002/wics.1222.
[18]	ZHOU Xiaoping, WANG Peipei, YANG Zhe, et al. A manifold learning two-tier beamforming scheme optimizes resource management in massive MIMO networks[J]. IEEE Access, 2020, 8: 22976–22987. doi: 10.1109/ACCESS.2020.2964615.
[19]	ZHU Guangxu, LIU Dongzhu, DU Yuqing, et al. Toward an intelligent edge: Wireless communication meets machine learning[J]. IEEE Communications Magazine, 2020, 58(1): 19–25. doi: 10.1109/MCOM.001.1900103.
[20]	ZHU Fenghao, WANG Xinquan, HUANG Chongwen, et al. Robust beamforming for RIS-aided communications: Gradient-based manifold meta learning[J]. IEEE Transactions on Wireless Communications, 2024, 23(11): 15945–15956. doi: 10.1109/TWC.2024.3435023.
[21]	DE SOUZA JUNIOR W, GUERRA D W M, MARINELLO FILHO J C, et al. Manifold-based optimizations for RIS-aided massive MIMO systems[J]. IEEE Open Journal of the Communications Society, 2024, 5: 7913–7940. doi: 10.1109/OJCOMS.2024.3512662.
[22]	DAI Linglong and WEI Xiuhong. Distributed machine learning based downlink channel estimation for RIS assisted wireless communications[J]. IEEE Transactions on Communications, 2022, 70(7): 4900–4909. doi: 10.1109/TCOMM.2022.3175175.
[23]	HAARNOJA T, ZHOU A, HARTIKAINEN K, et al. Soft actor-critic algorithms and applications[EB/OL]. https://arxiv.org/abs/1812.05905, 2018.