Load Balancing User Association and Resource Allocation Strategy in Time and Wavelength Division Multiplexed Passive Optical Network and Cloud Radio Access Network Joint Architecture
-
摘要: 在时分波分无源光网络(TWDM-PON)与云无线接入网(C-RAN)的联合架构中,由于无线域的负载不均衡问题,限制了网络整体的传输效率。为了充分利用TWDM-PON与C-RAN联合架构的网络资源,并保证用户的服务质量(QoS),该文提出一种负载平衡的用户关联与资源分配算法(LBUARA)。首先根据不同用户的服务质量需求以及分布式无线射频头端(RRH)的负载对用户的影响,构建用户收益函数。进而,在保证用户服务质量的前提下,根据网络状态建立随机博弈模型,并基于多智能体Q学习提出负载均衡的用户关联和资源分配算法,从而获得最优的用户关联与资源分配方案。仿真结果表明,所提的用户关联和资源分配策略能够实现网络的负载均衡,保证用户的服务质量,并提高网络吞吐量。Abstract: The load imbalance in the wireless domain limits the overall transmission efficiency of the network in the joint architecture of Time and Wavelength Division Multiplexed Passive Optical Network (TWDM-PON) and Cloud Radio Access Network (C-RAN). A Load Balancing User Association and Resource Allocation (LBUARA) algorithm is proposed to ensure the Quality of Service(QoS) of users, and make full use of network resources TWDM-PON jointly with C-RAN architecture. Firstly, the user revenue function is constructed according to the service quality requirements of different users and the impact of Remote Radio Head (RRH) load on users. Furthermore, a random game model is established according to the network state, under the premise of ensuring the quality of user service. A user association and resource allocation algorithm based on multi-agent Q-learning load balancing is proposed to obtain the optimal user association and resource allocation plan. The simulation results show that users association and resource allocation strategies mentioned can achieve load balancing network to ensure quality of service users, and improve network throughput.
-
1. 引言
伪随机序列在通信系统、雷达系统及流密码等方面有着极其广泛的应用[1]。在密码学领域的应用中,序列的线性复杂度性质是影响序列伪随机性的一个重要指标,为了能够抵抗B-M(Berlekamp-Massey)算法的攻击,一般要求序列的线性复杂度不小于其周期长度的一半。
已有大量文献研究了序列的线性复杂度,其中文献[2—6]研究了二元序列的线性复杂度,文献[7]确定了有限域
F4 上一类四元序列的线性复杂度,文献[8—10]计算了Galois环Z4={0,1,2,3} 上四元序列的线性复杂度,文献[11]给出了Z4 上基于模2p (p 为奇素数)的广义分圆的一类四元序列的线性复杂度,而周期为2p2 的四元序列尚未研究。因而,本文在文献[11]的基础上进行了推广,基于模2p2 的广义分圆定义了一类Z4 上的新四元序列,并求出了该序列的线性复杂度。值得一提的是,由于Z4 上的零因子使得计算变得困难,所以本文基于特征为4的Galois环理论来进行研究。设
p 是奇素数,g 是奇数,且g 是模p2 和模2p2 的公共本原元[12]。设S 是集合,定义aS≜{au(od2p2): u∈S} ,a+S≜{a+u(od2p2):u∈S} 。模2p2 的剩余类环为Z2p2={0,1,···,2p2−1} 。对i=0,1 ,令Di={g2k+i(od2p2),k=0,1,···,p(p−1)2−1} ,且Ei=2Di ,P0=pZ∗2p ,P1=2P0 ,即有Z2p2= ⋃1i=1(Di∪Ei∪Pi)∪{0,p2} 。显然,D0∪D1∪P0 是Z2p2∖{p2} 中所有奇数的集合,E0∪E1∪P1 是Z2p2∖{0} 中所有偶数的集合。定义Z4 上的四元序列(eu) 为eu={0,u=0或u∈D01,u∈D1∪P02,u=p2或u∈E13,u∈E0∪P1 下文将讨论序列
(eu) 的线性复杂度[13]。(eu) 的线性复杂度定义为满足eu+L+c1eu+L−1+···+ cL−1eu+1+cLeu=0,u≥0,c1,c2,···,cL∈Z4 的最小正整数L 。令C(x)=1+c1x+···+cLxL∈ Z4[x] ,显然C(0)=1 。设(eu) 的生成多项式为E(x)= e0+e1x+···+e2p2−1x2p2−1∈Z4[x] ,则LC(eu)=min{deg(C(x)):C(x)∈Z4[x],C(0)=1,E(x)C(x)≡0(odx2p2−1) (1) 2. 主要结论及其证明
2.1 主要结论
定理 1
(eu) 的线性复杂度满足LC(eu)={3p(p−1)/2+1,p≡3(od8)3p(p−1)/2+2,p≡−3(od8)2p(p−1)+1,p≡−1(od8)p(p−1)+2, p≡1(od8) 2.2 辅助引理
本小节给出证明主要结论所需的引理。如无特殊说明,本文中的多项式均属于
Z4[x] 。设
r 为2模p2 的阶,记GR(4r,4) 是阶为4r 且特征为4的Galois环,同构于剩余类环Z4[x]∖f(x) ,其中f(x)∈Z4[x] 是次数为r 的基本不可约多项式[14]。记GR(4r,4) 的单位群为GR∗(4r,4) ,因为p2|(2r−1) ,所以GR∗(4r,4) 包含了一个2r−1 阶的循环子群。任取β∈GR∗(4r,4) ,且阶为p2 。取γ=3β∈GR∗(4r,4) 则
γ 的阶为2p2 。由式(1),为确定(eu) 的线性复杂度,需计算E(γv) ,v=0,1,···,2p2−1 的值。引理 1[11] 设非零次多项式
F(x)∈Z4[x] ,若ξ,η∈GR(4r,4) 满足F(ξ)=F(η)=0 ,且η−ξ∈ GR∗(4r,4) ,则存在F1(x) ,F2(x) 使得F(x)=(x−ξ) ·F1(x)=(x−ξ)(x−η)F2(x) ,其中F1(x)=(x−η) ·F2(x) 。引理 2 (1) 若
F(γv)=0 ,v∈A ,则存在FA(x)∈GR(4r,4)[x] 使得F(x)=FA(x)∏v∈A(x−γv) 其中,
A 取为Di ,或Ei ,或Pi ,i=0,1 。(2) 若
F(γv)=0 ,v∈{p2}∪D0∪D1∪P0 ,则存在F3(x)∈GR(4r,4)[x] 使得F(x)=F3(x)(xp2 +1) 。(3) 若
F(0)=1 ,F(γv)=0 ,v∈Z2p2∖{0,p2} ,且F(±1)∈{0,2} ,则有deg(F(x))≥2p2−1 。进一步,若F(±1)=0 或F(±1)=2 ,则deg(F(x)) ≥2p2 。证明 (1) 只证
A=Di 的情形,其它情形同理可得。由γ 的选择,有(xp2−1)(x−1)=p2−1∏m=1(x−γ2m) ,因此p2=∏p2−1m=1(1−γm)(1+γm) ,所以当0≤m, n<2p2 且m≡/n(odp2) 时,γm−γn∈GR∗(4r, 4) ,从而,当F(γ)=0 ,m∈Di 时,由引理1得,∏m∈Di(x−γm)|F(x) ,即该引理得证。(2) 由(1)可得。
(3) 不妨设
F(−1)=0 。因为F(0)=1 ,则由(2)得F(x)=(xp2+1)F3(x) ,且2F3(x)≠0 ,显然(xp2−1)/(x−1)|2F3(x) ,从而deg(F(x))≥2p2 –1。进一步,若F(1)=0 ,则deg(F(x))≥2p2 。令F(±1)≠0 ,则F(±1)=2 且F(γm)=0 ,m∈D0 ∪D1∪P0 。由(1)存在Q(x)∈Z4[x] ,使得F(x)= Q(x)(xp2+1)/(x+1) ,且2Q(x)≠0 ,不难得到Q(−1)=2 ,则存在G(x)∈Z4(x) 使得Q(x)=(x+1) ⋅G(x)+2 ,即有,F(x)=(xp2+1) ·G(x) +2(xp2 +1)/(x+1) 。所以(xp2−1)|2F(x) ,即有deg(F(x))≥ 2p2 。 证毕除特殊说明外,本文中集合的下标均模2,且
i,j∈{0,1} 。引理 3 (1)
v∈Di ,则vDj=Di+j ,vEj=Ei+j ,vPj=Pj 。(2) 若
v∈Ei ,则vDj=Ei+j ,vPj=P1 ,且vEj={Ei+j,p≡±1(od8)Ei+j+1,p≡±3(od8) (3) 若
v∈P0 ,则vDj=P0 ,vEj=P1 ,vPj= Pj 。(4) 若
v∈P1 ,则vDj=vEj=vPj=P1 。(5)
P0=p2+P1 ,P1=p2+P0 。(6) 若
p≡±1(od8) ,则Di=p2+Ei ,Ei=p2 +Di ;若p≡±3(od8) ,则Di+1= p2+Ei ,Ei+1= p2+Di 。证明 (1) 只证
vDj=Di+j ,其余同理可证。对任意给定的v∈Di ,若u∈Dj ,则有v=gi+2k (od2p2) ,u=gj+2l(od2p2) ,0≤k,l<p(p−1)2 ,所以vu=gi+j+2(k+l)(od2p2) ,因此vu∈Di+j 。又因为|vDj|=|Di+j| ,所以vDj=Di+j 。(2) 仅考虑
vEj ,其余证明同(1)。首先,从 (1) 得vEj=2uEj=2Ei+j 。显然,对任意的ω∈2Ei+j ,有ω≡4a(od2p2) ,a∈Di+j ,即ω∈E0∪E1 ,则存在b∈D0∪D1 使得ω=2b(od2p2) ,从而b≡2a(odp2) 。当p≡±1(od8) ,即2是模p 的平方剩余[2],则b∈Di+j ,从而ω=2b∈Ei+j ,所以vEj=2Ei+j=Ei+j ; 当p≡±3(od8) 时,同理可证。(3)~(6)显然。 证毕
在
Z4[x] 中,令Di(x)=∑u∈Dixu ,Ei(x)= ∑u∈Eixu ,P0(x)=∑u∈P0xu ,P1(x)=∑u∈P1xu 。则E(x)=2xp2+D1(x)+P0(x)+3E0(x)+2E1(x)+3P1(x) (2) 注意到在
GR(4r,4) 上有D0(γ)+D1(γ)+P0(γ)=∑u∈D0∪D1∪P0γu=1 为计算
E(γv) ,需要如下几个引理。引理4 令
γ∈GR∗(4r,4) 的阶为2p2 ,则P0(γ)=1 。证明 因为
0=γ2p2−1=(γp−1)(P0(γ)+ {P_1}(\gamma ) + 1 + \left.{\gamma ^{{p^2}}}\big\right) = \left({\gamma ^{2p}} - 1\right)({P_1}(\gamma ) + 1) ,则由γ 的定义得P1(γ)=−1 ,P0(γ)=1 。 证毕接下来计算
D0(γ) 。记[i,j]=|(1+Di)∩Ej| ,[i,2]=|(1+Di)∩P1| ,i,j∈{0,1} 。引理 5 符号含义同上,则
\begin{array}{l}[0,0] = \left\{ \begin{array}{l}p(p - 5)/4, \; p \equiv 1{\rm{ }}\;(\!od 8)\\p(p - 3)/4, \; p \equiv - 1{\rm{ }}\;(\!od 8)\\p(p + 1)/4, \; p \equiv 3{\rm{ }}\;(\!od 8)\\p(p - 1)/4, \; p \equiv - 3{\rm{ }}\;(\!od 8)\end{array} \right.\\[0,1] = \left\{ \begin{array}{l}p(p - 1)/4, \; p \equiv 1{\rm{ }}\;(\!od 8)\\p(p + 1)/4, \; p \equiv - 1{\rm{ }}\;(\!od 8)\\p(p - 3)/4, \; p \equiv 3{\rm{ }}\;(\!od 8)\\p(p - 5)/4, \; p \equiv - 3{\rm{ }}\;(\!od 8)\end{array} \right.\\[0,1] = \left\{ \begin{array}{l}p - 1, \; p \equiv 1{\rm{ }}\;(\!od 4)\\0,\quad\quad{\rm{ }}p \equiv 3{\rm{ }}\;(\!od 4)\end{array} \right.\end{array} 证明 记
Hi={g2n+i(odp2):0≤n<p ⋅(p−1)/2} ,且R={0,p,···,(p−1)p} 。则{u(odp2):u∈Di}=Hi{2u(odp2):u∈Di}=Hi+l{u(odp2):u∈P1}=R∖{0} 当
p≡±1(od8) 时,l=0 ;否则l=1 。因此,[i,j]=|(1+Hi)∩Hj+l| ,[0,2]=|(1+H0)∩(R∖{0})| 。由|(1+Hi)∩Hj| 的取值[15]。 证毕引理 6 若
p≡±1(od8) ,则p2+2∈D0 ;否则p2+2∈D1 。证明 只证
p≡±1(od8) 时的情形。设p2+2∈D1 ,则存在整数q 使得p2+2≡g2n+1+2p2q ,即2是模p 的平方非剩余,矛盾,所以p2+2∈D0 。 证毕引理 7 令
ωp=D0(γ) ,则ωp={0或1,p≡1(od8)2或3,p≡−3(od8)0或3,p≡3(od4) 证明 只证
p≡1(od8) 时的情形,其余情形同理可证。由于(ωp)2=p(p−1)2−1∑l,m=0γg2l+g2m=p(p−1)2−1∑l,m=0γg2l(g2(l−m)+1)=p(p−1)2−1∑m,n=0γg2m(g2n+1) (3) 且
g2n+1(od2p2)∈E0∪E1∪P1∪{0} 。令λn=p(p−1)2−1∑m=0γg2m(g2n+1) 下面分3种情况讨论:
(1) 令
Ni={n, 0≤np(p−1)2, g2n+1(od2p2)∈Ei} 则
|Ni|=[0,i] 。当n∈Ni ,由引理3,引理6有λn=∑v∈Diγ2v={(−1)i+1ωp,p≡±1(od8)(−1)iωp,p≡±3(od8) (2) 令
N2={n, 0≤n<p(p−1)2, g2n+1(od2p2)∈Ei} 则
|Ni|=[0,i] 。当n∈N2 ,有λn=∑v∈P0γ2v=P0(γ2)=P0(−γp2+2)=−P0(γ) (3) 若
g2n+1≡0(od2p2) ,则p≡1(od4) 且n=p(p−1)/4 ,因而λn=p(p−1)/2 。即:由式(3)得(ωp)2=|N0|(−ωP)+|N1|ωP−|N2|+p(p− 1)/2 。又从引理5得(ωp)2=ωp ,即ωp∈ {0,1} 。 证毕引理 8 (1) 若
p≡±3(od8) ,则E(γv)={2,v∈D0∪P12ωp+2,v∈E0∪E10,v∈D1∪P0 否则
E(γv)={2ωp,v∈D0∪D10,v∈P0∪P12,v∈E0∪E1 (2) 当
v=0 时,E(γv)=3p2+p+2 ;当v= p2 时,E(γv)=2p 。证明 仅证 (1)中
p≡±3(od8) 的情形,其余证明类似。 对任意的v∈Ei∪P1 ,记v=2¯v ,其中¯v∈Di∪P0 ,由引理3(5)和(6)得,p2+2¯v ∈Dj+1∪P0 且有γv=γ2¯v=−γp2+2¯v ,又由引理3(1)可得D1(γv)=−∑u∈D1γu(p2+2¯v)={−D0(γω),¯v∈D0−D1(γω),¯v∈D1−P0(γω),¯v∈P0 进一步地,由引理3可得,当
v∈Z2p2∖{0,p2} 时,P0(γv)=(−1)v+1 ,P1(γv)=−1 ;且有D1(γv)={(−1)i+1ωp,v∈Di∪Ei(−1)i,v∈PiEi(γv)={(−1)i+jωp,v∈Dj∪Ej+1−1,v∈Pi 则根据式(2)结论得证。
定义
Γj(x)=∏v∈Dj(x−γv),Λj(x)=∏v∈Ej(x−γv)M(x)=∏v∈P0(x−γv),N(x)=∏v∈P1(x−γv) 引理 9
M(x) ,N(x) ,Γj(x) ,Λj(x)∈Z4[x] 。证明 显然
M(x) ,N(x)∈Z4[x] 。仅考虑Γ0(x) ,对Γ1(x) ,Λj(x) 同理可得。不难得到Γ0(x) 的系数满足am=(−1)m∑d1<⋯<dm,d1,⋯,dm∈D0γd1+⋯+dm,1≤m≤p(p−1)/2 设
γb 为和式中的一项,b≡∑mk=1dk(od2p2) ,b≡/0(odp2) 。由引理3易得x−γg2ndk|∏v∈D0 (x−λv) 所以
γg2nd1···γg2ndm=γg2nb 为和式的一项,即γb+γg2b+···+γgp(p−1)−2b=D0(γb) ,则存在b1,b2··· bn 使得am=(−1)m∑nk=0D0(γbk)+l ,其中l= |{a|a≡∑mk=1dk≡0(odp2)且d1<···<dm,d1,···, dm∈D0}| 。又与引理8的证明类似可得D0 (γbk)=ωp∈Z4 。 证毕由于
γv 为xp2+1 的根,v∈{p2}∪D0∪D1 ∪P0 ,则xp2+1=(x+1)Γ0(x)Γ1(x)M(x) (4) 同样地,有
xp2−1=(x−1)Λ0(x)Λ1(x)N(x) (5) 引理 10 存在
Vk(x)∈Z4[x] ,k=1,2,···,5 ,使得D1(x)+3E0(x)={−(xp2−1)(−ωp+Γ0(x)V1(x)),p≡±3(od8)(x−1)Γ0(x)Γ1(x)N(x)V2(x),p≡±1(od8) P0(x)+3P1(x)={(xp2−1)(−1+Γ1(x)V3(x)),p≡±3(od8)(xp2−1)P1(x),p≡±1(od8) 2xp2+2E1(x)={2(xp2−1)+(x−1)M(x)N(x)V4(x),p≡3(od4)2(xp2−1)+M(x)N(x)V5(x),p≡1(od4) 证明 仅对
p≡3(od8) 的情形进行证明,其余证明类似。由引理3得D1(x)+3E0(x)= −(xp2−1)D1(x) 。由引理8可知,当v∈D0 时,有D1(γv)=−ωp ,则存在V1(x) 使得D1(x)=−ωp +Γ0(x)V1(x) 。注意到
P0(x)+3P1(x)=∑u∈P1xu+p2+3∑u∈P1xu=(xp2−1)P1(x) 且
P1(γv)=P1(γ)=−1 ,v∈D1 ,则结论易证。因为
2xp2+2E1(x)=2(xp2−1)+2+2D1(x2) 。由引理8 (1),当v∈Pi 时,2+2D1(γv)=0 ,则由引理2,2+2D1(x)=M(x)N(x)G(x) ,其中G(x) ∈Z4[x] 。即2+2D1(x2)=M(x2)N(x2)G(x2) 。由引理3 (5)得N(x)=∏v∈P0(x−γv+p2) ,则N(x2) =∏v∈P1(x2−γu)=∏v∈P0(x−γu)(x−γp2+u) =M(x)N(x) 。另一方面,
2+2E1(1)=0 ,则存在V4(x) 使得M(x2)G(x2)=(x−1)V4(x) ,从而2xp2+2E1(x) =2(xp2−1)+(x−1)M(x)N(x)V4(x) 。综上,即可得
p≡3(od8) 时的结论。 证毕引理 11 存在
Wk(x)∈Z4[x] ,k=1,2,···,4 ,使得E(x)={(x−1)M(x)N(x)Γ1(x)W1(x),p≡3(od8)M(x)N(x)Γ1(x)W2(x),p≡−3(od8)(x−1)M(x)N(x)W3(x),p≡−1(od8)M(x)N(x)Γ0(x)Γ1(x)W4(x),p≡1(od8) 证明 仅证明
p≡3(od8) 的情形,其余证明同理。由引理10和式(5)可得E(x)=(x−1) ·N(x)H(x) ,其中,H(x)=Λ0(x)Λ1(x)(ωp+1−Γ0(x) ·Vp(x)+Γ1(x)Vp(x))+M(x)Vp(x) 。又由引理8易得若
v∈D0∪E0∪E1 ,则H(γv)≠0 ;若v∈D1∪P0 ,则H(γv)=0 ,则由引理1可得,存在W1(x)∈Z4[x] 且W1(γv)≠0 ,v∈D0∪E0∪E1 使得H(x)=M(x)Γ1(x)W1(x) 。 证毕2.3 定理1的证明
证明 若
p≡3(od8) 。由引理11和引理8 (2)可得W1(γv)≠0 ,v∈D0∪E0∪E1∪{p2} ,从而E(x)(x+1)Γ0(x)Λ0(x)Λ1(x)≡0(modx2p2−1) 所以
LC(eu)≤3p(p−1)/2+1 。又因为gcd((x−1) ·M(x)N(x)Γ1(x),(x+1)Γ0(x)Λ0(x)Λ1(x))=1 ,所以由式(1),式(4),式(5)和引理11得W1(x)C(x)≡ 0(od(x+1)Γ0(x)Λ0(x)Λ1(x)) ,则有W1(γv)C(γv) =0 ,v∈{D0∪E0∪E1∪{p2}} ,从而,若W1(γv) ∈GR∗(4r,4) ,则C(γv)=0 ;若2W1(γv)∈GR(4r, 4)∖{0} ,则2C(γv)=0 。显然有(x+1)Γ0(x)Λ0(x) ⋅Λ1(x)|2C(x) ,即LC(eu)≥3p(p−1)2+1 。因此,LC(eu)=3p(p−1)2+1 。其它情形同理可证。 证毕
3. 结束语
本文在
Z4 上定义了一类周期为2p2 的新四元广义分圆序列(eu) ,并研究了该序列的关联多项式和线性复杂度。结果表明,当p≡3(od4) 和p≡ −3(od8) 时,这类序列拥有好的线性复杂度,能够抵抗B-M算法的攻击,在保密通讯中可以有广泛的应用。此外,若定义序列(su) 为su={0,u=0或u∈D01,u∈D1∪P02,u=p2或u∈E1∪P13,u∈E0 与前面的证明类似可得,该序列的线性复杂度达到最大值,即当
p≡1(od4) 时,LC(su)=2p2 ;否则LC(su)=2p2−1 。 -
表 1 负载均衡的用户关联和资源分配算法
(1) 初始化episode,每个用户的Q值Qi(s,ai)以及ϕi(si,ai) (2) for each step of an episode to t steps do (3) for each UE i do (4) 在状态si时通过式(21)选择动作ai (5) 通过式(7)计算为每个用户分配的RB数量 (6) 通过式(13)计算Vi (7) 每个用户获取关联状态s′,设置 s′→s (8) 通过式(20)更新Qi(s,ai) (9) 更新ϕi(si,ai) (10) end for (11) if 当前状态集合S={1,1,···,1} (12) break (13) end if (14) 最终所有的用户得到关联策略(si,ai) -
[1] REN Hong, LIU Nan, PAN Cunhua, et al. Low-latency C-RAN: An next-generation wireless approach[J]. IEEE Vehicular Technology Magazine, 2018, 13(2): 48–56. doi: 10.1109/MVT.2018.2811244 [2] ZHAO Wentao and WANG Shaowei. Traffic density-based RRH selection for power saving in C-RAN[J]. IEEE Journal on Selected Areas in Communications, 2016, 34(12): 3157–3167. doi: 10.1109/JSAC.2016.2600414 [3] KALIL M, Al-DWEIK A, SHARKH M A, et al. A framework for joint wireless network virtualization and cloud radio access networks for next generation wireless networks[J]. IEEE Access, 2017, 5: 20814–20827. doi: 10.1109/ACCESS.2017.2746666 [4] SIMEONE O, MAEDER A, PENG Mugen, et al. Cloud radio access network: Virtualizing wireless access for dense heterogeneous systems[J]. Journal of Communications and Networks, 2016, 18(2): 135–149. doi: 10.1109/JCN.2016.000023 [5] ZHANG Zhiguo, CHEN Yanxu, CAI Shanyong, et al. Colorless-light and tunable-light-source schemes for TWDM and WDM PONs[J]. IEEE Communications Magazine, 2018, 56(8): 120–128. doi: 10.1109/MCOM.2018.1700751 [6] WANG Xinbo, CAVDAR C, WANG Lin, et al. Virtualized cloud radio access network for 5G transport[J]. IEEE Communications Magazine, 2017, 55(9): 202–209. doi: 10.1109/MCOM.2017.1600866 [7] 王汝言, 徐宁宁, 吴大鹏. 能耗和时延感知的虚拟化云无线接入网络资源分配机制[J]. 电子与信息学报, 2019, 41(1): 83–90. doi: 10.11999/JEIT180063WANG Ruyan, XU Ningning, and WU Dapeng. Energy consumption and delay-aware resource allocation mechanism for virtualization cloud radio access network[J]. Journal of Electronics &Information Technology, 2019, 41(1): 83–90. doi: 10.11999/JEIT180063 [8] ALNOMAN A, CARVALHO G H S, ANPALAGAN A, et al. Energy efficiency on fully cloudified mobile networks: Survey, challenges, and open issues[J]. IEEE Communications Surveys & Tutorials, 2018, 20(2): 1271–1291. doi: 10.1109/COMST.2017.2780238 [9] WU Dapeng, ZHANG Zhihao, WU Shaoen, et al. Biologically inspired resource allocation for network slices in 5G-enabled internet of things[J]. IEEE Internet of Things Journal, 2019, 6(6): 9266–9279. doi: 10.1109/JIOT.2018.2888543 [10] ALI M, RABBANI Q, NAEEM M, et al. Joint user association, power allocation, and throughput maximization in 5G H-CRAN networks[J]. IEEE Transactions on Vehicular Technology, 2017, 66(10): 9254–9262. doi: 10.1109/TVT.2017.2715229 [11] 王汝言, 李宏娟, 吴大鹏. 基于Stackelberg博弈的虚拟化无线传感网络资源分配策略[J]. 电子与信息学报, 2019, 41(2): 377–384. doi: 10.11999/JEIT180277WANG Ruyan, LI Hongjuan, and WU Dapeng. Stackelberg game-based resource allocation strategy in virtualized wireless sensor network[J]. Journal of Electronics &Information Technology, 2019, 41(2): 377–384. doi: 10.11999/JEIT180277 [12] ELHATTAB M K, ELMESALAWY M M, and ISMAIL T. Fronthaul-aware user association in 5G heterogeneous cloud radio access networks: A matching game perspective[C]. 2018 International Symposium on Networks, Computers and Communications (ISNCC), Rome, Italy, 2018. doi: 10.1109/ISNCC.2018.8531056. [13] WU Dapeng, LIU Jia, and YANG Zhigang. Bilateral satisfaction aware participant selection with MEC for mobile crowd sensing[J]. IEEE Access, 2020, 8: 48110–48122. doi: 10.1109/ACCESS.2020.2978774 [14] SU Gongchao, CHEN Bin, LIN Xiaohui, et al. A sequential auction game for QoS-aware user association in heterogeneous cellular networks[C]. Proceedings of 2018 Tenth International Conference on Ubiquitous and Future Networks (ICUFN), Prague, Czech Republic, 2018: 464–468. doi: 10.1109/ICUFN.2018.8436793. [15] GE Xin, LI Xiuhua, JIN Hu, et al. Joint user association and user scheduling for load balancing in heterogeneous networks[J]. IEEE Transactions on Wireless Communications, 2018, 17(5): 3211–3225. doi: 10.1109/TWC.2018.2808488 [16] PROSKOCHYLO A, ZRIAKHOV M, and AKULYNICHEV A. The effects of queueing algorithms on QoS for real-time traffic in process of load balancing[C]. 2018 International Scientific-Practical Conference Problems of Infocommunications. Science and Technology (PIC S&T), Kharkiv, Ukraine, 2018: 575–580. doi: 10.1109/INFOCOMMST.2018.8632161. [17] WANG Feng, CHEN Wen, TANG Hongying, et al. Joint optimization of user association, subchannel allocation, and power allocation in multi-cell multi-association OFDMA heterogeneous networks[J]. IEEE Transactions on Communications, 2017, 65(6): 2672–2684. doi: 10.1109/TCOMM.2017.2678986 [18] AWAIS M, AHMED A, NAEEM M, et al. Efficient joint user association and resource allocation for cloud radio access networks[J]. IEEE Access, 2017, 5: 1439–1448. doi: 10.1109/ACCESS.2017.2663758 [19] 王汝言, 池文祥, 张鸿. WOBAN中基于令牌桶算法的自适应流量整形策略[J]. 电子与信息学报, 2017, 39(6): 1401–1408. doi: 10.11999/JEIT160974WANG Ruyan, CHI Wenxiang, and ZHANG Hong. Adaptive traffic shaping policy based on token bucket algorithm of wireless-optical broadband access network[J]. Journal of Electronics &Information Technology, 2017, 39(6): 1401–1408. doi: 10.11999/JEIT160974 [20] WANG Kaidi, LIU Yuanwei, DING Zhiguo, et al. User association and power allocation for multi-cell non-orthogonal multiple access networks[J]. IEEE Transactions on Wireless Communications, 2019, 18(11): 5284–5298. doi: 10.1109/TWC.2019.2935433 [21] SCHERER W T, ADAMS S, and BELING P A. On the practical art of state definitions for Markov decision process construction[J]. IEEE Access, 2018, 6: 21115–21128. doi: 10.1109/ACCESS.2018.2819940 [22] JANG B, KIM M, HARERIMANA G, et al. Q-learning algorithms: A comprehensive classification and applications[J]. IEEE Access, 2019, 7: 133653–133667. doi: 10.1109/ACCESS.2019.2941229 [23] LI Dong, ZHANG Haijun, LONG Keping, et al. User association and power allocation based on q-learning in ultra dense heterogeneous networks[C]. 2019 IEEE Global Communications Conference (GLOBECOM), Waikoloa, USA, 2019: 1–5. doi: 10.1109/GLOBECOM38437.2019.9013455. [24] ZHAO Nan, LIANG Yingchang, NIYATO D, et al. Deep reinforcement learning for user association and resource allocation in heterogeneous cellular networks[J]. IEEE Transactions on Wireless Communications, 2019, 18(11): 5141–5152. doi: 10.1109/TWC.2019.2933417 [25] CHEN Xianfu, ZHAO Zhifeng, and ZHANG Honggang. Stochastic power adaptation with multiagent reinforcement learning for cognitive wireless mesh networks[J]. IEEE Transactions on Mobile Computing, 2013, 12(11): 2155–2166. doi: 10.1109/TMC.2012.178 [26] SEDIQ A B, GOHARY R H, SCHOENEN R, et al. Optimal tradeoff between sum-rate efficiency and Jain’s fairness index in resource allocation[J]. IEEE Transactions on Wireless Communications, 2013, 12(7): 3496–3509. doi: 10.1109/TWC.2013.061413.121703 -