Considering Workload Uncertainty in Strategy Gradient-Based Hyper-Heuristic Scheduling for Software Projects

SHEN Xiaoning; SHI Jiangyi; MA Yanzhao; CHEN Wenyan; SHE Juan

doi:10.11999/JEIT250769

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

SHEN Xiaoning, SHI Jiangyi, MA Yanzhao, CHEN Wenyan, SHE Juan. Considering Workload Uncertainty in Strategy Gradient-Based Hyper-Heuristic Scheduling for Software Projects[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250769

Citation:

SHEN Xiaoning, SHI Jiangyi, MA Yanzhao, CHEN Wenyan, SHE Juan. Considering Workload Uncertainty in Strategy Gradient-Based Hyper-Heuristic Scheduling for Software Projects[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250769

Citation:

SHEN Xiaoning, SHI Jiangyi, MA Yanzhao, CHEN Wenyan, SHE Juan. Considering Workload Uncertainty in Strategy Gradient-Based Hyper-Heuristic Scheduling for Software Projects[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250769

PDF( 3854 KB)

Considering Workload Uncertainty in Strategy Gradient-Based Hyper-Heuristic Scheduling for Software Projects

doi: 10.11999/JEIT250769 cstr: 32379.14.JEIT250769

SHEN Xiaoning^{1, 2, 3, 4},
SHI Jiangyi¹,
MA Yanzhao^{1
,
,},
CHEN Wenyan¹,
SHE Juan¹

1.
School of Automation, Nanjing University of Information Science & Technology, Nanjing 210044, China
2.
Jiangsu Provincial Key Laboratory of Big Data Analysis Technology, Nanjing 210044, China
3.
Jiangsu Engineering Research Center of Meteorological Energy Utilization and Control, Nanjing 210044, China
4.
Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology, Nanjing 210044, China

Funds: Supported by the National Natural Science Foundation of China (61502239), Supported by the Natural Science Foundation of Jiangsu Province, China (No.BK20150924)

Accepted Date: 2025-11-12
Rev Recd Date: 2025-11-12

Available Online: 2025-11-15

Abstract

Abstract

Objective The Software Project Scheduling Problem (SPSP) is critical for optimizing resource allocation and task sequencing in software development, directly impacting economic efficiency and market competitiveness. However, traditional SPSP models assume deterministic task attributes, ignoring pervasive uncertainties such as fluctuating efforts due to demand changes or estimation inaccuracies. These limitations often lead to infeasible or suboptimal scheduling solutions in real-world dynamic environments. To address this gap, this study establishes a novel multi-objective software project scheduling model that explicitly incorporates task effort uncertainty. The model employs asymmetric triangular interval type-2 fuzzy numbers to robustly characterize effort variability, ensuring a realistic representation of complex software development environments. The primary objective is to enhance decision-making quality under uncertainty by developing an efficient optimization algorithm that minimizes project duration while maximizing employee satisfaction, thereby improving scheduling robustness and adaptability in dynamic software projects. Methods A Policy Gradient-based Hyper-Heuristic Algorithm (PGHHA) is developed to address the formulated model. The algorithm framework consists of a High Level Strategy (HLS) and a set of Low Level Heuristics (LLHs). HLS employs an Actor-Critic reinforcement learning architecture, where the Actor network selects appropriate LLHs based on real-time evolutionary states characterized by population convergence and diversity, while the Critic network evaluates the quality of the selected actions. Eight LLHs are designed by combining two global search operators (the matrix crossover operator and the Jaya operator with random Jitter) with two local mining strategies (duration-based local search and satisfaction-based local search). Each LLH is further configured with two levels of neighborhood search depth (V1=5, V2=20), whose values are determined through Taguchi orthogonal experiments. Each individual is encoded as a real-valued task-employee effort matrix, and constraints such as skill coverage, the maximum dedication and the maximum participant limit are enforced during optimization. To accelerate convergence, a prioritized experience replay mechanism is integrated to sample and learn from historical interaction trajectories, thereby efficiently updating network parameters. Results and Discussions Experimental evaluations were conducted on 12 synthetic instances and three real-world software projects. The proposed strategies were validated and our algorithm PGHHA was compared with six state-of-the-art ones. Performance was measured using Hypervolume Ratio (HVR) and Inverted Generational Distance (IGD), with statistical significance assessed via Wilcoxon rank-sum tests at a significance level of 0.05. Results demonstrate that PGHHA significantly outperforms all comparison algorithms in both convergence and diversity across most test instances (Table 5, Table 6). Visual comparisons of Pareto fronts (Fig. 4, Fig. 5) further confirm that solutions obtained by PGHHA are located below those of other algorithms, reflecting enhanced convergence precision, while also exhibiting better spread and uniformity. Although PGHHA requires longer computational time due to the neural network training and experience replay mechanism (Fig. 6), the significant improvement in solution quality is considered acceptable given the typically longer cycles of software development projects. The incorporation of asymmetric triangular interval type-2 fuzzy numbers effectively handles task effort uncertainty, and the dynamic selection of LLH via the Actor-Critic framework, combined with prioritized experience replay, contributes to the algorithm’s robust performance in uncertain environments. These results validate that PGHHA provides a more effective scheduling support tool, balancing multiple objectives under uncertainty without compromising solution diversity. Conclusions This paper establishes a multi-objective software project scheduling model that incorporates task effort uncertainty using asymmetric triangular interval type-2 fuzzy numbers. To solve this model, a policy gradient-based hyper-heuristic algorithm is proposed, which employs an Actor-Critic reinforcement learning framework as the high-level strategy to dynamically select low-level heuristics according to the evolutionary state of the population. A prioritized experience replay mechanism is integrated to improve learning efficiency and convergence speed. Experimental results on synthetic and real-world instances demonstrate that: (1) The proposed algorithm achieves significantly better convergence and diversity in uncertain environments compared to six state-of-the-art algorithms; (2) The combination of global search operators and local mining strategies effectively balances exploration and exploitation during evolution; (3) The use of type-2 fuzzy numbers provides a more robust representation of effort uncertainty than type-1 fuzzy numbers. However, the current model is limited to single-project scenarios. In future research, the model will be extended to multi-project scheduling environments with shared resources and cross-project dependencies. Furthermore, the integration of more adaptive reward mechanisms and lightweight neural architectures will be explored to reduce computational cost while maintaining solution quality.
- Software Project Scheduling,
- Effort Uncertainty,
- Policy Gradient,
- Hyper-Heuristic Algorithm,
- Multi-Objective Optimization

FullText(HTML)

References(27)

References

[1]	HASTIE S. Standish group 2023 chaos report[R]. 2023. (查阅网上资料, 未找到本条文献信息, 请确认并补充报告编号).
[2]	CRAWFORD B, SOTO R, JOHNSON F, et al. A max–min ant system algorithm to solve the software project scheduling problem[J]. Expert Systems with Applications, 2014, 41(15): 6634–6645. doi: 10.1016/j.eswa.2014.05.003.
[3]	ALBA E and CHICANO J F. Software project management with GAs[J]. Information Sciences, 2007, 177(11): 2380–2401. doi: 10.1016/j.ins.2006.12.020.
[4]	LI Hongbo, ZHU Hanyu, ZHENG Linwen, et al. Software project scheduling with multitasking[J]. Economic Computation and Economic Cybernetics Studies and Research, 2023, 57(1): 153–170. doi: 10.24818/18423264/57.1.23.10.
[5]	LI Hongbo, ZHU Hanyu, ZHENG Linwen, et al. Software project scheduling under activity duration uncertainty[J]. Annals of Operations Research, 2024, 338(1): 477–512. doi: 10.1007/s10479-023-05343-0.
[6]	MASMOUDI M and HAÏT A. Project scheduling under uncertainty using fuzzy modelling and solving techniques[J]. Engineering Applications of Artificial Intelligence, 2013, 26(1): 135–149. doi: 10.1016/j.engappai.2012.07.012.
[7]	YU Hui, GAO Kaizhou, WU Naiqi, et al. Scheduling multiobjective dynamic surgery problems via Q-learning-based meta-heuristics[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2024, 54(6): 3321–3333. doi: 10.1109/TSMC.2024.3352522.
[8]	MAHDAVI A, SHIRAZI B, and REZAEIAN J. Toward a scalable type-2 fuzzy model for resource-constrained project scheduling problem[J]. Applied Soft Computing, 2021, 100: 106988. doi: 10.1016/j.asoc.2020.106988.
[9]	LI Junqing, LIU Zhengmin, LI Chengdong, et al. Improved artificial immune system algorithm for type-2 fuzzy flexible job shop scheduling problem[J]. IEEE Transactions on Fuzzy Systems, 2021, 29(11): 3234–3248. doi: 10.1109/TFUZZ.2020.3016225.
[10]	JI Jianjiao, GUO Yinan, GAO Xiaozhi, et al. Q-Learning-Based hyperheuristic evolutionary algorithm for dynamic task allocation of crowdsensing[J]. IEEE Transactions on Cybernetics, 2023, 53(4): 2211–2224. doi: 10.1109/TCYB.2021.3112675.
[11]	HUANG Yao, GUO Yinan, CHEN Guoyu, et al. Q-learning assisted multi-objective evolutionary optimization for low-carbon scheduling of open-pit mine trucks[J]. Swarm and Evolutionary Computation, 2025, 92: 101778. doi: 10.1016/j.swevo.2024.101778.
[12]	杨潇, 郭一楠, 吉建娇, 等. 异构群智感知PPO多目标任务指派方法[J]. 控制理论与应用, 2024, 41(6): 1056–1066. doi: 10.7641/CTA.2023.20950. YANG Xiao, GUO Yinan, JI Jianjiao, et al. PPO multi-objective task allocation method for heterogeneous crowd sensing[J]. Control Theory & Applications, 2024, 41(6): 1056–1066. doi: 10.7641/CTA.2023.20950.
[13]	CHEN Mengjiao, XU Jiyuan, ZHANG Wenyu, et al. A new customer-oriented multi-task scheduling model for cloud manufacturing considering available periods of services using an improved hyper-heuristic algorithm[J]. Expert Systems with Applications, 2025, 269: 126419. doi: 10.1016/j.eswa.2025.126419.
[14]	YANG Jinfeng, XU Hua, CHENG Jinhai, et al. A decomposition-based memetic algorithm to solve the biobjective green flexible job shop scheduling problem with interval type-2 fuzzy processing time[J]. Computers & Industrial Engineering, 2023, 183: 109513. doi: 10.1016/j.cie.2023.109513.
[15]	杨和林, 郑梦婷, 刘帅, 等. 恶意干扰下的无人机辅助边缘计算加权能耗与时延智能优化[J]. 电子与信息学报, 2024, 46(7): 2879–2887. doi: 10.11999/JEIT230986. YANG Helin, ZHENG Mengting, LIU Shuai, et al. Intelligent weighted energy consumption and delay optimization for UAV-assisted MEC under malicious jamming[J]. Journal of Electronics & Information Technology, 2024, 46(7): 2879–2887. doi: 10.11999/JEIT230986.
[16]	DEB K, PRATAP A, AGARWAL S, et al. A fast and elitist multiobjective genetic algorithm: NSGA-II[J]. IEEE Transactions on Evolutionary Computation, 2002, 6(2): 182–197. doi: 10.1109/4235.996017.
[17]	MINKU L L, SUDHOLT D, and YAO Xin. Improved evolutionary algorithm design for the project scheduling problem based on runtime analysis[J]. IEEE Transactions on Software Engineering, 2014, 40(1): 83–102. doi: 10.1109/TSE.2013.52.
[18]	CHEN Weineng and ZHANG Jun. Ant colony optimization for software project scheduling and staffing with an event-based scheduler[J]. IEEE Transactions on Software Engineering, 2013, 39(1): 1–17. doi: 10.1109/TSE.2012.17.
[19]	CHANG C K, JIANG H Y, DI Yu, et al. Time-line based model for software project scheduling with genetic algorithms[J]. Information and Software Technology, 2008, 50(11): 1142–1154. doi: 10.1016/j.infsof.2008.03.002.
[20]	SHEN Xiaoning, MINKU L L, MARTURI N, et al. A Q-learning-based memetic algorithm for multi-objective dynamic software project scheduling[J]. Information Sciences, 2018, 428: 1–29. doi: 10.1016/j.ins.2017.10.041.
[21]	MAHMUD S, ABBASI A, CHAKRABORTTY R K, et al. A self-adaptive hyper-heuristic based multi-objective optimisation approach for integrated supply chain scheduling problems[J]. Knowledge-Based Systems, 2022, 251: 109190. doi: 10.1016/j.knosys.2022.109190.
[22]	CIMBALA J M. Taguchi orthogonal arrays[EB/OL]. https://www.me.psu.edu/cimbala/me345web_Fall_2014/Lectures/Taguchi_orthogonal_arrays.pdf, 2025.
[23]	ZHAO Fuqing, DI Shilu, and WANG Ling. A hyperheuristic with Q-learning for the multiobjective energy-efficient distributed blocking flow shop scheduling problem[J]. IEEE Transactions on Cybernetics, 2023, 53(5): 3337–3350. doi: 10.1109/TCYB.2022.3192112.
[24]	SHAO Zhongshi, SHAO Weishi, CHEN Jianrui, et al. A feedback learning-based selection hyper-heuristic for distributed heterogeneous hybrid blocking flow-shop scheduling problem with flexible assembly and setup time[J]. Engineering Applications of Artificial Intelligence, 2024, 131: 107818. doi: 10.1016/j.engappai.2023.107818.
[25]	WU Xiuli, CONSOLI P, MINKU L, et al. An evolutionary hyper-heuristic for the software project scheduling problem[C]. Proceedings of the 14th International Conference on Parallel Problem Solving from Nature–PPSN XIV, Edinburgh, UK, 2016: 37–47. doi: 10.1007/978-3-319-45823-6_4.
[26]	LI Rui, GONG Wenyin, LU Chao, et al. A learning-based memetic algorithm for energy-efficient flexible job-shop scheduling with type-2 fuzzy processing time[J]. IEEE Transactions on Evolutionary Computation, 2023, 27(3): 610–620. doi: 10.1109/TEVC.2022.3175832.
[27]	ZHU Lilu, WU Feng, HU Yanfeng, et al. A heuristic multi-objective task scheduling framework for container-based clouds via actor-critic reinforcement learning[J]. Neural Computing and Applications, 2023, 35(13): 9687–9710. doi: 10.1007/s00521-023-08208-6.