Call For Paper September 2026
Research Article | Open Access | Download PDF
Volume 13 | Issue 8 | Year 2026 | Article Id. IJECE-V13I8P101 | DOI : https://doi.org/10.14445/23488549/IJECE-V13I8P101

HRL-TaskOpt: A Hierarchical Reinforcement Learning-Based Task Scheduling Framework for Multi-Cloud and Hybrid Environments

Krishna Rao Patwari, Raghvendra Kumar, J.S.V.R.S. Sastry

Received Revised Accepted Published
06 Jan 2026 20 Feb 2026 24 Jul 2026 31 Aug 2026

Citation :

Krishna Rao Patwari, Raghvendra Kumar, J.S.V.R.S. Sastry, "HRL-TaskOpt: A Hierarchical Reinforcement Learning-Based Task Scheduling Framework for Multi-Cloud and Hybrid Environments," International Journal of Electronics and Communication Engineering, vol. 13, no. 8, pp. 1-25, 2026. Crossref, https://doi.org/10.14445/23488549/IJECE-V13I8P101

Abstract

Cloud computing has emerged as a new paradigm, which entrusts task scheduling to ensure the satisfaction of stringent constraints on latency, energy, and resources for sustainably running real-time applications. State-of-the-art natural DRL-based scheduling solutions mainly rely heavily on DRL techniques and are either limited in scalability, adaptivity, or generality of workloads/infrastructures. State-of-the-art flat methods, such as DQN and actor–critic models, are not sufficiently effective at high levels of decision complexity and are not robust against varying system loads and task priorities. In this paper, we present HRL-TaskOpt, a novel Hierarchical Reinforcement Learning-based task scheduling framework that combines high-level global task offloading with millisecond-granularity local scheduling policies in an edge–cloud scenario. In the proposed framework, there are two levels of agents: a high-level policy that utilizes Proximal Policy Optimization (PPO) to select the optimal execution tiers (edge or cloud), and a low-level policy based on Deep Q-Networks (DQN) to manage scheduling within nodes (edge or cloud). Such decomposition enables HRL-TaskOpt to efficiently accommodate heterogeneous workloads and adapt to dynamically changing infrastructure. We use synthetically generated workloads that reflect the characteristics of real-world applications to demonstrate the effectiveness of our model and compare it with state-of-the-art models such as SA-DQN, DRL-DO, and GD-DRL. We experimentally demonstrate that HRL-TaskOpt achieves a time reduction of up to 21.4% for task completion (2,000-task workload vs. SA-DQN), an energy efficiency improvement of up to 20.3% (10,000-task workload vs. GD-DRL), and a task success rate improvement of up to 13.3% (averaged across baselines at the 2,000-task workload), compared to these models. In addition, robustness to resource failures and sensitivity to task-type diversity, demonstrated in images, validate the real-life usability of the model. HRL-TaskOpt offers a scalable and intelligent solution for adaptive task scheduling, making it an appealing candidate for deployment in next-generation edge–cloud continuum systems.

Keywords

Hierarchical Reinforcement Learning, Task scheduling, Edge–cloud continuum, Deep Q-Network, Proximal Policy Optimization.

References

  1. Alberto Robles-Enciso, and Antonio F. Skarmeta, “A Multi-Layer Guided Reinforcement Learning-based Tasks Offloading in Edge Computing,” Computer Networks, vol. 220, pp. 1-14, 2023.
    [    CrossRef] [Google Scholar] [Publisher Link]
  2. Yaqiang Zhang et al., “Online Scheduling Optimization for DAG-Based Requests through Reinforcement Learning in Collaboration Edge Networks,” IEEE Access, vol. 8, pp. 72985-72996, 2020.
    [    CrossRef] [Google Scholar] [Publisher Link]
  3. Daokun Qi et al., “Real-Time Scheduling of Power Grid Digital Twin Tasks in Cloud via Deep Reinforcement Learning,” Journal of Cloud Computing: Advances, Systems and Applications, vol. 13, no. 1, pp. 1-12, 2024.
    [    CrossRef] [Google Scholar] [Publisher Link]
  4. Joahannes B. D. da Costa et al., “Mobility and Deadline-Aware Task Scheduling Mechanism for Vehicular Edge Computing,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 10, pp. 11345-11359, 2023.
    [    CrossRef] [Google Scholar] [Publisher Link]
  5. Qi Zhang et al., “Task Offloading and Resource Scheduling in Hybrid Edge-Cloud Networks,” IEEE Access, vol. 9, pp. 85350-85366, 2021.
    [    CrossRef] [Google Scholar] [Publisher Link]
  6. Yan Gu et al., “Cost-Aware Cloud Workflow Scheduling using DRL and Simulated Annealing,” Digital Communications and Networks, vol. 10, no. 6, pp. 1590-1599, 2024.
    [    CrossRef] [Google Scholar] [Publisher Link]
  7. Shiyao Ding, and Donghui Lin, “Dynamic Task Allocation for Cost-Efficient Edge Cloud Computing,” 2020 IEEE International Conference on Services Computing (SCC), Beijing, China, pp. 218-225, 2020.
    [    CrossRef] [Google Scholar] [Publisher Link]
  8. Arslan Qadeer, and Myung Jong Lee, “Deep-Deterministic Policy Gradient Based Multi-Resource Allocation in Edge-Cloud System: A Distributed Approach,” IEEE Access, vol. 11, pp. 20381-20398, 2023.
    [CrossRef] [Google Scholar] [Publisher Link]
  9. Gorka Nieto et al., “Deep Reinforcement Learning Techniques for Dynamic Task Offloading in the 5G Edge-Cloud Continuum,” Journal of Cloud Computing, vol. 13, no. 1, pp. 1-24, 2024.
    [    CrossRef] [Google Scholar] [Publisher Link]
  10. Thomas Dreibholz, and Somnath Mazumdar, “Towards a Lightweight Task Scheduling Framework for Cloud and Edge Platform,” Internet of Things, vol. 21, pp. 1-16, 2023.
    [    CrossRef] [Google Scholar] [Publisher Link]
  11. Vibha Jain, Bijendra Kumar, and Aditya Gupta, “Cybertwin-Driven Resource Allocation using Deep Reinforcement learning in 6G-Enabled Edge Environment,” Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 8, pp. 5708-5720, 2022.
    [    CrossRef] [Google Scholar] [Publisher Link]
  12. Tao Zheng et al., “Deep Reinforcement Learning-Based Workload Scheduling for Edge Computing,” Journal of Cloud Computing, vol. 11, no. 1, pp. 1-13, 2022.
    [    CrossRef] [Google Scholar] [Publisher Link]
  13. Xu Zhao et al., “Low Load DIDS Task Scheduling based on Q-Learning in Edge Computing Environment,” Journal of Network and Computer Applications, vol. 188, 2021.
    [    CrossRef] [Google Scholar] [Publisher Link]
  14. Shanchen Pang et al., “Minimize Average Tasks Processing Time in Satellite Mobile Edge Computing Systems via a Deep Reinforcement Learning Method,” Journal of Cloud Computing, vol. 12, pp. 1-29, 2023.
    [    CrossRef] [Google Scholar] [Publisher Link]
  15. Chunglae Cho et al., “QoS-Aware Workload Distribution in Hierarchical Edge Clouds: A Reinforcement Learning Approach,” IEEE Access, vol. 8, pp.193297-193313, 2020.
    [    CrossRef] [Google Scholar] [Publisher Link]
  16. FAN Qi, Li Zhuo, and Chen Xin, “Deep Reinforcement Learning Based Task Scheduling in Edge Computing Networks,” 2020 IEEE/CIC International Conference on Communications in China (ICCC), Chongqing, China, pp. 835-840, 2020.
    [    CrossRef] [Google Scholar] [Publisher Link]
  17. Reena Panwar, and M. Supriya, “RLPRAF: Reinforcement Learning-Based Proactive Resource Allocation Framework for Resource Provisioning in Cloud Environment,” IEEE Access, vol. 12, pp. 95986-96007, 2024.
    [    CrossRef] [Google Scholar] [Publisher Link]
  18. Hojjat Baghban et al., “Edge-AI: IoT Request Service Provisioning in Federated Edge Computing Using Actor-Critic Reinforcement Learning,” IEEE Transactions on Engineering Management, vol. 71, pp. 12519-12528, 2024.
    [    CrossRef] [Google Scholar] [Publisher Link]
  19. Haonan Wu, Xiumei Yang, and Zhiyong Bu, “Task Offloading with Service Migration for Satellite Edge Computing: A Deep Reinforcement Learning Approach,” IEEE Access, vol. 12, pp. 25844-25856, 2024.
    [    CrossRef] [Google Scholar] [Publisher Link]
  20. Zhuo Chen, Peihong Wei, and Yan Li, “Combining Neural Network-Based Method with Heuristic Policy for Optimal Task Scheduling in Hierarchical Edge Cloud,” Digital Communications and Networks, vol. 9, no. 3, pp. 688-697, 2023.
    [    CrossRef] [Google Scholar] [Publisher Link]
  21. Samuel Rac, and Mats Brorsson, “Cost-Aware Service Placement and Scheduling in the Edge-Cloud Continuum,” ACM Transactions on Architecture and Code Optimization, vol. 21, no. 2, pp. 1-24, 2024.
    [    CrossRef] [Google Scholar] [Publisher Link]
  22. Shuo Zhang et al., “Data-Intensive Workflow Scheduling Strategy Based on Deep Reinforcement Learning in Multi-Clouds,” Journal of Cloud Computing, vol. 12, no. 1, pp. 1-12, 2023.
    [    CrossRef] [Google Scholar] [Publisher Link]
  23. Shreshth Tuli et al., “Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments Using A3C Learning and Residual Recurrent Neural Networks,” IEEE Transactions on Mobile Computing, vol. 21, no. 3, pp. 940-954, 2022.
    [    CrossRef] [Google Scholar] [Publisher Link]
  24. Qi Qi et al., “Scalable Parallel Task Scheduling for Autonomous Driving Using Multi-Task Deep Reinforcement Learning,” IEEE Transactions on Vehicular Technology, vol. 69, no. 11, pp. 13861-13874, 2020.
    [    CrossRef] [Google Scholar] [Publisher Link]
  25. Bassem Sellami et al., “Deep Reinforcement Learning for Energy-Efficient Task Scheduling in SDN-based IoT Network,” 2020 IEEE 19th International Symposium on Network Computing and Applications (NCA), Cambridge, MA, USA, pp. 1-4, 2020.
    [    CrossRef] [Google Scholar] [Publisher Link]
  26. Xiaokang Zhou et al., “Edge-Enabled Two-Stage Scheduling Based on Deep Reinforcement Learning for Internet of Everything,” IEEE Internet of Things Journal, vol. 10, no. 4, pp. 3295-3304, 2023.
    [    CrossRef] [Google Scholar] [Publisher Link]
  27. S. Sudheer Mangalampalli et al., “Multi-Objective Prioritized Task Scheduler Using Improved Asynchronous Advantage Actor Critic (a3c) Algorithm in Multi Cloud Environment,” IEEE Access, vol. 12, pp. 11354-11377, 2024.
    [    CrossRef] [Google Scholar] [Publisher Link]
  28. Heba Nashaat et al., “DRL-Based Distributed Task Offloading Framework in Edge-Cloud Environment,” IEEE Access, vol. 12, pp. 33580-33594, 2024.
    [    CrossRef] [Google Scholar] [Publisher Link]
  29. Siyu Yuan et al., “Joint Optimization of DNN Partition and Continuous Task Scheduling for Digital Twin-Aided MEC Network With Deep Reinforcement Learning,” IEEE Access, vol. 11, pp. 27099-27110, 2023.
    [    CrossRef] [Google Scholar] [Publisher Link]
  30. Guanjin Qu et al., “DMRO: A Deep Meta Reinforcement Learning-Based Task Offloading Framework for Edge-Cloud Computing,” IEEE Transactions on Network and Service Management, vol. 18, no. 3, pp. 3448-3459, 2021.
    [    CrossRef] [Google Scholar] [Publisher Link]
  31. Aadharsh Roshan Nandhakumar et al., “EdgeAISim: A Toolkit for Simulation and Modelling of AI Models in Edge Computing Environments,” Measurement: Sensors, vol. 31, pp. 1-14, 2024.
    [    CrossRef] [Google Scholar] [Publisher Link]
  32. Mostafa Raeisi-Varzaneh et al., “Resource Scheduling in Edge Computing: Architecture, Taxonomy, Open Issues,” IEEE Access, vol. 11, pp. 25329-25350, 2023.
    [    CrossRef] [Google Scholar] [Publisher Link]
  33. Peng Liu et al., “Reinforcement Learning Empowered Multi-AGV Offloading Scheduling in Edge-Cloud IIoT,” Journal of Cloud Computing, vol. 11, pp. 1-14, 2022.
    [    CrossRef] [Google Scholar] [Publisher Link]
  34. Yunmeng Dong et al., “A High-Efficient Joint ’Cloud-Edge’ Aware Strategy for Task Deployment and Load Balancing,” IEEE Access, vol. 9, pp. 12791-12802, 2021.
    [    CrossRef] [Google Scholar] [Publisher Link]
  35. Shilin Wen et al., “Fast DRL-based Scheduler Configuration Tuning for Reducing Tail Latency in Edge-Cloud Jobs,” Journal of Cloud Computing, vol. 12, no. 1, pp. 1-32, 2023.
    [    CrossRef] [Google Scholar] [Publisher Link]
  36. Bassem Sellami et al., “Energy-Aware Task Scheduling and Offloading Using Deep Reinforcement Learning in SDN-Enabled IoT Network,” Computer Networks, 210, pp. 1-20, 2022.
    [    CrossRef] [Google Scholar] [Publisher Link]
  37. Lulu Chen et al., “IoT Microservice Deployment in Edge-Cloud Hybrid Environment Using Reinforcement Learning,” IEEE Internet of Things Journal, vol. 8, no. 6, pp. 12610-12622, 2021.
    [    CrossRef] [Google Scholar] [Publisher Link]
  38. Shida Lu et al., “QoS-Aware Task Scheduling in Cloud-Edge Environment,” IEEE Access, vol. 9, pp. 56496-56505, 2021.
    [    CrossRef] [Google Scholar] [Publisher Link]
  39. Shengli Pan et al., “Dependency-Aware Computation Offloading in Mobile Edge Computing: A Reinforcement Learning Approach,” IEEE Access, vol. 7, pp. 134742-134753, 2019.
    [    CrossRef] [Google Scholar] [Publisher Link]
  40. Manan Tomar et al., “Mirror Descent Policy Optimization,” arXiv preprint, pp. 1-21, 2020.
    [    CrossRef] [Google Scholar] [Publisher Link]
  41. Volodymyr Mnih et al., “Human-Level Control through Deep Reinforcement Learning,” Nature, vol. 518, pp. 529-533, 2015.
    [    CrossRef] [Google Scholar] [Publisher Link]