Research and Application Deep Q-Network Algorithm for Automatic Navigation for Omnidirectional Mobile Robots

International Journal of Electronics and Communication Engineering

© 2021 by SSRG - IJECE Journal

Volume 8 Issue 1

Year of Publication : 2021

Authors : Tran Thi Huong, Pham Thi Thu Ha, Pham Van Bang

10.14445/23488549/IJECE-V8I1P103

How to Cite?

Tran Thi Huong, Pham Thi Thu Ha, Pham Van Bang, "Research and Application Deep Q-Network Algorithm for Automatic Navigation for Omnidirectional Mobile Robots," SSRG International Journal of Electronics and Communication Engineering, vol. 8,  no. 1, pp. 12-17, 2021. Crossref, https://doi.org/10.14445/23488549/IJECE-V8I1P103

Abstract:

In this paper, the setting up of the Deep Q-Network (DQN) algorithm in the mobile robot's Gazebo simulation environment has been calculated, designed, and controlled. Building experiments aim to make the robot model learn the best actions to control and navigate in an environment with many obstacles. For the mobile robot to move in an obstacle environment, the robot will then automatically control to avoid these obstacles. After that, the robot can remain within a specific limit, the more rewards it accumulates. The authors have performed various tests with many parameters and demonstrated the performance curves on the simulation. The research results will be the basis for designing and establishing control algorithms for current and future mobile robots for application in programming techniques and automation control in industrial control. 

Keywords:

Artificial intelligence, mobile robots, Obstacle robots, autonomous navigation, reinforcement learning, deep Q-learning.

References:

[1] Nguyen Thanh Tuan, Base Deep learning, The Legrand Orange Book. Version 2, last update, August 2020.
[2] Charu C. Aggarwal, Neural Networks and Deep Learning, Springer International Publishing AG, part of Springer Nature, 2018.
[3] Vu Thi Thuy Nga, Ong Xuan Loc, Trinh Hai Nam, Enhanced learning in automatic control with Matlab Simulink, Hanoi Polytechnic Publishing House, 2020.
[4] X. Ruan, D. Ren, X. Zhu, and J. Huang, Mobile Robot Navigation based on Deep Reinforcement Learning, Chinese Control And Decision Conference (CCDC), 2019.
[5] Han, J., and Seo, Y., Mobile robot path planning with surrounding point set and path improvement, Appl. Soft Comp. 57, 35–47.
[6] V. Matt and N. Aran, Deep reinforcement learning approach to autonomous driving, ed: arXiv, 2017.
[7] Li, G., and Chou, W, Path planning for mobile robot using self-adaptive learning particle swarm optimization, Sci. China Inform. Sci. 61, 052204–052213. doi: 10.1007/s11432-016-9115-2, 2018.
[8] Andrea Bacciotti, Stability and Control of Linear Systems, Publishing Ltd; Springer Nature Switzerland AG, 2019.
[9] Bakdi, A., Hentout, A., Boutami, H., Maoudj, A., Hachour, O., and Bouzouia, B., Optimal path planning and execution for mobile robots using genetic algorithm and adaptive fuzzy-logic control, Robot. Autonomous Syst. 89, (2017) 95–109. doi:10.1016/j.robot.2016.12.008.
[10] I. Zamora, N. G. Lopez, V. M. Vilches, and A. H. Cordero, Extending the open-air gym for robotics: A toolkit for reinforcement learning using ros and gazebo, arXiv preprint arXiv:1608.05742, 2016.
[11] Gu, S.; Holly, E.; Lillicrap, T.; Levine, S. Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Marina Bay Sands, Singapore, 3389–3396, 29 (2017).
[12] A. D. Pambudi, T. Agustinah, and R. Effendi, Reinforcement Point and Fuzzy Input Design of Fuzzy Q-Learning for Mobile Robot Navigation System, 2019 International Conference of Artificial Intelligence and Information Technology (ICAIIT), 2019.
[13] Do Quang Hiep, Ngo Manh Tien, Nguyen Manh Cuong, Pham Tien Dung, Tran Van Manh, Nguyen Tien Kiem, Nguyen Duc Duy, An Approach to Design Navigation System for Omnidirectional Mobile Robot Based on ROS, (IJMERR); 11(9) (2020) 1502-1508.
[14] X. Ruan, D. Ren, X. Zhu, and J. Huang, Mobile Robot Navigation based on Deep Reinforcement Learning, Chinese Control And Decision Conference (CCDC), 2019.
[15] N. Navarro-Guerrero, C. Weber, P. Schroeter, and S. Wermter, Real-world reinforcement learning for an autonomous humanoid robot, Robotics and Autonomous Systems, 2012.
[16] Saleem, Y.; Yau, K.L.A.; Mohamad, H.; Ramli, N.; Rehmani, M.H.;
Ni, Q. Clustering and Reinforcement Learning-Based Routing for Cognitive Radio Networks. IEEE Wirel. Commun. 2017.
[17] Z. Miljković, M. Mitić, M. Lazarević, and B. Babić, Neural network reinforcement learning for visual control of robot manipulators, Expert Systems with Applications, 40 (2013) 1721–1736.
[18] Pham Ngoc Sam, Tran Duc Chuyen, Research and Designing a Positioning System, Timeline Chemical Mapping for Multi-Direction Mobile Robot, 7(11) (2020) 7-12, Publishing by SSRG - IJECE Journal.
[19] Shota Ohnishi, Eiji Uchibe, Yotaro Yamaguchi, Kosuke Nakanishi, Yuji Yasui, and Shin Ishii, constrained Deep Q-Learning Gradually Approaching Ordinary Q-Learning, 13 (2019) 7-12, Publishing by Frontiers in Neurorobotics Journal, December.
[20] Roan Van Hoa, L. K. Lai, Le Thi Hoan, Mobile Robot Navigation Using Deep Reinforcement Learning in Unknown Environments, SSRG International Journal of Electrical and Electronics Engineering (SSRG-IJEEE), 7(8) (2020) 15-20.
[21] Song Han, Luo Li and Xinbin Li Deep Q-Network-Based Cooperative Transmission Joint Strategy Optimization Algorithm for Energy Harvesting-Powered Underwater Acoustic Sensor Networks, Sensors (Basel) 2020 Nov; 20(22): 6519. Published online 2020 November 14.
[22] Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.A.; Veness, J.; Bellemare, M.G.; Graves, A.; Riedmiller, M.; Fidjeland, A.K.; Ostrovski, G.; et al. Human-level control through deep reinforcement learning. Nature 2015.
[23] Fu X, Du J, Guo Y, Liu M, Dong T, et al. A machine learning framework for stock selection. arXiv, cited 2018 August 2018.
[24] Ganggang Guo, Yulei Rao, Feida Zhu, Fang Xu, Innovative deep matching algorithm for stock portfolio selection using deep stock profiles, PLoS One. 15(11) (2020), Published online November.
[25] Wu, Y.; Tan, H.; Peng, J.; Zhang, H.; He, H. Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus. Appl. 247, (2019) 454-466, Energy.