Single Agent Learning Algorithms for Decision making in Diagnostic Applications

International Journal of Computer Science and Engineering

© 2016 by SSRG - IJCSE Journal

Volume 3 Issue 5

Year of Publication : 2016

Authors : Deepak A. Vidhate, Dr. Parag Kulkarni

10.14445/23488387/IJCSE-V3I5P109

How to Cite?

Deepak A. Vidhate, Dr. Parag Kulkarni, "Single Agent Learning Algorithms for Decision making in Diagnostic Applications," SSRG International Journal of Computer Science and Engineering , vol. 3,  no. 5, pp. 46-52 , 2016. Crossref, https://doi.org/10.14445/23488387/IJCSE-V3I5P109

Abstract:

The output of the system is a sequence of actions in some applications. There is no such measure as the best action in any in-between state; an action is excellent if it is part of a good policy. A single action is not important; the policy is important that is the sequence of correct actions to reach the goal. To be able to generate a policy the machine learning programs should able to assess the quality of policies and learn from past good action sequences. Learning is the basic capacity of intelligent agents. An agent changes its behaviour based on its previous experiences through learning. An intelligent agent must be formalized by knowledge and be able to act on this knowledge. In many single-agent systems for learning the policy of an agent in uncertain environments, the reinforcement learning techniques have been applied successfully. Many existing singleagent models for sequential decision making are derived from a general model and are distinguished by assumptions. Q-learning algorithms are used for this purpose. Single agent learning model is given in this paper. Four single agent reinforcement learning algorithms are implemented and results are compared. Single agent Q-learning Algorithm and Sarsa Learning Algorithm gives some results for the problem. However adding eligibility traces in single agent learning algorithms i.e. Q(λ) learning and Sarsa(λ) learning gives performs better than the previous algorithms. The paper shows the results of all four algorithms and performance comparisons among them.

Keywords:

Q-learning, Reinforcement learning, Sarsa Learning, Single Agent.

References:

[1] Babak Nadjar Araabi, Sahar Mastoureshgh, and Majid Nili Ahmadabadi ―A Study on Expertise of Agents and Its Effects on Cooperative Q-Learning IEEE Transactions on Evolutionary Computation, vol:14, pp:23-57, 2010
[2] Young-Cheol Choi, Student Member, Hyo-Sung Ahn ―A Survey on Multi-Agent Reinforcement Learning: Coordination Problems, IEEE/ASME International Conference on Mechatronics and Embedded Systems and Applications, pp. 81 – 86, 2010.
[3] Zahra Abbasi, Mohammad Ali Abbasi ―Reinforcement Distribution in a Team of Cooperative Q-learning Agent, Proceedings of the 9th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, IEEE Computer Society
[4] La-mei GAO, Jun ZENG, Jie WU, Min LI ―Cooperative Reinforcement Learning Algorithm to Distributed Power System based on Multi-Agent 2009 3rd International Conference on Power Electronics Systems and Applications Digital Reference: K210509035
[5] Adnan M. Al-Khatib ―Cooperative Machine Learning Method World of Computer Science and Information Technology Journal (WCSIT) ISSN: 2221-0741 Vol.1, No.9, 380-383, 2011.
[6] Ethem Alpaydin ―Introduction to Machine Learning Second Edition, MIT Press by PHI.
[7] Tom Mitchell ―Machine Learning McGraw Hill International Edition.
[8] Liviu Panait Sean Luke ―Cooperative Multi-Agent Learning: The State of the Art, published in Journal of Autonomous Agents and Multi-Agent Systems Volume 11 Issue 3, pp. 387 – 434, 05.
[9] Jun-Yuan Tao, De-Sheng Li ―Cooperative Strategy Learning In Multi-Agent Environment With Continuous State Space, IEEE International Conference on Machine Learning and Cybernetics, pp.2107 – 2111, 2006.
[10] Dr. Hamid R. Berenji David Vengerov ―Learning, Cooperation, and Coordination in Multi-Agent Systems, in Proceedings of 9th IEEE International Conference on Fuzzy Systems, 2000.
[11] M.V. Nagendra Prasad & Victor R. Lesser ―Learning Situation-Specific Coordination in Cooperative Multi-agent Systems‖ in Journal of Autonomous Agents and Multi-Agent Systems, Volume 2 Issue 2, pp. 173 – 207, 1999.
[12] Ronen Brafman & Moshe Tennenholtz ―Learning to Coordinate Efficiently: A Model-based Approach, in Journal of Artificial Intelligence Research, Volume 19 Issue 1, pp. 11-23, 2003.
[13] Michael Kinney & Costas Tsatsoulis ―Learning Communication Strategies in Multiagent Systems, in Journal of Applied Intelligence, Volume 9 Issue 1, pp 71-91, 1998.
[14] Chern Han Yong & Risto Miikkulainen ―Coevolution of Role-Based Cooperation in Multi-Agent Systems, in technical Report AI07-338, University of Texas at Austin, 2007.