Agerl Based Enhanced Map Reduce Technique in Cloud Scheduling

International Journal of Computer Science and Engineering

© 2016 by SSRG - IJCSE Journal

Volume 3 Issue 10

Year of Publication : 2016

Authors : S.Selvi, Dr.B.Kalaavathi

10.14445/23488387/IJCSE-V3I10P111

How to Cite?

S.Selvi, Dr.B.Kalaavathi, "Agerl Based Enhanced Map Reduce Technique in Cloud Scheduling," SSRG International Journal of Computer Science and Engineering , vol. 3,  no. 10, pp. 19-24, 2016. Crossref, https://doi.org/10.14445/23488387/IJCSE-V3I10P111

Abstract:

Today’s real time big data applications mostly rely on map-reduce (M-R) framework of Hadoop File System (HDFS). Hadoop makes the complexity of such applications in a simpler manner. This paper works on two goals: maximizing resource utilization and reducing the overall job completion time. Based on the goals proposed, we have developed Agent Centric Enhanced Reinforcement Learning Algorithm (AGERL) .The algorithm concentrates in four dimensions: variable partitioning of tasks, calculation of progress ratio of processing tasks including delays, XMPP based multi attribute query posting and Hopkins statistics assessment based dynamic cluster restructuring . An Enhanced Reinforcement Learning Process with the above features is employed to achieve the proposed goal. Finally performance gain is theoretically proved.

Keywords:

map reduce, Hopkins, multi attribute query, reinforcement learning.

References:

[1] Lena Mashayekhy, Mahyar Movahed Nejad, Daniel Grosu, Quan Zhang, Weisong Shi, ”Energy-Aware Scheduling of MapReduce Jobs for Big Data Applications”, IEEE Transactions on Parallel and Distributed Systems, vol. 25,September 2014.
[2] Liya Thomas, Syama R,” Survey on MapReduce Scheduling Algorithms”, International Journal of Computer Applications (0975 – 8887) Volume 95– No.23, June 2014.
[3] Zhuo Tang, Min Liu, Kenli Li, Keqin Li,” An Optimized MapReduce Workflow Scheduling Algorithm for Heterogeneous Computing”, The Journal of Supercomputing, Volume 72 Issue 6, June 2016,Pg no. 2059-2079.
[4] Joel Wolf, Deepak Rajan, Kirsten Hildrum, Rohit Khandekar, Vibhore Kumar, Sujay Parekh, Kun-Lung Wu and Andrey Balmin,” FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads”, ,” in Proc. ACM/IFIP/USENIX 11th Int. Conf. Middleware, 2010, pp. 1–20.
[5] Jord`a Polo, Claris Castillo, David Carrera, Yolanda Becerra, Ian Whalley, Malgorzata Steinder2, Jordi Torres, and Eduard Ayguade,” Resource-Aware Adaptive Scheduling for MapReduce Clusters”, Proceedings of 12th International Middleware Conference, Lisbon, Portugal, pg no. 187-207 ,December 12-16, 2011.
[6] Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy,Scott Shenker,Ion Stoica,” Job Scheduling for Multi- User MapReduce Clusters”, EECS Department University of California, Berkeley Technical Report No. UCB/EECS-2009-55,April 30,2009.
[7] R.Thangaselvi, S.Ananthbabu, R.Aruna,” An Efficient Mapreduce Scheduling Algorithm in Hadoop”, International Journal of Engineering Research & Science (IJOER), Vol-1, Issue- 9, December, 2015.
[8] J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S.-H. Bae, J. Qiu, and G. Fox, “Twister: A runtime for iterative MapReduce,” in Proc. 19th ACM Int. Symp. High Performance Distributed Computing, pg no.810–818,2010.
[9] A. Verma, L. Cherkasova, and R. H. Campbell, “ARIA: Automatic resource inference and allocation for MapReduce environments,”in Proc. 8th ACM Int. Conf. Autonomic Comput., 2011, pp. 235–244.
[10] Y. Song, Y. Sun, and W. Shi, “A two-tiered on-demand resource allocation mechanism for VM-based data centers,” IEEE Trans. Services Comput., vol. 6, no. 1, pp. 116–129, Jan. 2013.
[11] M. Pastorelli, A. Barbuzzi, D. Carra, M. Dell’Amico, and P. Michiardi,“HFSP: Size-based scheduling for Hadoop,” in Proc. IEEE Int. Conf. Big Data, 2013, pp. 51–59.
[12] Nenavath Srinivas Naik, Atul Negi, V. N. Sastry, “Performance Improvement of MapReduce Framework in Heterogeneous Context using Reinforcement Learning”, 2nd International Symposium on Big Data and Cloud Computing,2015.
[13]Gayathri, Selvi and Kalavathi “ An Efficient Performance and Monetary Cost Optimization on Resource Allocation in Cloud” International Journal of Innovative Science, Engineering & Technology, Vol. 2 Issue 5, May 2015.