Author : B. Anusha 1
Date of Publication :20th November 2017
Abstract: In MapReduce, many task scheduling algorithms have been proposed to enhance data locality and to shorten task turnaround time, but most of them best awareness on scheduling Map tasks, as opposed to scheduling reduce responsibilities. Hence, using them in a digital MapReduce cluster may motive a low reduce-data locality. Besides, most of present scheduling algorithms are designed to gain the node locality and rack locality for traditional MapReduce clusters, in preference to reaching the VPS-locality and Cen locality for virtual MapReduce clusters. Consequently, adopting them in a virtual MapReduce cluster might be not able to provide an excessive map-data locality. In order to offer the correct scheduling scheme for a tenant to attain a high map-and-reduce information locality as well as enhance job performance in his/her virtual MapReduce cluster, on this paper we propose a hybrid job-driven scheduling scheme (JoSS) by way of presenting scheduling in 3 levels: job, Map task, and reduce task.
Reference :
-
- Ming-Chang Lee, Jia-Chun Lin, and Ramin Yahyapour, "Hybrid Job-Driven Scheduling for Virtual MapReduce Clusters", IEEE Transactions On Parallel And Distributed Systems, Vol. 27, No. 6, JUNE 2016.
- J. Dean and S. Ghemawat, “MapReduce: Simplified data processing on large clusters,” Commun. ACM, vol. 51, no. 1, pp. 107–113,2008.
- S. Chen and S. Schlosser, “Map-Reduce meets wider varieties ofapplications,” Intel Res., Santa Clara, CA, USA, Tech. Rep. IRPTR-08-05, 2008.
- B. White, T. Yeh, J. Lin, and L. Davis, “Web-scale computer visionusing mapreduce for multimedia data mining,” in Proc. 10th Int.Workshop Multimedia Data Mining, Jul. 2010, pp. 1–10
- Z. Guo, G. Fox, and M. Zhou, “Investigation of data locality inmapreduce,” in Proc. 12th IEEE/ACM Int. Symp. Cluster, Cloud GridComput., May 2012, pp. 419–426.
- C. He, Y. Lu, and D. Swanson, “Matchmaking: A new mapreducescheduling technique,” in Proc. IEEE 3rd Int. Conf. Cloud Comput.Technol. Sci., Nov. 2011, pp. 40–47.
- M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker,and I. Stoica, “Delay scheduling: A simple technique for achievinglocality and fairness in cluster scheduling,” in Proc. 5th Eur. Conf.Comput. Syst., Apr. 2010, pp. 265–278.
- X. Bu, J. Rao, and C.-Z. Xu, “Interference and locality-aware task scheduling for MapReduce applications in virtual clusters,” inProc. 22nd Int. Symp. High-Perform. Parallel Distrib. Comput., Jun.2013, pp. 227–238.
- C. Tian, H. Zhou, Y. He, and L. Zha, “A dynamic mapreducescheduler for heterogeneous workloads,” in Proc. IEEE 8th Int.Conf. Grid Cooperative Comput., 2009, pp. 218–224.
- J. Polo, D. Carrera, Y. Becerra, J. Torres, E. Ayguade, M. Steinder,and I. Whalley, “Performancedriven task co-scheduling for mapreduce environments,” in Proc. IEEE Netw. Oper. Manage. Symp.,2010, pp. 373–380.