Scheduling MapReduce tasks on virtual MapReduce clusters from a tenant's perspective
2015 | conference paper. A publication of Göttingen
Jump to: Cite & Linked | Documents & Media | Details | Version history
Cite this publication
Scheduling MapReduce tasks on virtual MapReduce clusters from a tenant's perspective
Lin, J.-C.; Lee, M.-C. & Yahyapour, R. (2015)
2014 IEEE International Conference on Big Data (Big Data) pp. 141-146. IEEE International Conference on Big Data (Big Data), Washington, DC, USA. DOI: https://doi.org/10.1109/BigData.2014.7004223
Documents & Media
Details
- Authors
- Lin, Jia-Chun; Lee, Ming-Chang; Yahyapour, Ramin
- Abstract
- Renting a set of virtual private servers (VPSs for short) from a VPS provider to establish a virtual MapReduce cluster is cost-efficient for a company/organization. To shorten job turnaround time and keep data locality as high as possible in this type of environment, this paper proposes a Best-Fit Task Scheduling scheme (BFTS for short) from a tenant's perspective. BFTS schedules each map task to a VPS that can finish the task earlier than the other VPSs by predicting and comparing the time required by every VPS to retrieve the map-input data, execute the map task, and become idle in an online manner. Furthermore, BFTS schedules each reduce task to a VPS that is close to most VPSs that execute the related map tasks. We conduct extensive experiments to compare BFTS with several scheduling algorithms employed by Hadoop. The experimental results show that BFTS is better than the other tested algorithms in terms of map-data locality, reduce-data locality, and job turnaround time. The overhead incurred by BFTS is also evaluated, which is inevitable but acceptable compared with the other algorithms.
- Issue Date
- 2015
- Organization
- Gesellschaft für wissenschaftliche Datenverarbeitung
- Conference
- IEEE International Conference on Big Data (Big Data)
- ISBN
- 978-1-4799-5666-1
- Conference Place
- Washington, DC, USA
- Event start
- 2014-10-27
- Event end
- 2014-10-30
- Language
- English