Context-Aware Scheduling for Apache Hadoop over Pervasive Environments

Guilherme Cassales 1 Andrea Schwertner Charão 1 Manuele Kirsch Pinheiro 2 Carine Souveyet 2 Luiz Angelo Steffenel 3
1 LSC - Laboratório de Sistemas de Computação
2 CRI - Centre de Recherche en Informatique de Paris 1
3 CRESTIC - Centre de Recherche en Sciences et Technologies de l'Information et de la Communication - EA 3804
Abstract : This article proposes to improve Apache Hadoop scheduling through the usage of context-awareness. Apache Hadoop is the most popular implementation of the MapReduce paradigm for distributed computing, but its design doesn't adapt automatically to computing nodes' context and capabilities. By introducing context-awareness into Hadoop, we intent to dynamically adapt its scheduling to the execution environment. This is a necessary feature in the context of pervasive grids, which are heterogeneous, dynamic and shared environments. The solution has been incorporated into Hadoop and evaluated through controlled experiments. The experiments demonstrate that context-awareness provides comparative performance gains, especially when part of the resources disappear during execution.
Keywords : Context-Awareness Map Reduce Apache Hadoop Task Scheduling Pervasive Grid Big Data
Document type :
Conference papers
Complete list of metadatas

Cited literature [18 references]  Display  Hide  Download
https://hal-paris1.archives-ouvertes.fr/hal-01196986
Contributor : Manuele Kirsch Pinheiro <>
Submitted on : Tuesday, September 15, 2015 - 12:28:37 PM
Last modification on : Friday, June 14, 2019 - 12:56:08 PM
Long-term archiving on : Tuesday, December 29, 2015 - 12:12:15 AM

File

1-s2.0-S1877050915008583-main....
Publication funded by an institution

Identifiers

Collections

UNIV-PARIS1 | CRI | URCA | CRESTIC | GRID5000

Citation

Guilherme Cassales, Andrea Schwertner Charão, Manuele Kirsch Pinheiro, Carine Souveyet, Luiz Angelo Steffenel. Context-Aware Scheduling for Apache Hadoop over Pervasive Environments. 6th International Conference on Ambient Systems, Networks and Technologies (ANT 2015), Jun 2015, Londres, United Kingdom. pp.202-209, ⟨10.1016/j.procs.2015.05.058⟩. ⟨hal-01196986⟩

Export

BibTeX TEI DC DCterms EndNote

Share

Metrics

Record views

179

Files downloads

359

Contact

support.ccsd.cnrs.fr
hal.support@ccsd.cnrs.fr
Logo CCSD