Energy-efficient hadoop for big data analytics and computing: A systematic review and research insights

WT Wu, WW Lin, CH Hsu, LG He - Future Generation Computer Systems, 2018 - Elsevier
As the demands for big data analytics keep growing rapidly in scientific applications and
online services, MapReduce and its open-source implementation Hadoop gained popularity …

ishuffle: Improving hadoop performance with shuffle-on-write

Y Guo, J Rao, D Cheng, X Zhou - IEEE transactions on parallel …, 2016 - ieeexplore.ieee.org
Hadoop is a popular implementation of the MapReduce framework for running data-
intensive jobs on clusters of commodity servers. Shuffle, the all-to-all input data fetching …

Hopper: Decentralized speculation-aware cluster scheduling at scale

X Ren, G Ananthanarayanan, A Wierman… - Proceedings of the 2015 …, 2015 - dl.acm.org
As clusters continue to grow in size and complexity, providing scalable and predictable
performance is an increasingly important challenge. A crucial roadblock to achieving …

Fairness in resource allocation: Foundation and applications

HS Bin-Obaid, TB Trafalis - … , Data Mining, and Applications: NET, Moscow …, 2020 - Springer
This paper presents a comprehensive review of fairness in resource allocation and its
foundation. Fairness is applied when the resources divided on multiple demands are limited …

Coupling task progress for mapreduce resource-aware scheduling

J Tan, X Meng, L Zhang - 2013 Proceedings IEEE INFOCOM, 2013 - ieeexplore.ieee.org
Schedulers are critical in enhancing the performance of MapReduce/Hadoop in presence of
multiple jobs with different characteristics and performance goals. Though current …

Characterization and optimization of memory-resident MapReduce on HPC systems

Y Wang, R Goldstone, W Yu… - 2014 IEEE 28th …, 2014 - ieeexplore.ieee.org
MapReduce is a widely accepted framework for addressing big data challenges. Recently, it
has also gained broad attention from scientists at the US leadership computing facilities as a …

A trust-aware mechanism for cloud federation formation

L Mashayekhy, MM Nejad… - IEEE Transactions on …, 2019 - ieeexplore.ieee.org
Cloud providers can form cloud federations by pooling their resources together to balance
their loads, reduce their costs, and manage demand spikes. However, forming cloud …

Deadline-aware MapReduce job scheduling with dynamic resource availability

D Cheng, X Zhou, Y Xu, L Liu… - IEEE transactions on …, 2018 - ieeexplore.ieee.org
As MapReduce is becoming ubiquitous in large-scale data analysis, many recent studies
have shown that the performance of MapReduce could be improved by different job …

Reciprocal resource fairness: Towards cooperative multiple-resource fair sharing in iaas clouds

H Liu, B He - SC'14: Proceedings of the International …, 2014 - ieeexplore.ieee.org
Resource sharing in virtualized environments have been demonstrated significant benefits
to improve application performance and resource/energy efficiency. However, resource …

Enabling fast failure recovery in shared Hadoop clusters: towards failure-aware scheduling

O Yildiz, S Ibrahim, G Antoniu - Future Generation Computer Systems, 2017 - Elsevier
Hadoop emerged as the de facto state-of-the-art system for MapReduce-based data
analytics. The reliability of Hadoop systems depends in part on how well they handle …