High-performance design of YARN MapReduce on modern HPC clusters with Lustre and RDMA

M Wasi-ur-Rahman, X Lu, NS Islam… - 2015 IEEE …, 2015 - ieeexplore.ieee.org
The viability and benefits of running MapReduce over modern High Performance Computing
(HPC) clusters, with high performance interconnects and parallel file systems, have attracted …

A comprehensive study of MapReduce over lustre for intermediate data placement and shuffle strategies on HPC clusters

MD Wasi-ur-Rahman, NS Islam, X Lu… - IEEE Transactions on …, 2016 - ieeexplore.ieee.org
With high performance interconnects and parallel file systems, running MapReduce over
modern High Performance Computing (HPC) clusters has attracted much attention due to its …

Approaches of enhancing interoperations among high performance computing and big data analytics via augmentation

AR Pathak, M Pandey, SS Rautaray - Cluster Computing, 2020 - Springer
The dawn of exascale computing and its convergence with big data analytics has greatly
spurred research interests. The reasons are straightforward. Traditionally, high performance …

MR-Advisor: A comprehensive tuning, profiling, and prediction tool for MapReduce execution frameworks on HPC clusters

M Wasi-ur-Rahman, NS Islam, X Lu, D Shankar… - Journal of Parallel and …, 2018 - Elsevier
MapReduce is the most popular parallel computing framework for big data processing which
allows massive scalability across distributed computing environment. Advanced RDMA …

On a software-based self-test methodology and its application

CHP Wen, LC Wang, KT Cheng, K Yang… - 23rd IEEE VLSI Test …, 2005 - ieeexplore.ieee.org
Software-based self-test (SBST) was originally proposed for cost reduction in SOC test
environment. Previous studies have focused on using SBST for screening logic defects …

Boldio: A hybrid and resilient burst-buffer over lustre for accelerating big data i/o

D Shankar, X Lu, DK Panda - … Conference on Big Data (Big Data …, 2016 - ieeexplore.ieee.org
The limitation of local storage space in the HPC environments has placed an unprecedented
demand on the performance of the underlying shared parallel file systems. This has …

Maximizing the benefit of RDMA at end hosts

X Wang, H Song, CT Nguyen… - IEEE INFOCOM 2021 …, 2021 - ieeexplore.ieee.org
RDMA is increasingly deployed in data center to meet the demands of ultra-low latency, high
throughput and low CPU overhead. However, it is not easy to migrate existing applications …

Convergence of high performance computing, big data, and machine learning applications on containerized infrastructures

P Liu - 2023 - upcommons.upc.edu
(English) The convergence of High Performance Computing (HPC), Big Data (BD), and
Machine Learning (ML) in the computing continuum is being pursued in earnest across the …

MR-Advisor: A comprehensive tuning tool for advising HPC users to accelerate MapReduce applications on supercomputers

M Wasi-Ur-Rahman, NS Islam, X Lu… - 2016 28th …, 2016 - ieeexplore.ieee.org
MapReduce is the most popular parallel computing framework for big data processing which
allows massive scalability across distributed computing environment. Advanced RDMA …

Program simplification as a means of approximating undecidable propositions

M Harman, C Fox, R Hierons… - Proceedings Seventh …, 1999 - ieeexplore.ieee.org
We describe an approach which mixes testing, slicing, transformation and formal verification
to investigate speculative hypotheses concerning a program, formulated during program …