Characterizing deep-learning I/O workloads in TensorFlow

SWD Chien, S Markidis, CP Sishtla… - 2018 IEEE/ACM 3rd …, 2018 - ieeexplore.ieee.org
The performance of Deep-Learning (DL) computing frameworks rely on the performance of
data ingestion and checkpointing. In fact, during the training, a considerable high number of …

End-to-end i/o portfolio for the summit supercomputing ecosystem

S Oral, SS Vazhkudai, F Wang, C Zimmer… - Proceedings of the …, 2019 - dl.acm.org
The I/O subsystem for the Summit supercomputer, No. 1 on the Top500 list, and its
ecosystem of analysis platforms is composed of two distinct layers, namely the in-system …

Access patterns and performance behaviors of multi-layer supercomputer i/o subsystems under production load

JL Bez, AM Karimi, AK Paul, B **e, S Byna… - Proceedings of the 31st …, 2022 - dl.acm.org
Scientific computing workloads at HPC facilities have been shifting from traditional
numerical simulations to AI/ML applications for training and inference while processing and …

Automating distributed tiered storage management in cluster computing

H Herodotou, E Kakoulli - arxiv preprint arxiv:1907.02394, 2019 - arxiv.org
Data-intensive platforms such as Hadoop and Spark are routinely used to process massive
amounts of data residing on distributed file systems like HDFS. Increasing memory sizes and …

{StreamCache}: Revisiting Page Cache for File Scanning on Fast Storage Devices

Z Li, G Zhang - 2024 USENIX Annual Technical Conference (USENIX …, 2024 - usenix.org
Buffered I/O via page cache is used for file scanning in many cases as page cache can
provide buffering, data aggregation, I/O alignment and prefetching transparently. However …

Flash-oriented Coded Storage: Research Status and Future Directions

Z Li, G Zhang, Y Wang - ACM Transactions on Storage, 2024 - dl.acm.org
Flash-based solid-state drives (SSDs) have been widely adopted in various storage
systems, manifesting better performance than their forerunner HDDs. However, the …

UMAMI: a recipe for generating meaningful metrics through holistic I/O performance analysis

GK Lockwood, W Yoo, S Byna, NJ Wright… - Proceedings of the 2nd …, 2017 - dl.acm.org
I/O efficiency is essential to productivity in scientific computing, especially as many scientific
domains become more data-intensive. Many characterization tools have been used to …

TOKIO on ClusterStor: Connecting standard tools to enable holistic I/O performance analysis

GK Lockwood, NJ Wright, S Snyder, P Carns, G Brown… - 2018 - escholarship.org
At present, I/O performance analysis requires different tools to characterize individual
components of the I/O subsystem, and institutional I/O expertise is relied upon to translate …

An empirical study of I/O separation for burst buffers in HPC systems

D Koo, J Lee, J Liu, EK Byun, JH Kwak… - Journal of Parallel and …, 2021 - Elsevier
To meet the exascale I/O requirements for the High-Performance Computing (HPC), a new
I/O subsystem, Burst Buffer, based on solid state drives (SSD), has been developed …

Performance characterization of scientific workflows for the optimal use of burst buffers

CS Daley, D Ghoshal, GK Lockwood, S Dosanjh… - Future Generation …, 2020 - Elsevier
Scientific discoveries are increasingly dependent upon the analysis of large volumes of data
from observations and simulations of complex phenomena. Scientists compose the complex …