The high-performance computing I/O stack has been complex due to multiple software
layers, the inter-dependencies among these layers, and the different performance tuning …

Datacenter-scale AI training clusters consisting of thousands of domain-specific accelerators
(DSA) are used to train increasingly-complex deep learning models. These clusters rely on a …

Training Deep Neural Networks (DNNs) is resource-intensive and time-consuming. While
prior research has explored many different ways of reducing DNN training time, the impact of …

Nowdays, it is prevalent to train deep learning (DL) models in cloud-native platforms that
actively leverage containerization and orchestration technologies for high elasticity, low and …

Deep learning training (DLT) applications exhibit unique I/O workload behaviors that pose
new challenges for storage system design. DLT is I/O intensive since data samples need to …

I/O is emerging as a major bottleneck for machine learning training, especially in distributed
environments. Indeed, at large scale, I/O takes as much as 85% of training time. Addressing …

We introduce Quiver, an informed storage cache for deep learning training (DLT) jobs in a
cluster of GPUs. Quiver employs domain-specific intelligence within the caching layer, to …

Parallel File Systems (PFSs) are frequently deployed on leadership High Performance
Computing (HPC) systems to ensure efficient I/O, persistent storage and scalable …

In the age of big data, deep learning has emerged as a powerful tool to extract insight and
exploit its value, both in industry and scientific applications. One common pattern emerging …

Stochastic gradient descent (SGD) is the most prevalent algorithm for training Deep Neural
Networks (DNN). SGD iterates the input data set in each training epoch processing data …