Large Language Models (LLMs) have demonstrated remarkable capabilities in important
tasks such as natural language understanding and language generation, and thus have the …

The advent of machine learning (ML) and Artificial intelligence (AI) has brought about a
significant transformation across multiple industries, as it has facilitated the automation of …

Large foundation models, including large language models, vision transformers, diffusion,
and large language model based multimodal models, are revolutionizing the entire machine …

Large foundation models, including large language models (LLMs), vision transformers
(ViTs), diffusion, and LLM-based multimodal models, are revolutionizing the entire machine …

Pipeline parallelism is a key technique for training large language models within GPU
clusters. However, it often leads to a memory imbalance problem, where certain GPUs face …

Training of large-scale deep learning models necessitates parallelizing the model and data
across numerous devices, and the choice of parallelism strategy substantially depends on …

DNN models are becoming increasingly larger to achieve unprecedented accuracy, and the
accompanying increased computation and memory requirements necessitate the …

Foundation models have superior performance across a wide array of machine learning
tasks. The training of these models typically involves model parallelism (MP) to navigate the …

Deep learning is experiencing a rise in foundation models that are expected to lead in
various fields. The massive number of parameters necessitates the use of tensor model …

Although deep models have been widely explored in solving partial differential equations
(PDEs), previous works are primarily limited to data only with up to tens of thousands of …