The llama 3 herd of models
Modern artificial intelligence (AI) systems are powered by foundation models. This paper
presents a new set of foundation models, called Llama 3. It is a herd of language models …
presents a new set of foundation models, called Llama 3. It is a herd of language models …
Ladder: Enabling Efficient {Low-Precision} Deep Learning Computing through Hardware-aware Tensor Transformation
The increasing demand for improving deep learning model performance has led to a
paradigm shift in supporting low-precision computation to harness the robustness of deep …
paradigm shift in supporting low-precision computation to harness the robustness of deep …
Language models scale reliably with over-training and on downstream tasks
Scaling laws are useful guides for derisking expensive training runs, as they predict
performance of large models using cheaper, small-scale experiments. However, there …
performance of large models using cheaper, small-scale experiments. However, there …
Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving
Solving mathematical problems requires advanced reasoning abilities and presents notable
challenges for large language models. Previous works usually synthesize data from …
challenges for large language models. Previous works usually synthesize data from …
Allo: A programming model for composable accelerator design
Special-purpose hardware accelerators are increasingly pivotal for sustaining performance
improvements in emerging applications, especially as the benefits of technology scaling …
improvements in emerging applications, especially as the benefits of technology scaling …
Smart parallel automated cryo-electron tomography
F Eisenstein, Y Fukuda, R Danev - Nature Methods, 2024 - nature.com
In situ cryo-electron tomography enables investigation of macromolecules in their native
cellular environment. Samples have become more readily available owing to recent …
cellular environment. Samples have become more readily available owing to recent …
NVILA: Efficient frontier visual language models
Visual language models (VLMs) have made significant advances in accuracy in recent
years. However, their efficiency has received much less attention. This paper introduces …
years. However, their efficiency has received much less attention. This paper introduces …
[HTML][HTML] Advancing state of health estimation for electric vehicles: Transformer-based approach leveraging real-world data
K Nakano, S Vögler, K Tanaka - Advances in Applied Energy, 2024 - Elsevier
The widespread adoption of electric vehicles (EVs) underscores the urgent need for
innovative approaches to estimate their lithium-ion batteries' state of health (SOH), which is …
innovative approaches to estimate their lithium-ion batteries' state of health (SOH), which is …
Liger kernel: Efficient triton kernels for llm training
Training Large Language Models (LLMs) efficiently at scale presents a formidable
challenge, driven by their ever-increasing computational demands and the need for …
challenge, driven by their ever-increasing computational demands and the need for …
Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference
Encoder-only transformer models such as BERT offer a great performance-size tradeoff for
retrieval and classification tasks with respect to larger decoder-only models. Despite being …
retrieval and classification tasks with respect to larger decoder-only models. Despite being …