With the increasing popularity of large deep learning model-serving workloads, there is a
pressing need to reduce the energy consumption of a model-serving cluster while …

The rapid evolution and widespread adoption of generative large language models (LLMs)
have made them a pivotal workload in various applications. Today, LLM inference clusters …

Large language model (LLM) serving is becoming an increasingly critical workload for cloud
providers. Existing LLM serving systems focus on interactive requests, such as chatbots and …

Today's large language models (LLMs) can solve challenging question-answering tasks,
and prompt engineering techniques, such as chain-of-thought (CoT), have gained attention …

The rising demand for generative large language models (LLMs) poses challenges for
thermal and power management in cloud datacenters. Traditional techniques often are …

The expanding context windows in large language models (LLMs) have greatly enhanced
their capabilities in various applications, but they also introduce significant challenges in …

Large Language Model (LLM) workloads have distinct prefill and decode phases with
different compute and memory requirements which should ideally be accounted for when …

Efficient scheduling is crucial for interactive Large Language Model (LLM) applications,
where low request completion time directly impacts user engagement. Size-based …

Augmented Large Language Models (LLMs) enhance the capabilities of standalone LLMs
by integrating external data sources through API calls. In interactive LLM applications …

As large language models (LLMs) grow in popularity for their diverse capabilities, improving
the efficiency of their inference systems has become increasingly critical. Batching LLM …