The context window of large language models (LLMs) is rapidly increasing, leading to a
huge variance in resource usage between different requests as well as between different …

AI workloads, often hosted in multi-tenant cloud environments, require vast computational
resources but suffer inefficiencies due to limited tenant-provider coordination. Tenants lack …

Large Language Models (LLMs) are increasingly being deployed in applications such as
chatbots, code editors, and conversational agents. A key feature of LLMs is their ability to …

Recent advances in generative AI have led to large multi-modal models (LMMs) capable of
simultaneously processing inputs of various modalities such as text, images, video, and …

Large Language Models (LLMs) such as GPT-4 and Llama3 can already comprehend
complex commands and process diverse tasks. This advancement facilitates their …

Large language models (LLMs) have achieved remarkable performance and are widely
deployed in various applications, while the serving of LLM inference has raised concerns …