Key-Value (KV) cache has become a bottleneck of LLMs for long-context generation.
Despite the numerous efforts in this area, the optimization for the decoding phase is …

Foundation models, such as large language models (LLMs) and large vision-language
models (LVLMs), have gained significant attention for their remarkable performance across …