Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs

[Free GPT-4]
[DeepSeek]
[PDF] arxiv.org

How does quantization affect multilingual LLMs?

K Marchisio, S Dash, H Chen, D Aumiller… - arxiv preprint arxiv …, 2024 - arxiv.org
Quantization techniques are widely used to improve inference speed and deployment of
large language models. While a wide body of work examines the impact of quantization on …
Enregistrer Citer Cité 5 fois Autres articles Les 4 versionsFree GPT-4 DeepSeek Version HTML
[Free GPT-4]
[DeepSeek]
[PDF] arxiv.org

VcLLM: Video Codecs are Secretly Tensor Codecs

C Xu, Y Wu, X Yang, B Chen, M Lentz, D Zhuo… - arxiv preprint arxiv …, 2024 - arxiv.org
As the parameter size of large language models (LLMs) continues to expand, the need for a
large memory footprint and high communication bandwidth have become significant …
Enregistrer Citer Cité 1 fois Autres articles Les 4 versionsFree GPT-4 DeepSeek Version HTML