OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure

[Free GPT-4]
[DeepSeek]
[PDF] arxiv.org

AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures

S Zhang, H Wang, D Ma, Z Zhu, L Chen, K Lan… - arxiv preprint arxiv …, 2024 - arxiv.org
Speculative Decoding (SD) is a popular lossless technique for accelerating the inference of
Large Language Models (LLMs). We show that the decoding speed of SD frameworks with …
저장 인용 관련 학술자료 전체 2개의 버전 HTML 버전