From decoding to meta-generation: Inference-time algorithms for large language models

S Welleck, A Bertsch, M Finlayson… - arxiv preprint arxiv …, 2024 - arxiv.org
One of the most striking findings in modern research on large language models (LLMs) is
that scaling up compute during training leads to better results. However, less attention has …

Self-introspective decoding: Alleviating hallucinations for large vision-language models

F Huo, W Xu, Z Zhang, H Wang, Z Chen… - arxiv preprint arxiv …, 2024 - arxiv.org
While Large Vision-Language Models (LVLMs) have rapidly advanced in recent years, the
prevalent issue known as thehallucination'problem has emerged as a significant bottleneck …

Vacode: Visual augmented contrastive decoding

S Kim, B Cho, S Bae, S Ahn, SY Yun - arxiv preprint arxiv:2408.05337, 2024 - arxiv.org
Despite the astonishing performance of recent Large Vision-Language Models (LVLMs),
these models often generate inaccurate responses. To address this issue, previous studies …

Pseudo-RIS: Distinctive Pseudo-Supervision Generation for Referring Image Segmentation

S Yu, PH Seo, J Son - European Conference on Computer Vision, 2024 - Springer
We propose a new framework that automatically generates high-quality segmentation masks
with their referring expressions as pseudo supervisions for referring image segmentation …

Enhancing LLM Capabilities Beyond Scaling Up

W Yin, M Chen, R Zhang, B Zhou… - Proceedings of the …, 2024 - aclanthology.org
General-purpose large language models (LLMs) are progressively expanding both in scale
and access to unpublic training data. This has led to notable progress in a variety of AI …

Adversarial contrastive decoding: Boosting safety alignment of large language models via opposite prompt optimization

Z Zhao, X Zhang, K Xu, X Hu, R Zhang, Z Du… - arxiv preprint arxiv …, 2024 - arxiv.org
With the widespread application of Large Language Models (LLMs), it has become a
significant concern to ensure their safety and prevent harmful responses. While current safe …

Towards fast inference: Exploring and improving blockwise parallel drafts

T Kim, AT Suresh, K Papineni, M Riley, S Kumar… - arxiv preprint arxiv …, 2024 - arxiv.org
Despite the remarkable strides made by autoregressive language models, their potential is
often hampered by the slow inference speeds inherent in sequential token generation …

What if...?: Counterfactual inception to mitigate hallucination effects in large multimodal models

J Kim, YJ Kim, YM Ro - arxiv preprint arxiv:2403.13513, 2024 - arxiv.org
This paper presents a way of enhancing the reliability of Large Multimodal Models (LMMs) in
addressing hallucination effects, where models generate incorrect or unrelated responses …

T-REG: Preference Optimization with Token-Level Reward Regularization

W Zhou, S Zhang, L Zhao, T Meng - arxiv preprint arxiv:2412.02685, 2024 - arxiv.org
Reinforcement learning from human feedback (RLHF) has been crucial in aligning large
language models (LLMs) with human values. Traditionally, RLHF involves generating …

Pensieve: Retrospect-then-compare mitigates visual hallucination

D Yang, B Cao, G Chen, C Jiang - arxiv preprint arxiv:2403.14401, 2024 - arxiv.org
Multi-modal Large Language Models (MLLMs) demonstrate remarkable success across
various vision-language tasks. However, they suffer from visual hallucination, where the …