We propose an efficient method to ground pretrained text-only language models to the
visual domain, enabling them to process arbitrarily interleaved image-and-text data, and …

Storytelling and narrative are fundamental to human experience, intertwined with our social
and cultural engagement. As such, researchers have long attempted to create systems that …

Recent language models generate false but plausible-sounding text with surprising
frequency. Such “hallucinations” are an obstacle to the usability of language-based AI …

Although large language models (LLMs) have demonstrated impressive potential on simple
tasks, their breadth of scope, lack of transparency, and insufficient controllability can make …

The ability to convey relevant and faithful information is critical for many tasks in conditional
generation and yet remains elusive for neural seq-to-seq models whose outputs often reveal …

The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily
long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the …

Mamba, a special case of the State Space Model, is gaining popularity as an alternative to
template-based deep learning approaches in medical image analysis. While transformers …

Despite recent progress of pre-trained language models on generating fluent text, existing
methods still suffer from incoherence problems in long-form text generation tasks that …

Lexically constrained text generation aims to control the generated text by incorporating
some pre-specified keywords into the output. Previous work injects lexical constraints into …

Recent, rapid advances in deep generative models for protein design have focused on small
proteins with lots of data. Such models perform poorly on large proteins with limited natural …