Pre-trained language models for text generation: A survey

J Li, T Tang, WX Zhao, JY Nie, JR Wen - ACM Computing Surveys, 2024 - dl.acm.org
Text Generation aims to produce plausible and readable text in human language from input
data. The resurgence of deep learning has greatly advanced this field, in particular, with the …

[HTML][HTML] Progress in machine translation

H Wang, H Wu, Z He, L Huang, KW Church - Engineering, 2022 - Elsevier
After more than 70 years of evolution, great achievements have been made in machine
translation. Especially in recent years, translation quality has been greatly improved with the …

Diffuseq: Sequence to sequence text generation with diffusion models

S Gong, M Li, J Feng, Z Wu, LP Kong - arxiv preprint arxiv:2210.08933, 2022 - arxiv.org
Recently, diffusion models have emerged as a new paradigm for generative models.
Despite the success in domains using continuous signals such as vision and audio …

Diffusion-lm improves controllable text generation

X Li, J Thickstun, I Gulrajani… - Advances in neural …, 2022 - proceedings.neurips.cc
Controlling the behavior of language models (LMs) without re-training is a major open
problem in natural language generation. While recent works have demonstrated successes …

Squeezellm: Dense-and-sparse quantization

S Kim, C Hooper, A Gholami, Z Dong, X Li… - arxiv preprint arxiv …, 2023 - arxiv.org
Generative Large Language Models (LLMs) have demonstrated remarkable results for a
wide range of tasks. However, deploying these models for inference has been a significant …

Maskgit: Masked generative image transformer

H Chang, H Zhang, L Jiang, C Liu… - Proceedings of the …, 2022 - openaccess.thecvf.com
Generative transformers have experienced rapid popularity growth in the computer vision
community in synthesizing high-fidelity and high-resolution images. The best generative …

Difusco: Graph-based diffusion solvers for combinatorial optimization

Z Sun, Y Yang - Advances in neural information processing …, 2023 - proceedings.neurips.cc
Abstract Neural network-based Combinatorial Optimization (CO) methods have shown
promising results in solving various NP-complete (NPC) problems without relying on hand …

Seamless: Multilingual Expressive and Streaming Speech Translation

L Barrault, YA Chung, MC Meglioli, D Dale… - arxiv preprint arxiv …, 2023 - arxiv.org
Large-scale automatic speech translation systems today lack key features that help machine-
mediated communication feel seamless when compared to human-to-human dialogue. In …

Transformers learn shortcuts to automata

B Liu, JT Ash, S Goel, A Krishnamurthy… - arxiv preprint arxiv …, 2022 - arxiv.org
Algorithmic reasoning requires capabilities which are most naturally understood through
recurrent models of computation, like the Turing machine. However, Transformer models …

Going deeper with image transformers

H Touvron, M Cord, A Sablayrolles… - Proceedings of the …, 2021 - openaccess.thecvf.com
Transformers have been recently adapted for large scale image classification, achieving
high scores shaking up the long supremacy of convolutional neural networks. However the …