How efficiency shapes human language

E Gibson, R Futrell, SP Piantadosi, I Dautriche… - Trends in cognitive …, 2019 - cell.com
Cognitive science applies diverse tools and perspectives to study human language.
Recently, an exciting body of work has examined linguistic phenomena through the lens of …

Advances in natural language processing

J Hirschberg, CD Manning - Science, 2015 - science.org
Natural language processing employs computational techniques for the purpose of learning,
understanding, and producing human language content. Early computational approaches to …

Holistic evaluation of language models

P Liang, R Bommasani, T Lee, D Tsipras… - arxiv preprint arxiv …, 2022 - arxiv.org
Language models (LMs) are becoming the foundation for almost all major language
technologies, but their capabilities, limitations, and risks are not well understood. We present …

Semantic reconstruction of continuous language from non-invasive brain recordings

J Tang, A LeBel, S Jain, AG Huth - Nature Neuroscience, 2023 - nature.com
A brain–computer interface that decodes continuous language from non-invasive recordings
would have many scientific and practical applications. Currently, however, non-invasive …

On the opportunities and risks of foundation models

R Bommasani, DA Hudson, E Adeli, R Altman… - arxiv preprint arxiv …, 2021 - arxiv.org
AI is undergoing a paradigm shift with the rise of models (eg, BERT, DALL-E, GPT-3) that are
trained on broad data at scale and are adaptable to a wide range of downstream tasks. We …

Embers of autoregression show how large language models are shaped by the problem they are trained to solve

RT McCoy, S Yao, D Friedman, MD Hardy… - Proceedings of the …, 2024 - pnas.org
The widespread adoption of large language models (LLMs) makes it important to recognize
their strengths and limitations. We argue that to develop a holistic understanding of these …

Why does surprisal from larger transformer-based language models provide a poorer fit to human reading times?

BD Oh, W Schuler - Transactions of the Association for Computational …, 2023 - direct.mit.edu
This work presents a linguistic analysis into why larger Transformer-based pre-trained
language models with more parameters and lower perplexity nonetheless yield surprisal …

The neural architecture of language: Integrative modeling converges on predictive processing

M Schrimpf, IA Blank, G Tuckute… - Proceedings of the …, 2021 - National Acad Sciences
The neuroscience of perception has recently been revolutionized with an integrative
modeling approach in which computation, brain function, and behavior are linked across …

Holistic evaluation of language models

R Bommasani, P Liang, T Lee - … of the New York Academy of …, 2023 - Wiley Online Library
Abstract Language models (LMs) like GPT‐3, PaLM, and ChatGPT are the foundation for
almost all major language technologies, but their capabilities, limitations, and risks are not …

Large-scale evidence for logarithmic effects of word predictability on reading time

C Shain, C Meister, T Pimentel… - Proceedings of the …, 2024 - National Acad Sciences
During real-time language comprehension, our minds rapidly decode complex meanings
from sequences of words. The difficulty of doing so is known to be related to words' …