How efficiency shapes human language
Cognitive science applies diverse tools and perspectives to study human language.
Recently, an exciting body of work has examined linguistic phenomena through the lens of …
Recently, an exciting body of work has examined linguistic phenomena through the lens of …
Advances in natural language processing
Natural language processing employs computational techniques for the purpose of learning,
understanding, and producing human language content. Early computational approaches to …
understanding, and producing human language content. Early computational approaches to …
Holistic evaluation of language models
Language models (LMs) are becoming the foundation for almost all major language
technologies, but their capabilities, limitations, and risks are not well understood. We present …
technologies, but their capabilities, limitations, and risks are not well understood. We present …
Semantic reconstruction of continuous language from non-invasive brain recordings
A brain–computer interface that decodes continuous language from non-invasive recordings
would have many scientific and practical applications. Currently, however, non-invasive …
would have many scientific and practical applications. Currently, however, non-invasive …
On the opportunities and risks of foundation models
AI is undergoing a paradigm shift with the rise of models (eg, BERT, DALL-E, GPT-3) that are
trained on broad data at scale and are adaptable to a wide range of downstream tasks. We …
trained on broad data at scale and are adaptable to a wide range of downstream tasks. We …
Embers of autoregression show how large language models are shaped by the problem they are trained to solve
The widespread adoption of large language models (LLMs) makes it important to recognize
their strengths and limitations. We argue that to develop a holistic understanding of these …
their strengths and limitations. We argue that to develop a holistic understanding of these …
Why does surprisal from larger transformer-based language models provide a poorer fit to human reading times?
BD Oh, W Schuler - Transactions of the Association for Computational …, 2023 - direct.mit.edu
This work presents a linguistic analysis into why larger Transformer-based pre-trained
language models with more parameters and lower perplexity nonetheless yield surprisal …
language models with more parameters and lower perplexity nonetheless yield surprisal …
The neural architecture of language: Integrative modeling converges on predictive processing
The neuroscience of perception has recently been revolutionized with an integrative
modeling approach in which computation, brain function, and behavior are linked across …
modeling approach in which computation, brain function, and behavior are linked across …
Holistic evaluation of language models
Abstract Language models (LMs) like GPT‐3, PaLM, and ChatGPT are the foundation for
almost all major language technologies, but their capabilities, limitations, and risks are not …
almost all major language technologies, but their capabilities, limitations, and risks are not …
Large-scale evidence for logarithmic effects of word predictability on reading time
During real-time language comprehension, our minds rapidly decode complex meanings
from sequences of words. The difficulty of doing so is known to be related to words' …
from sequences of words. The difficulty of doing so is known to be related to words' …