Modern machine learning has subverted and bypassed the theoretical framework of
Chomsky's generative approach to linguistics, including its core claims to particular insights …

Transformer-based Large Language Models (LLMs) have become a fixture in modern
machine learning. Correspondingly, significant resources are allocated towards research …

For humans, language production and comprehension is sensitive to the hierarchical
structure of sentences. In natural language processing, past work has questioned how …

This chapter critically examines the potential contributions of modern language models to
theoretical linguistics. Despite their focus on engineering goals, these models' ability to …

Transformers trained on natural language data have been shown to learn its hierarchical
structure and generalize to sentences with unseen syntactic structures without explicitly …

Recursion is a prominent feature of human language, and fundamentally challenging for self-
attention due to the lack of an explicit recursive-state tracking mechanism. Consequently …

Accurate syntactic representations are essential for robust generalization in natural
language. Recent work has found that pre-training can teach language models to rely on …

Both humans and large language models are able to learn language without explicit
structural supervision. What inductive biases make this learning possible? We address this …

We reassess the Knowledge Neuron (KN) Thesis: an interpretation of the mechanism
underlying the ability of large language models to recall facts from a training corpus. This …

Transformer-based language models are effective but complex, and understanding their
inner workings is a significant challenge. Previous research has primarily explored how …