Theoretical accounts of the N400 are divided as to whether the amplitude of the N400
response to a stimulus reflects the extent to which the stimulus was predicted, the extent to …

Tokenization is a foundational step in natural language processing (NLP) tasks, bridging
raw text and language models. Existing tokenization approaches like Byte-Pair Encoding …

The recent success of large language models (LLMs) has been predominantly driven by
curating the training dataset composition, scaling of model architectures and dataset sizes …

Despite it being the cornerstone of BPE, the most common tokenization algorithm, the
importance of compression in the tokenization process is still unclear. In this paper, we …

While subword tokenizers such as BPE and WordPiece are typically used to build
vocabularies for NLP models, the method of decoding text into a sequence of tokens from …

Subword tokenization has become the de-facto standard for tokenization, although
comparative evaluations of subword vocabulary quality across languages are scarce …

Tokenisation is the first step in almost all NLP tasks, and state-of-the-art transformer-based
language models all use subword tokenisation algorithms to process input text. Existing …

Natural language is composed of words, but modern LLMs process sub-words as input. A
natural question raised by this discrepancy is whether LLMs encode words internally, and if …

The role of subword segmentation in relation to capturing morphological patterns in LLMs is
currently not well explored. Ideally, one would train models like GPT using various …

Gender-inclusive NLP research has documented the harmful limitations of gender binary-
centric large language models (LLM), such as the inability to correctly use gender-diverse …