Learning representations that generalize under distribution shifts is critical for building
robust machine learning models. However, despite significant efforts in recent years …

We analyze identifiability as a possible explanation for the ubiquity of linear properties
across language models, such as the vector difference between the representations of" …

Motivated by interpretability and reliability, we investigate how neural networks represent
knowledge during graph learning, We find hints of universality, where equivalent …

In this paper, we introduce** harmonic loss** as an alternative to the standard cross-entropy
loss for training neural networks and large language models (LLMs). Harmonic loss enables …

Entity tracking is essential for complex reasoning. To perform in-context entity tracking,
language models (LMs) must bind an entity to its attribute (eg, bind a container to its content) …

The association between language and (non-linguistic) thinking ability in humans has long
been debated, and recently, neuroscientific evidence of brain activity patterns has been …

This study investigates the internal reasoning mechanism of language models during
symbolic multi-step reasoning, motivated by the question of whether chain-of-thought (CoT) …