Understanding AI systems' inner workings is critical for ensuring value alignment and safety.
This review explores mechanistic interpretability: reverse-engineering the computational …

As large language models (LLMs) grow more powerful, concerns around potential harms
like toxicity, unfairness, and hallucination threaten user trust. Ensuring beneficial alignment …

Deep learning models often suffer from a lack of interpretability due to polysemanticity,
where individual neurons are activated by multiple unrelated semantics, resulting in unclear …

Interpreting and controlling the internal mechanisms of large language models (LLMs) is
crucial for improving their trustworthiness and utility. Recent efforts have primarily focused …

Superposition refers to encoding representations of multiple features within a single neuron,
which is common in transformers. This property allows neurons to combine and represent …

Disentangling polysemantic neurons is at the core of many current approaches to
interpretability of large language models. Here we attempt to study how disentanglement …