Analysis methods in neural language processing: A survey

Y Belinkov, J Glass - … of the Association for Computational Linguistics, 2019 - direct.mit.edu
The field of natural language processing has seen impressive progress in recent years, with
neural network models replacing many of the traditional systems. A plethora of new models …

Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned

E Voita, D Talbot, F Moiseev, R Sennrich… - ar** layers of pre-trained transformer models
H Sajjad, F Dalvi, N Durrani, P Nakov - Computer Speech & Language, 2023 - Elsevier
Transformer-based NLP models are trained using hundreds of millions or even billions of
parameters, limiting their applicability in computationally constrained environments. While …

Identifying and controlling important neurons in neural machine translation

A Bau, Y Belinkov, H Sajjad, N Durrani, F Dalvi… - arxiv preprint arxiv …, 2018 - arxiv.org
Neural machine translation (NMT) models learn representations containing substantial
linguistic information. However, it is not clear if such information is fully distributed or if some …

Neuron-level interpretation of deep nlp models: A survey

H Sajjad, N Durrani, F Dalvi - Transactions of the Association for …, 2022 - direct.mit.edu
Abstract The proliferation of Deep Neural Networks in various domains has seen an
increased need for interpretability of these models. Preliminary work done along this line …