All bark and no bite: Rogue dimensions in transformer language models obscure representational quality

W Timkey, M Van Schijndel - arxiv preprint arxiv:2109.04404, 2021 - arxiv.org
Similarity measures are a vital tool for understanding how language models represent and
process language. Standard representational similarity measures such as cosine similarity …

Can transformer be too compositional? analysing idiom processing in neural machine translation

V Dankers, CG Lucas, I Titov - arxiv preprint arxiv:2205.15301, 2022 - arxiv.org
Unlike literal expressions, idioms' meanings do not directly follow from their parts, posing a
challenge for neural machine translation (NMT). NMT models are often unable to translate …

SemEval-2022 task 2: Multilingual idiomaticity detection and sentence embedding

HT Madabushi, E Gow-Smith, M Garcia… - arxiv preprint arxiv …, 2022 - arxiv.org
This paper presents the shared task on Multilingual Idiomaticity Detection and Sentence
Embedding, which consists of two subtasks:(a) a binary classification task aimed at …

AStitchInLanguageModels: Dataset and methods for the exploration of idiomaticity in pre-trained language models

HT Madabushi, E Gow-Smith, C Scarton… - arxiv preprint arxiv …, 2021 - arxiv.org
Despite their success in a variety of NLP tasks, pre-trained language models, due to their
heavy reliance on compositionality, fail in effectively capturing the meanings of multiword …

Construction grammar provides unique insight into neural language models

L Weissweiler, T He, N Otani, DR Mortensen… - arxiv preprint arxiv …, 2023 - arxiv.org
Construction Grammar (CxG) has recently been used as the basis for probing studies that
have investigated the performance of large pretrained language models (PLMs) with respect …

[PDF][PDF] Processamento de Linguagem Natural: conceitos, técnicas e aplicações em português

HM Caseli, MGV Nunes - 2024 - repositorio.usp.br
O Processamento de Linguagem Natural (PLN) surgiu praticamente ao mesmo tempo que
os computadores, por volta da década de 1940, já que a tradução automática entre línguas …

Semantics of Multiword Expressions in Transformer-Based Models: A Survey

F Miletić, SS Walde - … of the Association for Computational Linguistics, 2024 - direct.mit.edu
Multiword expressions (MWEs) are composed of multiple words and exhibit variable
degrees of compositionality. As such, their meanings are notoriously difficult to model, and it …

ID10M: Idiom identification in 10 languages

S Tedeschi, F Martelli, R Navigli - Findings of the Association for …, 2022 - aclanthology.org
Idioms are phrases which present a figurative meaning that cannot be (completely) derived
by looking at the meaning of their individual components. Identifying and understanding …

Distilling hypernymy relations from language models: On the effectiveness of zero-shot taxonomy induction

D Jain, LE Anke - arxiv preprint arxiv:2202.04876, 2022 - arxiv.org
In this paper, we analyze zero-shot taxonomy learning methods which are based on
distilling knowledge from language models via prompting and sentence scoring. We show …

A systematic search for compound semantics in pretrained BERT architectures

F Miletić, SS im Walde - Proceedings of the 17th Conference of the …, 2023 - aclanthology.org
To date, transformer-based models such as BERT have been less successful in predicting
compositionality of noun compounds than static word embeddings. This is likely related to a …