Prompting gpt-3 to be reliable
Large language models (LLMs) show impressive abilities via few-shot prompting.
Commercialized APIs such as OpenAI GPT-3 further increase their use in real-world …
Commercialized APIs such as OpenAI GPT-3 further increase their use in real-world …
A primer in BERTology: What we know about how BERT works
Transformer-based models have pushed state of the art in many areas of NLP, but our
understanding of what is behind their success is still limited. This paper is the first survey of …
understanding of what is behind their success is still limited. This paper is the first survey of …
Shortcut learning of large language models in natural language understanding
Shortcut Learning of Large Language Models in Natural Language Understanding Page 1 110
COMMUNICATIONS OF THE ACM | JANUARY 2024 | VOL. 67 | NO. 1 research IMA GE B Y …
COMMUNICATIONS OF THE ACM | JANUARY 2024 | VOL. 67 | NO. 1 research IMA GE B Y …
Out of order: How important is the sequential order of words in a sentence in natural language understanding tasks?
Do state-of-the-art natural language understanding models care about word order-one of the
most important characteristics of a sequence? Not always! We found 75% to 90% of the …
most important characteristics of a sequence? Not always! We found 75% to 90% of the …
Towards interpreting and mitigating shortcut learning behavior of NLU models
Recent studies indicate that NLU models are prone to rely on shortcut features for prediction,
without achieving true language understanding. As a result, these models fail to generalize …
without achieving true language understanding. As a result, these models fail to generalize …
A survey on measuring and mitigating reasoning shortcuts in machine reading comprehension
The issue of shortcut learning is widely known in NLP and has been an important research
focus in recent years. Unintended correlations in the data enable models to easily solve …
focus in recent years. Unintended correlations in the data enable models to easily solve …
How effective is BERT without word ordering? implications for language understanding and data privacy
Ordered word sequences contain the rich structures that define language. However, it's often
not clear if or how modern pretrained language models utilize these structures. We show …
not clear if or how modern pretrained language models utilize these structures. We show …
Why machine reading comprehension models learn shortcuts?
Recent studies report that many machine reading comprehension (MRC) models can
perform closely to or even better than humans on benchmark datasets. However, existing …
perform closely to or even better than humans on benchmark datasets. However, existing …
Reasoning about goals, steps, and temporal ordering with WikiHow
We propose a suite of reasoning tasks on two types of relations between procedural events:
goal-step relations (" learn poses" is a step in the larger goal of" doing yoga") and step-step …
goal-step relations (" learn poses" is a step in the larger goal of" doing yoga") and step-step …
Moral concerns are differentially observable in language
Abstract Language is a psychologically rich medium for human expression and
communication. While language usage has been shown to be a window into various …
communication. While language usage has been shown to be a window into various …