[PDF][PDF] Language model behavior: A comprehensive survey

TA Chang, BK Bergen - Computational Linguistics, 2024 - direct.mit.edu
Transformer language models have received widespread public attention, yet their
generated text is often surprising even to NLP researchers. In this survey, we discuss over …

Gender and representation bias in GPT-3 generated stories

L Lucy, D Bamman - Proceedings of the third workshop on …, 2021 - aclanthology.org
Using topic modeling and lexicon-based word similarity, we find that stories generated by
GPT-3 exhibit many known gender stereotypes. Generated stories depict different topics and …

Stereoty** Norwegian salmon: An inventory of pitfalls in fairness benchmark datasets

SL Blodgett, G Lopez, A Olteanu, R Sim… - Proceedings of the …, 2021 - aclanthology.org
Auditing NLP systems for computational harms like surfacing stereotypes is an elusive goal.
Several recent efforts have focused on benchmark datasets consisting of pairs of contrastive …

Entity-based knowledge conflicts in question answering

S Longpre, K Perisetla, A Chen, N Ramesh… - arxiv preprint arxiv …, 2021 - arxiv.org
Knowledge-dependent tasks typically use two sources of knowledge: parametric, learned at
training time, and contextual, given as a passage at inference time. To understand how …

Picking on the same person: Does algorithmic monoculture lead to outcome homogenization?

R Bommasani, KA Creel, A Kumar… - Advances in …, 2022 - proceedings.neurips.cc
As the scope of machine learning broadens, we observe a recurring theme of algorithmic
monoculture: the same systems, or systems that share components (eg datasets, models) …

Towards controllable biases in language generation

E Sheng, KW Chang, P Natarajan, N Peng - arxiv preprint arxiv …, 2020 - arxiv.org
We present a general approach towards controllable societal biases in natural language
generation (NLG). Building upon the idea of adversarial triggers, we develop a method to …

Demystifying ChatGPT: An in-depth survey of OpenAI's robust large language models

P Bhattacharya, VK Prasad, A Verma, D Gupta… - … Methods in Engineering, 2024 - Springer
Recent advancements in natural language processing (NLP) have catalyzed the
development of models capable of generating coherent and contextually relevant …

On measures of biases and harms in NLP

S Dev, E Sheng, J Zhao, A Amstutz, J Sun… - arxiv preprint arxiv …, 2021 - arxiv.org
Recent studies show that Natural Language Processing (NLP) technologies propagate
societal biases about demographic groups associated with attributes such as gender, race …

Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?

H An, C Acquaye, C Wang, Z Li, R Rudinger - arxiv preprint arxiv …, 2024 - arxiv.org
We examine whether large language models (LLMs) exhibit race-and gender-based name
discrimination in hiring decisions, similar to classic findings in the social sciences (Bertrand …

Measuring and mitigating name biases in neural machine translation

J Wang, B Rubinstein, T Cohn - … of the 60th Annual Meeting of the …, 2022 - aclanthology.org
Abstract Neural Machine Translation (NMT) systems exhibit problematic biases, such as
stereotypical gender bias in the translation of occupation terms into languages with …