Glitch tokens in large language models: Categorization taxonomy and effective detection

Y Li, Y Liu, G Deng, Y Zhang, W Song, L Shi… - Proceedings of the …, 2024 - dl.acm.org
With the expanding application of Large Language Models (LLMs) in various domains, it
becomes imperative to comprehensively investigate their unforeseen behaviors and …

Cctest: Testing and repairing code completion systems

Z Li, C Wang, Z Liu, H Wang, D Chen… - 2023 IEEE/ACM 45th …, 2023 - ieeexplore.ieee.org
Code completion, a highly valuable topic in the software development domain, has been
increasingly promoted for use by recent advances in large language models (LLMs). To …

Improving machine translation systems via isotopic replacement

Z Sun, JM Zhang, Y **ong, M Harman… - Proceedings of the 44th …, 2022 - dl.acm.org
Machine translation plays an essential role in people's daily international communication.
However, machine translation systems are far from perfect. To tackle this problem …

Natural test generation for precise testing of question answering software

Q Shen, J Chen, JM Zhang, H Wang, S Liu… - Proceedings of the 37th …, 2022 - dl.acm.org
Question answering (QA) software uses information retrieval and natural language
processing techniques to automatically answer questions posed by humans in a natural …

Metamorphic testing of deep learning compilers

D **ao, Z Liu, Y Yuan, Q Pang, S Wang - Proceedings of the ACM on …, 2022 - dl.acm.org
The prosperous trend of deploying deep neural network (DNN) models to diverse hardware
platforms has boosted the development of deep learning (DL) compilers. DL compilers take …

Mttm: Metamorphic testing for textual content moderation software

W Wang, J Huang, W Wu, J Zhang… - 2023 IEEE/ACM 45th …, 2023 - ieeexplore.ieee.org
The exponential growth of social media platforms such as Twitter and Facebook has
revolutionized textual communication and textual content publication in human society …

A survey on large language models for software engineering

Q Zhang, C Fang, Y **e, Y Zhang, Y Yang… - arxiv preprint arxiv …, 2023 - arxiv.org
Software Engineering (SE) is the systematic design, development, and maintenance of
software applications, underpinning the digital infrastructure of our modern mainworld. Very …

Testing your question answering software via asking recursively

S Chen, S **, X **e - 2021 36th IEEE/ACM International …, 2021 - ieeexplore.ieee.org
Question Answering (QA) is an attractive and challenging area in NLP community. There are
diverse algorithms being proposed and various benchmark datasets with different topics and …

An image is worth a thousand toxic words: A metamorphic testing framework for content moderation software

W Wang, J Huang, J Huang, C Chen… - 2023 38th IEEE/ACM …, 2023 - ieeexplore.ieee.org
The exponential growth of social media platforms has brought about a revolution in
communication and content dissemination in human society. Nevertheless, these platforms …

Nmtsloth: understanding and testing efficiency degradation of neural machine translation systems

S Chen, C Liu, M Haque, Z Song, W Yang - Proceedings of the 30th ACM …, 2022 - dl.acm.org
Neural Machine Translation (NMT) systems have received much recent attention due to their
human-level accuracy. While existing works mostly focus on either improving accuracy or …