Ammus: A survey of transformer-based pretrained models in natural language processing
KS Kalyan, A Rajasekharan, S Sangeetha - arxiv preprint arxiv …, 2021 - arxiv.org
Transformer-based pretrained language models (T-PTLMs) have achieved great success in
almost every NLP task. The evolution of these models started with GPT and BERT. These …
almost every NLP task. The evolution of these models started with GPT and BERT. These …
Large language models for cyber security: A systematic literature review
The rapid advancement of Large Language Models (LLMs) has opened up new
opportunities for leveraging artificial intelligence in various domains, including cybersecurity …
opportunities for leveraging artificial intelligence in various domains, including cybersecurity …
Artificial intelligence for the metaverse: A survey
Along with the massive growth of the Internet from the 1990s until now, various innovative
technologies have been created to bring users breathtaking experiences with more virtual …
technologies have been created to bring users breathtaking experiences with more virtual …
Charformer: Fast character transformers via gradient-based subword tokenization
State-of-the-art models in natural language processing rely on separate rigid subword
tokenization algorithms, which limit their generalization ability and adaptation to new …
tokenization algorithms, which limit their generalization ability and adaptation to new …
Between words and characters: A brief history of open-vocabulary modeling and tokenization in NLP
What are the units of text that we want to model? From bytes to multi-word expressions, text
can be analyzed and generated at many granularities. Until recently, most natural language …
can be analyzed and generated at many granularities. Until recently, most natural language …
Better robustness by more coverage: Adversarial training with mixup augmentation for robust fine-tuning
Pretrained language models (PLMs) perform poorly under adversarial attacks. To improve
the adversarial robustness, adversarial data augmentation (ADA) has been widely adopted …
the adversarial robustness, adversarial data augmentation (ADA) has been widely adopted …
Analogy generation by prompting large language models: A case study of instructgpt
We propose a novel application of prompting Pre-trained Language Models (PLMs) to
generate analogies and study how to design effective prompts for two task settings …
generate analogies and study how to design effective prompts for two task settings …
Bridging the gap between indexing and retrieval for differentiable search index with query generation
The Differentiable Search Index (DSI) is an emerging paradigm for information retrieval.
Unlike traditional retrieval architectures where index and retrieval are two different and …
Unlike traditional retrieval architectures where index and retrieval are two different and …
PMANet: Malicious URL detection via post-trained language model guided multi-level feature attention network
The expansion of the Internet has led to the widespread proliferation of malicious URLs,
becoming a primary vector for cyber threats. Detecting malicious URLs is now essential for …
becoming a primary vector for cyber threats. Detecting malicious URLs is now essential for …
Square one bias in NLP: Towards a multi-dimensional exploration of the research manifold
The prototypical NLP experiment trains a standard architecture on labeled English data and
optimizes for accuracy, without accounting for other dimensions such as fairness …
optimizes for accuracy, without accounting for other dimensions such as fairness …