Ammus: A survey of transformer-based pretrained models in natural language processing

KS Kalyan, A Rajasekharan, S Sangeetha - ar** capable language models
(LM). Despite this, pretraining data design is critically under-documented and often guided …