We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and
reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is …

The current trend of scaling language models involves increasing both parameter count and
training dataset size. Extrapolating this trend suggests that training dataset size may soon be …

Finetuning large language models (LLMs) on instructions leads to vast performance
improvements on natural language tasks. We apply instruction tuning using code …

We introduce MADLAD-400, a manually audited, general domain 3T token monolingual
dataset based on CommonCrawl, spanning 419 languages. We discuss the limitations …

Over the past few years, Large Language Models of Code (Code LLMs) have started to have
a significant impact on programming practice. Code LLMs are also emerging as building …

Pretrained large language models (LLMs) have significantly improved code generation. As
these models scale up, there is an increasing need for the output to handle more intricate …

Pre-trained language models like ChatGPT have significantly improved code generation. As
these models scale up, there is an increasing need for the output to handle more intricate …

Code understanding and generation have fast become some of the most popular
applications of language models (LMs). Nonetheless, research on multilingual aspects of …

Large language models (LLMs) have showcased remarkable skills, rivaling or even
exceeding human intelligence in certain areas. Their proficiency in translation is notable, as …

Low-Rank Adaptation (LoRA) has recently gained attention for fine-tuning foundation
models by incorporating trainable low-rank matrices, thereby reducing the number of …