The twenty-first century is presenting humankind with unprecedented environmental and
medical challenges. The ability to design novel proteins tailored for specific purposes would …

Deep learning is catalyzing a scientific revolution fueled by big data, accessible toolkits, and
powerful computational resources, impacting many fields, including protein structural …

Advances in artificial intelligence (AI) are fueling a new paradigm of discoveries in natural
sciences. Today, AI has started to advance natural sciences by improving, accelerating, and …

Learning effective protein representations is critical in a variety of tasks in biology such as
predicting protein function or structure. Existing approaches usually pretrain protein …

As opposed to scaling-up protein language models (PLMs), we seek improving performance
via protein-specific optimization. Although the proportionality between the language model …

Unsupervised protein language models trained across millions of diverse sequences learn
structure and function of proteins. Protein language models studied to date have been …

We investigate the capability of a transformer pretrained on natural language to generalize
to other modalities with minimal finetuning--in particular, without finetuning of the self …

The rapid increase in the number of proteins in sequence databases and the diversity of
their functions challenge computational approaches for automated function prediction. Here …

Protein language models have shown remarkable success in learning biological information
from protein sequences. However, most existing models are limited by either autoencoding …

In the field of artificial intelligence, a combination of scale in data and model capacity
enabled by unsupervised learning has led to major advances in representation learning and …