Eliminating position bias of language models: A mechanistic approach

Z Wang, H Zhang, X Li, KH Huang, C Han, S Ji… - arxiv preprint arxiv …, 2024 - arxiv.org
Position bias has proven to be a prevalent issue of modern language models (LMs), where
the models prioritize content based on its position within the given context. This bias often …

Tokenizing 3D Molecule Structure with Quantized Spherical Coordinates

K Gao, Y Wang, H Guan, Z Wang, Q Pei… - arxiv preprint arxiv …, 2024 - arxiv.org
The application of language models (LMs) to molecular structure generation using line
notations such as SMILES and SELFIES has been well-established in the field of …

Bio2Token: All-atom tokenization of any biomolecular structure with Mamba

A Liu, A Elaldi, N Russell, O Viessmann - arxiv preprint arxiv:2410.19110, 2024 - arxiv.org
Efficient encoding and representation of large 3D molecular structures with high fidelity is
critical for biomolecular design applications. Despite this, many representation learning …