Transformers are slow and memory-hungry on long sequences, since the time and memory
complexity of self-attention are quadratic in sequence length. Approximate attention …

AlphaFold2 revolutionized structural biology with the ability to predict protein structures with
exceptionally high accuracy. Its implementation, however, lacks the code and data required …

Modeling long range dependencies in sequential data is a fundamental step towards
attaining human-level performance in many modalities such as text, vision, audio and video …

Comparing probability distributions is a fundamental problem in data sciences. Simple
norms and divergences such as the total variation and the relative entropy only compare …

Optimal transport has recently been reintroduced to the machine learning community thanks
in part to novel efficient optimization procedures allowing for medium to large scale …

Proteins' biological functions are defined by the geometric and chemical structure of their 3D
molecular surfaces. Recent works have shown that geometric deep learning can be used on …

A proper parametrization of state transition matrices of linear state-space models (SSMs)
followed by standard nonlinearities enables them to efficiently learn representations from …

Graph deep learning has recently emerged as a powerful ML concept allowing to generalize
successful deep neural architectures to non-euclidean structured data. Such methods have …

In these Lecture Notes, we provide a comprehensive introduction to the most recent
advances in the application of machine learning methods in quantum sciences. We cover …

Attention based models such as Transformers involve pairwise interactions between data
points, modeled with a learnable attention matrix. Importantly, this attention matrix is …