Transformers play a central role in the inner workings of large language models. We
develop a mathematical framework for analyzing Transformers based on their interpretation …

Graph neural networks (GNNs) can process graphs of different sizes, but their ability to
generalize across sizes, specifically from small to large graphs, is still not well understood. In …

Attention based models such as Transformers involve pairwise interactions between data
points, modeled with a learnable attention matrix. Importantly, this attention matrix is …

In practice, encoding invariances into models improves sample complexity. In this work, we
study this phenomenon from a theoretical perspective. In particular, we provide minimax …

Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space
to model functions by neural networks as the curse of dimensionality (CoD) cannot be …

Self-attention and masked self-attention are at the heart of Transformers' outstanding
success. Still, our mathematical understanding of attention, in particular of its Lipschitz …

We consider universal approximations of symmetric and anti-symmetric functions, which are
important for applications in quantum physics, as well as other scientific and engineering …

We study the approximation of functions which are invariant with respect to certain
permutations of the input indices using flow maps of dynamical systems. Such invariant …

In this paper, we aim at establishing an approximation theory and a learning theory of
distribution regression via a fully connected neural network (FNN). In contrast to the classical …

We establish a deep learning theory for distribution regression with deep convolutional
neural networks (DCNNs). Deep learning based on structured deep neural networks has …