Attention layers, as commonly used in transformers, form the backbone of modern deep
learning, yet there is no mathematical description of their benefits and deficiencies as …

Graph Transformer (GT) recently has emerged as a new paradigm of graph learning
algorithms, outperforming the previously popular Message Passing Neural Network (MPNN) …

Set function learning has emerged as a crucial area in machine learning, addressing the
challenge of modeling functions that take sets as inputs. Unlike traditional machine learning …

A main object of our study is multiset functions—that is, permutation-invariant functions over
inputs of varying sizes. Deep Sets, proposed by Zaheer et al.(2017), provides a universal …

Set representation has become ubiquitous in deep learning for modeling the inductive bias
of neural networks that are insensitive to the input order. DeepSets is the most widely used …

Few neural architectures lend themselves to provable learning with gradient based
methods. One popular model is the single-index model, in which labels are produced by …

We study separations between two fundamental models (or\emph {Ans\" atze}) of
antisymmetric functions, that is, functions $ f $ of the form $ f (x_ {\sigma (1)},\ldots, x …

Abstract Message Passing Neural Networks (MPNN) has been the leading architecture for
machine learning on graphs. Its theoretical study focuses on increasing expressive power …

Symmetric functions, which take as input an unordered, fixed-size set, find practical
application in myriad physical settings based on indistinguishable points or particles, and …

Despite the widespread empirical success of deep neural networks over the past decade, a
comprehensive understanding of their mathematical properties remains elusive, which limits …