Conv-basis: A new paradigm for efficient attention inference and gradient computation in transformers

Y Liang, H Liu, Z Shi, Z Song, Z Xu, J Yin - arxiv preprint arxiv:2405.05219, 2024 - arxiv.org
The self-attention mechanism is the key to the success of transformers in recent Large
Language Models (LLMs). However, the quadratic computational cost $ O (n^ 2) $ in the …

[LIBRO][B] Numerical fourier analysis

G Plonka, D Potts, G Steidl, M Tasche - 2018 - Springer
The Applied and Numerical Harmonic Analysis (ANHA) book series aims to provide the
engineering, mathematical, and scientific communities with significant developments in …

Harnessing sparsity over the continuum: Atomic norm minimization for superresolution

Y Chi, MF Da Costa - IEEE Signal Processing Magazine, 2020 - ieeexplore.ieee.org
At the core of many sensing and imaging applications, the signal of interest can be modeled
as a linear superposition of translated or modulated versions of some template [eg, a point …

Super-resolution limit of the ESPRIT algorithm

W Li, W Liao, A Fannjiang - IEEE transactions on information …, 2020 - ieeexplore.ieee.org
The problem of imaging point objects can be formulated as estimation of an unknown atomic
measure from its M+ 1 consecutive noisy Fourier coefficients. The standard resolution of this …

Fourier circuits in neural networks: Unlocking the potential of large language models in mathematical reasoning and modular arithmetic

J Gu, C Li, Y Liang, Z Shi, Z Song… - arxiv preprint arxiv …, 2024 - openreview.net
In the evolving landscape of machine learning, a pivotal challenge lies in deciphering the
internal representations harnessed by neural networks and Transformers. Building on recent …

Super-resolution of point sources via convex programming

C Fernandez-Granda - Information and Inference: A Journal of …, 2016 - academic.oup.com
We consider the problem of recovering a signal consisting of a superposition of point
sources from low-resolution data with a cutoff frequency. If the distance between the sources …

Minimal pole representation and controlled analytic continuation of Matsubara response functions

L Zhang, E Gull - Physical Review B, 2024 - APS
Analytic continuation is a central step in the simulation of finite-temperature field theories in
which numerically obtained Matsubara data are continued to the real frequency axis for a …

[LIBRO][B] Algorithmic aspects of machine learning

A Moitra - 2018 - books.google.com
This book bridges theoretical computer science and machine learning by exploring what the
two sides can teach each other. It emphasizes the need for flexible, tractable models that …

Super-resolution of near-colliding point sources

D Batenkov, G Goldman… - Information and Inference …, 2021 - academic.oup.com
We consider the problem of stable recovery of sparse signals of the form from their spectral
measurements, known in a bandwidth with absolute error not exceeding. We consider the …

Over-the-air array calibration of mmWave phased array in beam-steering mode based on measured complex signals

Z Wang, F Zhang, H Gao, O Franek… - … on Antennas and …, 2021 - ieeexplore.ieee.org
Accurate knowledge of initial complex excitation coefficients for phased-array elements is
essential to ensure accurate array performance. In the literature, many array calibration …