Regularization of polynomial networks for image recognition

GG Chrysos, B Wang, J Deng… - Proceedings of the …, 2023‏ - openaccess.thecvf.com
Abstract Deep Neural Networks (DNNs) have obtained impressive performance across
tasks, however they still remain as black boxes, eg, hard to theoretically analyze. At the …

Deep neural networks tend to extrapolate predictably

K Kang, A Setlur, C Tomlin, S Levine - arxiv preprint arxiv:2310.00873, 2023‏ - arxiv.org
Conventional wisdom suggests that neural network predictions tend to be unpredictable and
overconfident when faced with out-of-distribution (OOD) inputs. Our work reassesses this …

Linear Complexity Self-Attention With Order Polynomials

F Babiloni, I Marras, J Deng, F Kokkinos… - … on Pattern Analysis …, 2023‏ - ieeexplore.ieee.org
Self-attention mechanisms and non-local blocks have become crucial building blocks for
state-of-the-art neural architectures thanks to their unparalleled ability in capturing long …

Comparing spectral bias and robustness for two-layer neural networks: Sgd vs adaptive random fourier features

A Kammonen, L Liang, A Pandey… - arxiv preprint arxiv …, 2024‏ - arxiv.org
We present experimental results highlighting two key differences resulting from the choice of
training algorithm for two-layer neural networks. The spectral bias of neural networks is well …

Quadratic residual multiplicative filter neural networks for efficient approximation of complex sensor signals

MU Demirezen - IEEE Access, 2023‏ - ieeexplore.ieee.org
In this research, we present an innovative Quadratic Residual Multiplicative Filter Neural
Network (QRMFNN) to effectively learn extremely complex sensor signals as a low …

Polynomial Implicit Neural Framework for Promoting Shape Awareness in Generative Models

U Nath, R Singh, A Shukla, K Kulkarni… - International Journal of …, 2024‏ - Springer
Polynomial functions have been employed to represent shape-related information in 2D and
3D computer vision, even from the very early days of the field. In this paper, we present a …

The extrapolation power of implicit models

J Decugis, AY Tsai, M Emerling, A Ganesh… - arxiv preprint arxiv …, 2024‏ - arxiv.org
In this paper, we investigate the extrapolation capabilities of implicit deep learning models in
handling unobserved data, where traditional deep neural networks may falter. Implicit …

TMPNN: high-order polynomial regression based on Taylor map factorization

A Ivanov, S Ailuro - Proceedings of the AAAI Conference on Artificial …, 2024‏ - ojs.aaai.org
The paper presents Taylor Map Polynomial Neural Network (TMPNN), a novel form of very
high-order polynomial regression, in which the same coefficients for a lower-to-moderate …

Polynomial Neural Barrier Certificate Synthesis of Hybrid Systems via Counterexample Guidance

H Zhao, B Liu, L Dehbi, H **e, Z Yang… - IEEE Transactions on …, 2024‏ - ieeexplore.ieee.org
This article presents a novel approach to the safety verification of hybrid systems by
synthesizing neural barrier certificates (BCs) via counterexample-guided neural network …

Architecture Design: From Neural Networks to Foundation Models

G Chrysos - 2024 IEEE 11th International Conference on Data …, 2024‏ - ieeexplore.ieee.org
Historically, we are taught to use task-dependent architecture design and objectives to
tackle data science tasks. Counter intuitively, this dogma has been proven (partly) wrong by …