In deep learning, different kinds of deep networks typically need different optimizers, which
have to be chosen after multiple trials, making the training process inefficient. To relieve this …

As a powerful modelling method, piecewise linear neural networks (PWLNNs) have proven
successful in various fields, most recently in deep learning. To apply PWLNN methods, both …

Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-
structured data. Recently, GCNs and subsequent variants have shown superior performance …

We provide the first global optimization landscape analysis of Neural Collapse--an intriguing
empirical phenomenon that arises in the last-layer classifiers and features of neural …

Deep neural networks (DNNs) have demonstrated dominating performance in many fields;
since AlexNet, networks used in practice are going wider and deeper. On the theoretical …

Recent works have cast some light on the mystery of why deep nets fit any data and
generalize despite being very overparametrized. This paper analyzes training and …

Gradient descent finds a global minimum in training deep neural networks despite the
objective function being non-convex. The current paper proves gradient descent achieves …

Super resolution (SR) methods typically assume that the low-resolution (LR) image was
downscaled from the unknown high-resolution (HR) image by a fixedideal'downscaling …

Diffusion models have recently revolutionized the field of image synthesis due to their ability
to generate photorealistic images. However one of the major drawbacks of diffusion models …

Abstract Spiking Neural Networks (SNNs) have recently attracted significant research
interest as the third generation of artificial neural networks that can enable low-power event …