The remarkable performance of deep Convolutional neural networks (CNNs) is generally
attributed to their deeper and wider architectures, which can come with significant …

We consider the training of structured neural networks where the regularizer can be non-
smooth and possibly non-convex. While popular machine learning libraries have resorted to …

We introduce a new class of nonlinear models for functional data based on neural networks.
Deep learning has been very successful in nonlinear modeling, but there has been little …

We first propose a decentralized proximal stochastic gradient tracking method (DProxSGT)
for nonconvex stochastic composite problems, with data heterogeneously distributed on …

We propose a learning framework based on stochastic Bregman iterations, also known as
mirror descent, to train sparse neural networks with an inverse scale space approach. We …

Low-rank model compression is a widely used technique for reducing the computational
load when training machine learning models. However, existing methods often rely on …

The parameters of a neural network are naturally organized in groups, some of which might
not contribute to its overall performance. To prune out unimportant groups of parameters, we …

The leaky ReLU network with a group sparse regularization term has been widely used in
the recent years. However, training such network yields a nonsmooth nonconvex …

This paper proposes an algorithm (RMDA) for training neural networks (NNs) with a
regularization term for promoting desired structures. RMDA does not incur computation …

We study the training of regularized neural networks where the regularizer can be non-
smooth and non-convex. We propose a unified framework for stochastic proximal gradient …