Learning relus via gradient descent

M Soltanolkotabi - Advances in neural information …, 2017 - proceedings.neurips.cc
In this paper we study the problem of learning Rectified Linear Units (ReLUs) which are
functions of the form $\vct {x}\mapsto\max (0,\langle\vct {w},\vct {x}\rangle) $ with $\vct …

Linear neural network layers promote learning single-and multiple-index models

S Parkinson, G Ongie, R Willett - arxiv preprint arxiv:2305.15598, 2023 - arxiv.org
This paper explores the implicit bias of overparameterized neural networks of depth greater
than two layers. Our framework considers a family of networks of varying depths that all have …

Algebraic variety models for high-rank matrix completion

G Ongie, R Willett, RD Nowak… - … on Machine Learning, 2017 - proceedings.mlr.press
We consider a non-linear generalization of low-rank matrix completion to the case where the
data belongs to an algebraic variety, ie, each data point is a solution to a system of …

Non-iterative recovery from nonlinear observations using generative models

J Liu, Z Liu - Proceedings of the IEEE/CVF Conference on …, 2022 - openaccess.thecvf.com
In this paper, we aim to estimate the direction of an underlying signal from its nonlinear
observations following the semi-parametric single index model (SIM). Unlike for …

Least squares estimation in the monotone single index model

F Balabdaoui, C Durot, H Jankowski - Bernoulli, 2019 - projecteuclid.org
We study the monotone single index model where a real response variable $ Y $ is linked to
a $ d $-dimensional covariate $ X $ through the relationship $ E [Y| X]=\Psi_ {0}(\alpha^{T} …

Active learning for single neuron models with lipschitz non-linearities

A Gajjar, C Musco, C Hegde - International Conference on …, 2023 - proceedings.mlr.press
We consider the problem of active learning for single neuron models, also sometimes called
“ridge functions”, in the agnostic setting (under adversarial label noise). Such models have …

Inference in high-dimensional single-index models under symmetric designs

H Eftekhari, M Banerjee, Y Ritov - Journal of Machine Learning Research, 2021 - jmlr.org
The problem of statistical inference for regression coefficients in a high-dimensional single-
index model is considered. Under elliptical symmetry, the single index model can be …

Semiparametric efficiency in convexity constrained single-index model

AK Kuchibhotla, RK Patra, B Sen - Journal of the American …, 2023 - Taylor & Francis
We consider estimation and inference in a single-index regression model with an unknown
convex link function. We introduce a convex and Lipschitz constrained least-square …

Fast algorithms for demixing sparse signals from nonlinear observations

M Soltani, C Hegde - IEEE Transactions on Signal Processing, 2017 - ieeexplore.ieee.org
We study the problem of demixing a pair of sparse signals from noisy, nonlinear
observations of their superposition. Mathematically, we consider a nonlinear signal …

Misspecified phase retrieval with generative priors

Z Liu, X Wang, J Liu - Advances in Neural Information …, 2022 - proceedings.neurips.cc
In this paper, we study phase retrieval under model misspecification and generative priors.
In particular, we aim to estimate an $ n $-dimensional signal $\mathbf {x} $ from $ m $ iid …