This survey describes probabilistic algorithms for linear algebraic computations, such as
factorizing matrices and solving linear systems. It focuses on techniques that have a proven …

Using backpropagation to compute gradients of objective functions for optimization has
remained a mainstay of machine learning. Backpropagation, or reverse-mode differentiation …

In this paper, we discuss an unsupervised deep learning (DL) method for solving time
domain electromagnetic simulations. Compared to the conventional approach, our method …

We develop a new family of variance reduced stochastic gradient descent methods for
minimizing the average of a very large number of smooth functions. Our method—JacSketch …

We propose a novel randomized first order optimization method---SEGA (SkEtched GrAdient
method)---which progressively throughout its iterations builds a variance-reduced estimate …

Large matrices arise in many machine learning and data analysis applications, including as
representations of datasets, graphs, model weights, and first and second-order derivatives …

We develop a family of reformulations of an arbitrary consistent linear system into a
stochastic problem. The reformulations are governed by two user-defined parameters: a …

In this paper, we propose a new randomized second-order optimization algorithm—
Stochastic Subspace Cubic Newton (SSCN)—for minimizing a high dimensional convex …

We consider stochastic second-order methods for minimizing smooth and strongly-convex
functions under an interpolation condition satisfied by over-parameterized models. Under …

In this paper, we study the problem of minimizing a sum of smooth and strongly convex
functions split over the nodes of a network in a decentralized fashion. We propose the …