Solving inverse problems without the use of derivatives or adjoints of the forward model is
highly desirable in many applications arising in science and engineering. In this paper we …

We propose energy natural gradient descent, a natural gradient method with respect to a
Hessian-induced Riemannian metric as an optimization algorithm for physics-informed …

Wasserstein distributionally robust optimization (DRO) aims to find robust and generalizable
solutions by hedging against data perturbations in Wasserstein distance. Despite its recent …

Bayesian inference problems require sampling or approximating high-dimensional
probability distributions. The focus of this paper is on the recently introduced Stein …

We consider the development of practical stochastic quasi-Newton, and in particular
Kronecker-factored block diagonal BFGS and L-BFGS methods, for training deep neural …

Minimizing a convex function of a measure with a sparsity-inducing penalty is a typical
problem arising, eg, in sparse spikes deconvolution or two-layer neural networks training …

Abstract Information geometry and optimal transport are two distinct geometric frameworks
for modeling families of probability measures. During the recent years, there has been a …

Sequence-to-sequence models are commonly trained via maximum likelihood estimation
(MLE). However, standard MLE training considers a word-level objective, predicting the next …

We design and compute first-order implicit-in-time variational schemes with high-order
spatial discretization for initial value gradient flows in generalized optimal transport metric …

We propose efficient numerical schemes for implementing the natural gradient descent
(NGD) for a broad range of metric spaces with applications to PDE-based optimization …