In 1963, Polyak proposed a simple condition that is sufficient to show a global linear
convergence rate for gradient descent. This condition is a special case of the Łojasiewicz …

Consider the consensus problem of minimizing f(x)=i=1^nf_i(x), where x∈R^p and each f_i
is only known to the individual agent i in a connected network of n agents. To solve this …

We improve a recent gurantee of Bach and Moulines on the linear convergence of SGD for
smooth and strongly convex objectives, reducing a quadratic dependence on the strong …

Nonconvex minimax problems appear frequently in emerging machine learning
applications, such as generative adversarial networks and adversarial learning. Simple …

It is believed that Gradient Descent (GD) induces an implicit bias towards good
generalization in training machine learning models. This paper provides a fine-grained …

We prove the convergence to minima and estimates on the rate of convergence for the
stochastic gradient descent method in the case of not necessarily locally convex nor …

We study the implicit regularization of mini-batch stochastic gradient descent, when applied
to the fundamental problem of least squares regression. We leverage a continuous-time …

Abstract Polyak-Łojasiewicz (PL)(Polyak, 1963) condition is a weaker condition than the
strong convexity but suffices to ensure a global convergence for the Gradient Descent …

Abstract Stochastic Gradient Descent (SGD) is being used routinely for optimizing non-
convex functions. Yet, the standard convergence theory for SGD in the smooth non-convex …

Large over-parametrized models learned via stochastic gradient descent (SGD) methods
have become a key element in modern machine learning. Although SGD methods are very …