Support vector machine (SVM) is an optimal margin based classification technique in
machine learning. SVM is a binary linear classifier which has been extended to non-linear …

The rise of big data has led to new demands for machine learning (ML) systems to learn
complex models, with millions to billions of parameters, that promise adequate capacity to …

We identify and formalize a fundamental gradient descent phenomenon resulting in a
learning proclivity in over-parameterized neural networks. Gradient Starvation arises when …

Support vector machines (SVM), as one of effective and popular classification tools, have
been widely applied in various fields. However, they may incur prohibitive computational …

Although disentangled representations are often said to be beneficial for downstream tasks,
current empirical and theoretical understanding is limited. In this work, we provide evidence …

Stochastic gradient descent is popular for large scale optimization but has slow
convergence asymptotically due to the inherent variance. To remedy this problem, we …

In the problem of domain generalization (DG), there are labeled training data sets from
several related prediction problems, and the goal is to make accurate predictions on future …

Data analytics stands to benefit from the increasing availability of datasets that are held
without their conceptual relationships being explicitly known. When collected, these datasets …

Stochastic Gradient Descent (SGD) has become popular for solving large scale supervised
machine learning optimization problems such as SVM, due to their strong theoretical …

This chapter focuses on providing a complete grasp of the foundations of machine learning
(ML). Machine learning is a rapidly evolving domain with wide-ranging applications, from …