Randomized numerical linear algebra: A perspective on the field with an eye to software

R Murray, J Demmel, MW Mahoney… - arxiv preprint arxiv …, 2023 - arxiv.org
Randomized numerical linear algebra-RandNLA, for short-concerns the use of
randomization as a resource to develop improved algorithms for large-scale linear algebra …

Robust, randomized preconditioning for kernel ridge regression

M Díaz, EN Epperly, Z Frangella, JA Tropp… - arxiv preprint arxiv …, 2023 - arxiv.org
This paper investigates two randomized preconditioning techniques for solving kernel ridge
regression (KRR) problems with a medium to large number of data points ($10^ 4\leq N\leq …

Effective dimension adaptive sketching methods for faster regularized least-squares optimization

J Lacotte, M Pilanci - Advances in neural information …, 2020 - proceedings.neurips.cc
We propose a new randomized algorithm for solving L2-regularized least-squares problems
based on sketching. We consider two of the most popular random embeddings, namely …

Optimal randomized first-order methods for least-squares problems

J Lacotte, M Pilanci - International Conference on Machine …, 2020 - proceedings.mlr.press
We provide an exact analysis of a class of randomized algorithms for solving
overdetermined least-squares problems. We consider first-order methods, where the …

Training quantized neural networks to global optimality via semidefinite programming

B Bartan, M Pilanci - International Conference on Machine …, 2021 - proceedings.mlr.press
Neural networks (NNs) have been extremely successful across many tasks in machine
learning. Quantization of NN weights has become an important topic due to its impact on …

Fast and forward stable randomized algorithms for linear least-squares problems

EN Epperly - SIAM Journal on Matrix Analysis and Applications, 2024 - SIAM
Iterative sketching and sketch-and-precondition are randomized algorithms used for solving
overdetermined linear least-squares problems. When implemented in exact arithmetic, these …

Faster least squares optimization

J Lacotte, M Pilanci - arxiv preprint arxiv:1911.02675, 2019 - arxiv.org
We investigate iterative methods with randomized preconditioners for solving
overdetermined least-squares problems, where the preconditioners are based on a random …

Optimal shrinkage for distributed second-order optimization

F Zhang, M Pilanci - International Conference on Machine …, 2023 - proceedings.mlr.press
In this work, we address the problem of Hessian inversion bias in distributed second-order
optimization algorithms. We introduce a novel shrinkage-based estimator for the resolvent of …

Fast randomized least-squares solvers can be just as accurate and stable as classical direct solvers

EN Epperly, M Meier, Y Nakatsukasa - arxiv preprint arxiv:2406.03468, 2024 - arxiv.org
One of the greatest success stories of randomized algorithms for linear algebra has been the
development of fast, randomized algorithms for highly overdetermined linear least-squares …

Surrogate-based autotuning for randomized sketching algorithms in regression problems

Y Cho, JW Demmel, M Dereziński, H Li, H Luo… - arxiv preprint arxiv …, 2023 - arxiv.org
Algorithms from Randomized Numerical Linear Algebra (RandNLA) are known to be
effective in handling high-dimensional computational problems, providing high-quality …