Recent advances in vision-language pre-training have pushed the state-of-the-art on
various vision-language tasks, making machines more capable of multi-modal writing …

Recent advances in NLP have significantly improved the performance of language models
on a variety of tasks. While these advances are largely driven by the availability of large …

Adaptive gradient methods have been widely adopted in training large-scale deep neural
networks, especially large foundation models. Despite the huge success in practice, their …

Heavy-ball momentum with decaying learning rates is widely used with SGD for optimizing
deep learning models. In contrast to its empirical popularity, the understanding of its …

In the burgeoning realm of artificial intelligence, foundation models stand out as a pivotal
advancement. Foundation models centralize the information from broad data across various …

Recent advances in NLP have significantly improved the performance of language models
on a variety of tasks. While these advances are largely driven by the availability of large …