This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"
-
Updated
Jul 17, 2021
#
deep-learning-optimizers
Here are 5 public repositories matching this topic...
Advanced Mathematical Optimization & Deep Learning Optimizers from scratch. Covers KKT duality, L-BFGS, proximal methods (ADMM, FISTA), stochastic algorithms (SVRG, Lion), and cutting-edge deep learning optimizers like K-FAC, Shampoo, Sophia, SAM, and Muon. Bridging strict convex calculus with large-scale Transformer training.
python admm convex-optimization distributed-optimization l-bfgs stochastic-optimization mathematical-optimization deep-learning-optimizers proximal-gradient transformer-training
-
Updated
May 27, 2026 - Jupyter Notebook
Semester project on the impact of label noise on deep learning optimization
-
Updated
Jan 17, 2023 - Python
Guava Disease Classification using DenseNet-Reloaded and ADOPT optimizer. Achieves 99.73% accuracy in Anthracnose, Fruit Fly, and Healthy Guava detection.
-
Updated
Jan 3, 2025 - Jupyter Notebook
Toy implementations of popular deep learning optimizers from scratch in JAX
-
Updated
Jun 20, 2021 - Jupyter Notebook
Improve this page
Add a description, image, and links to the deep-learning-optimizers topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the deep-learning-optimizers topic, visit your repo's landing page and select "manage topics."