Back to Tools
Optimizer Simulator
Visualize how different optimization algorithms navigate loss landscapes.
Landscape
Curvature Control
X Curvature (a)1.0
Y Curvature (b)10.0
High ratio creates a narrow valley (bad condition number).
Hyperparameters
Learning Rate0.01
Momentum 0.9
Adam 0.9
Adam 0.999
Loading Optimizer Visualization...
Iterations: 0
Insight:
Notice how Adam and RMSProp adapt their step sizes to the curvature (moving fast in flat directions, slow in steep ones), often converging faster on the Quadratic Bowl with high curvature. However, Momentum builds up velocity and can oscillate or overshoot, but often escapes shallow local minima better than pure SGD.
Notice how Adam and RMSProp adapt their step sizes to the curvature (moving fast in flat directions, slow in steep ones), often converging faster on the Quadratic Bowl with high curvature. However, Momentum builds up velocity and can oscillate or overshoot, but often escapes shallow local minima better than pure SGD.