.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "api/gallery/optimization/momentum/plot_02_adam.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_api_gallery_optimization_momentum_plot_02_adam.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_api_gallery_optimization_momentum_plot_02_adam.py:


Adam on a badly scaled problem
====================================

A quadratic whose two coordinates differ in curvature by a factor of
1000. Fixed-step gradient descent must use a step small enough for the
stiff coordinate, so it crawls along the flat one. Adam rescales each
coordinate by its root-mean-square gradient and moves both at a similar
speed.

.. GENERATED FROM PYTHON SOURCE LINES 13-18

.. code-block:: Python

    import numpy as np

    from mathematicskit.optimization import Adam, GradientDescent
    from mathematicskit.optimization.visualizers.plots import plot_contour_path








.. GENERATED FROM PYTHON SOURCE LINES 19-21

f(x, y) = x^2 / 1000 + y^2
-----------------------------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 21-32

.. code-block:: Python


    f = lambda x: x[0] ** 2 / 1000.0 + x[1] ** 2
    grad = lambda x: np.array([2.0 * x[0] / 1000.0, 2.0 * x[1]])
    x0 = np.array([-1.5, 2.5])

    gd = GradientDescent(alpha=0.4, max_iter=300).minimize(f, grad, x0)
    adam = Adam(alpha=0.05, max_iter=300).minimize(f, grad, x0)
    print(f"after 300 steps: gradient descent x = {gd.x.round(4)}, Adam x = {adam.x.round(4) + 0.0}")

    ax = plot_contour_path(f, gd, x_range=(-2.0, 0.5), y_range=(-1.0, 3.0), label="gradient descent")
    plot_contour_path(f, adam, ax=ax, x_range=(-2.0, 0.5), y_range=(-1.0, 3.0), label="Adam (Kingma & Ba, 2014)")



.. image-sg:: /api/gallery/optimization/momentum/images/sphx_glr_plot_02_adam_001.png
   :alt: Optimizer iterate path
   :srcset: /api/gallery/optimization/momentum/images/sphx_glr_plot_02_adam_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    after 300 steps: gradient descent x = [-1.1798  0.    ], Adam x = [0. 0.]

    <Axes: title={'center': 'Optimizer iterate path'}, xlabel='x', ylabel='y'>




.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.074 seconds)


.. _sphx_glr_download_api_gallery_optimization_momentum_plot_02_adam.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_02_adam.ipynb <plot_02_adam.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_02_adam.py <plot_02_adam.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_02_adam.zip <plot_02_adam.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_