.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "api/gallery/optimization/conjugate_gradient/plot_01_cg_vs_gradient_descent.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_api_gallery_optimization_conjugate_gradient_plot_01_cg_vs_gradient_descent.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_api_gallery_optimization_conjugate_gradient_plot_01_cg_vs_gradient_descent.py:


Nonlinear conjugate gradient vs. gradient descent
=======================================================

Compares Polak-Ribiere nonlinear CG against plain gradient descent on
the same ill-conditioned quadratic bowl -- CG's conjugate search
directions avoid the zig-zagging that slows gradient descent down.

.. GENERATED FROM PYTHON SOURCE LINES 11-15

.. code-block:: Python

    from mathematicskit.optimization import GradientDescent, NonlinearConjugateGradient, quadratic_bowl, quadratic_bowl_grad
    from mathematicskit.optimization.utils.comparison import compare_optimizers
    from mathematicskit.optimization.visualizers.plots import plot_convergence_comparison








.. GENERATED FROM PYTHON SOURCE LINES 16-18

Compare both methods from the same starting point
-----------------------------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 18-27

.. code-block:: Python


    optimizers = {
        "gradient_descent": GradientDescent(alpha=0.03, tol=1e-8, max_iter=2000),
        "conjugate_gradient": NonlinearConjugateGradient(tol=1e-8, max_iter=2000),
    }
    results = compare_optimizers(optimizers, quadratic_bowl, quadratic_bowl_grad, [5.0, -3.0])
    for name, result in results.items():
        print(f"{name}: {result.iterations} iterations, x = {result.x}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    gradient_descent: 658 iterations, x = [ 9.88032321e-009 -3.56148881e-102]
    conjugate_gradient: 38 iterations, x = [2.23413435e-09 5.77512021e-10]




.. GENERATED FROM PYTHON SOURCE LINES 28-30

Plot the convergence-rate comparison
-----------------------------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 30-32

.. code-block:: Python


    plot_convergence_comparison(results, quadratic_bowl, f_star=0.0)



.. image-sg:: /api/gallery/optimization/conjugate_gradient/images/sphx_glr_plot_01_cg_vs_gradient_descent_001.png
   :alt: Convergence-rate comparison
   :srcset: /api/gallery/optimization/conjugate_gradient/images/sphx_glr_plot_01_cg_vs_gradient_descent_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <Axes: title={'center': 'Convergence-rate comparison'}, xlabel='iteration', ylabel='f(x_k) - f*'>




.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.044 seconds)


.. _sphx_glr_download_api_gallery_optimization_conjugate_gradient_plot_01_cg_vs_gradient_descent.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_01_cg_vs_gradient_descent.ipynb <plot_01_cg_vs_gradient_descent.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_01_cg_vs_gradient_descent.py <plot_01_cg_vs_gradient_descent.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_01_cg_vs_gradient_descent.zip <plot_01_cg_vs_gradient_descent.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_