.. _sphx_glr_api_gallery_optimization_stochastic:

Stochastic approximation
------------------------

The Robbins-Monro iteration: stochastic gradient descent on noisy gradients.


.. raw:: html

  <div id='sg-tag-list' class='sphx-glr-tag-list'></div>


.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Minimizes a quadratic using only noisy gradient observations. With Robbins and Monro&#x27;s step sizes a_k = a_0/(k+1), the noise averages out and the error shrinks like 1/\sqrt{k}. A constant step size instead stalls at a noise floor.">

.. only:: html

  .. image:: /api/gallery/optimization/stochastic/images/thumb/sphx_glr_plot_01_robbins_monro_thumb.png
    :alt:

  :doc:`/api/gallery/optimization/stochastic/plot_01_robbins_monro`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Robbins-Monro: stochastic gradient descent with decaying steps</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /api/gallery/optimization/stochastic/plot_01_robbins_monro