.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "api/gallery/calculus/autodiff/plot_01_reverse_mode_gradients.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_api_gallery_calculus_autodiff_plot_01_reverse_mode_gradients.py: Reverse-mode automatic differentiation for multivariable gradients ========================================================================= A minimal backpropagation-style engine: one forward pass builds a computation graph, then one backward pass computes the gradient with respect to every input in a single traversal -- the technique underlying modern deep-learning frameworks, applied here to a small analytic function. .. GENERATED FROM PYTHON SOURCE LINES 13-17 .. code-block:: Python import math from mathematicskit.calculus.systems.autodiff import Variable, gradient .. GENERATED FROM PYTHON SOURCE LINES 18-20 Gradient of a simple multivariable function --------------------------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 20-31 .. code-block:: Python f = lambda x, y: x * x * y + y.sin() * x x0, y0 = 2.0, 0.5 grad = gradient(f, [x0, y0]) print(f"gradient at ({x0}, {y0}): {grad}") # Closed-form check: df/dx = 2xy + sin(y), df/dy = x^2 + x*cos(y) expected_dx = 2.0 * x0 * y0 + math.sin(y0) expected_dy = x0**2 + x0 * math.cos(y0) print(f"closed-form gradient: [{expected_dx}, {expected_dy}]") .. rst-class:: sphx-glr-script-out .. code-block:: none gradient at (2.0, 0.5): [2.479425538604203, 5.7551651237807455] closed-form gradient: [2.479425538604203, 5.7551651237807455] .. GENERATED FROM PYTHON SOURCE LINES 32-34 Shared subexpressions accumulate gradient contributions correctly ------------------------------------------------------------------------ .. GENERATED FROM PYTHON SOURCE LINES 34-39 .. code-block:: Python x = Variable(3.0) y = x * x + x # used twice: y = x^2 + x, dy/dx = 2x + 1 y.backward() print(f"d(x^2 + x)/dx at x=3: {x.grad} (expected {2 * 3.0 + 1})") .. rst-class:: sphx-glr-script-out .. code-block:: none d(x^2 + x)/dx at x=3: 7.0 (expected 7.0) .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.001 seconds) .. _sphx_glr_download_api_gallery_calculus_autodiff_plot_01_reverse_mode_gradients.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_01_reverse_mode_gradients.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_01_reverse_mode_gradients.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_01_reverse_mode_gradients.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_