.. _sphx_glr_api_gallery_probability_markov_chain:

Markov chains
---------------

Stationary distributions and gambler's-ruin absorption probabilities.


.. raw:: html

  <div id='sg-tag-list' class='sphx-glr-tag-list'></div>


.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="A Markov chain moves between states with probabilities that depend only on the current state, not on how it got there, so the whole process is described by one transition matrix. In gambler&#x27;s ruin, a gambler with capital i (out of a target N) makes fair even-money bets until reaching 0 (ruin) or N (target); the capital after each bet is a Markov chain on the states 0..N with two absorbing ends. Both the absorption probabilities and the expected number of steps have simple closed forms for a fair game, used here to check the numerical solve.">

.. only:: html

  .. image:: /api/gallery/probability/markov_chain/images/thumb/sphx_glr_plot_01_gamblers_ruin_thumb.png
    :alt:

  :doc:`/api/gallery/probability/markov_chain/plot_01_gamblers_ruin`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Markov chains: gambler's ruin as an absorbing chain</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="A machine is working (0), degraded (1), or broken (2), jumping between states at the rates in the generator Q. The transition matrix P(t) = e^{tQ} solves Kolmogorov&#x27;s forward and backward equations and relaxes to the stationary distribution \pi, the solution of \pi Q = 0.">

.. only:: html

  .. image:: /api/gallery/probability/markov_chain/images/thumb/sphx_glr_plot_02_continuous_time_chain_thumb.png
    :alt:

  :doc:`/api/gallery/probability/markov_chain/plot_02_continuous_time_chain`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Kolmogorov's equations: a continuous-time Markov chain</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /api/gallery/probability/markov_chain/plot_01_gamblers_ruin
   /api/gallery/probability/markov_chain/plot_02_continuous_time_chain