.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "api/gallery/statistics/hypothesis_tests/plot_01_student_t_test.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_api_gallery_statistics_hypothesis_tests_plot_01_student_t_test.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_api_gallery_statistics_hypothesis_tests_plot_01_student_t_test.py:


Student's t-distribution and the small-sample t-test
=====================================================

With only a handful of measurements, the standardized mean
:math:`t = (\bar x - \mu_0)/(s/\sqrt n)` is not normal: estimating the
spread from the same small sample gives it heavier tails. Gosset
("Student", 1908) derived its exact distribution. This example shows
the heavy tails, how a normal cutoff misleads for small samples, and
the one- and two-sample t-tests built on Student's distribution.

.. GENERATED FROM PYTHON SOURCE LINES 14-20

.. code-block:: Python

    import matplotlib.pyplot as plt
    import numpy as np
    from scipy import stats

    from mathematicskit.statistics import cohens_d, one_sample_t_test, two_sample_t_test








.. GENERATED FROM PYTHON SOURCE LINES 21-23

Heavier tails than the normal
-----------------------------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 23-35

.. code-block:: Python


    x = np.linspace(-5.0, 5.0, 400)
    fig, ax = plt.subplots()
    ax.plot(x, stats.norm.pdf(x), "k--", label="standard normal")
    for df in (1, 3, 10):
        ax.plot(x, stats.t.pdf(x, df), label=f"Student t, {df} df")
    ax.set_xlabel("t")
    ax.set_ylabel("density")
    ax.set_yscale("log")
    ax.set_ylim(1e-4, 0.5)
    ax.legend()




.. image-sg:: /api/gallery/statistics/hypothesis_tests/images/sphx_glr_plot_01_student_t_test_001.png
   :alt: plot 01 student t test
   :srcset: /api/gallery/statistics/hypothesis_tests/images/sphx_glr_plot_01_student_t_test_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <matplotlib.legend.Legend object at 0x11af14830>



.. GENERATED FROM PYTHON SOURCE LINES 36-42

Why the normal cutoff fails for small samples
-----------------------------------------------------

Draw many samples of size 5 from a population whose mean really is
:math:`\mu_0`. Rejecting when :math:`|t| > 1.96` (the normal 5% cutoff)
rejects far too often; Student's cutoff restores the 5% rate.

.. GENERATED FROM PYTHON SOURCE LINES 42-51

.. code-block:: Python


    rng = np.random.default_rng(0)
    n = 5
    samples = rng.normal(loc=10.0, scale=2.0, size=(20000, n))
    t_stats = (samples.mean(axis=1) - 10.0) / (samples.std(axis=1, ddof=1) / np.sqrt(n))
    t_cutoff = stats.t.ppf(0.975, n - 1)
    print(f"normal cutoff 1.96:      false-rejection rate {np.mean(np.abs(t_stats) > 1.96):.3f}")
    print(f"Student cutoff {t_cutoff:.3f}: false-rejection rate {np.mean(np.abs(t_stats) > t_cutoff):.3f}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    normal cutoff 1.96:      false-rejection rate 0.121
    Student cutoff 2.776: false-rejection rate 0.050




.. GENERATED FROM PYTHON SOURCE LINES 52-54

Gosset's setting: a small batch of brewing measurements
--------------------------------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 54-63

.. code-block:: Python


    batch = np.array([10.8, 11.6, 10.2, 12.1, 11.4, 10.9])
    one = one_sample_t_test(batch, mu0=10.0)
    print(f"one-sample t = {one.statistic:.3f} on {one.df:.0f} df, p = {one.p_value:.4f}")

    old_recipe = rng.normal(loc=10.0, scale=0.8, size=8)
    new_recipe = rng.normal(loc=11.0, scale=0.8, size=8)
    two = two_sample_t_test(old_recipe, new_recipe)
    print(f"two-sample t = {two.statistic:.3f}, p = {two.p_value:.4f}, Cohen's d = {cohens_d(old_recipe, new_recipe):.3f}")




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    one-sample t = 4.257 on 5 df, p = 0.0080
    two-sample t = -2.257, p = 0.0405, Cohen's d = -1.128





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.039 seconds)


.. _sphx_glr_download_api_gallery_statistics_hypothesis_tests_plot_01_student_t_test.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_01_student_t_test.ipynb <plot_01_student_t_test.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_01_student_t_test.py <plot_01_student_t_test.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_01_student_t_test.zip <plot_01_student_t_test.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_