.. _winesknnsplitstratrst:

========================================
Découpage stratifié apprentissage / test
========================================


.. only:: html

    **Links:** :download:`notebook <wines_knn_split_strat.ipynb>`, :downloadlink:`html <wines_knn_split_strat2html.html>`, :download:`PDF <wines_knn_split_strat.pdf>`, :download:`python <wines_knn_split_strat.py>`, :downloadlink:`slides <wines_knn_split_strat.slides.html>`, :githublink:`GitHub|_doc/notebooks/lectures/wines_knn_split_strat.ipynb|*`


Lorsqu’une classe est sous-représentée, il y a peu de chances que la
répartition apprentissage test conserve la distribution des classes.

.. code:: ipython3

    %matplotlib inline

.. code:: ipython3

    from papierstat.datasets import load_wines_dataset
    df = load_wines_dataset()
    X = df.drop(['quality', 'color'], axis=1)
    y = df['quality']

On divise en base d’apprentissage et de test avec la fonction
`train_test_split <http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html>`__.

.. code:: ipython3

    from sklearn.model_selection import train_test_split
    X_train, X_test, y_train, y_test = train_test_split(X, y)

.. code:: ipython3

    import pandas
    ys = pandas.DataFrame(dict(y=y_train))
    ys['base'] = 'train'
    ys2 = pandas.DataFrame(dict(y=y_test))
    ys2['base'] = 'test'
    ys = pandas.concat([ys, ys2])
    ys['compte'] = 1
    piv = ys.groupby(['base', 'y'], as_index=False).count().pivot('y', 'base', 'compte')
    piv['ratio'] = piv['test'] / piv['train']
    piv






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th>base</th>
          <th>test</th>
          <th>train</th>
          <th>ratio</th>
        </tr>
        <tr>
          <th>y</th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>3</th>
          <td>13.0</td>
          <td>17.0</td>
          <td>0.764706</td>
        </tr>
        <tr>
          <th>4</th>
          <td>54.0</td>
          <td>162.0</td>
          <td>0.333333</td>
        </tr>
        <tr>
          <th>5</th>
          <td>539.0</td>
          <td>1599.0</td>
          <td>0.337086</td>
        </tr>
        <tr>
          <th>6</th>
          <td>713.0</td>
          <td>2123.0</td>
          <td>0.335846</td>
        </tr>
        <tr>
          <th>7</th>
          <td>267.0</td>
          <td>812.0</td>
          <td>0.328818</td>
        </tr>
        <tr>
          <th>8</th>
          <td>39.0</td>
          <td>154.0</td>
          <td>0.253247</td>
        </tr>
        <tr>
          <th>9</th>
          <td>NaN</td>
          <td>5.0</td>
          <td>NaN</td>
        </tr>
      </tbody>
    </table>
    </div>



On voit le ratio entre les deux classes est à peu près égal à 1/3 sauf
pour les notes sous-représentées. On utilise une répartition stratifiée
: la distribution d’une variable, les labels, sera la même dans les
bases d’apprentissages et de de tests. On s’inspire de l’exemple
`StratifiedShuffleSplit <http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedShuffleSplit.html>`__.

.. code:: ipython3

    from sklearn.model_selection import StratifiedShuffleSplit
    split = StratifiedShuffleSplit(n_splits=1, test_size=0.33)
    train_index, test_index = list(split.split(X, y))[0]
    len(train_index), len(test_index)




.. parsed-literal::
    (4352, 2145)



.. code:: ipython3

    X_train, y_train = X.iloc[train_index, :], y[train_index]
    X_test, y_test = X.iloc[test_index, :], y[test_index]
    y_train.shape, y_test.shape




.. parsed-literal::
    ((4352,), (2145,))



.. code:: ipython3

    ys = pandas.DataFrame(dict(y=y_train))
    ys['base'] = 'train'
    ys2 = pandas.DataFrame(dict(y=y_test))
    ys2['base'] = 'test'
    ys = pandas.concat([ys, ys2])
    ys['compte'] = 1
    piv = ys.groupby(['base', 'y'], as_index=False).count().pivot('y', 'base', 'compte')
    piv['ratio'] = piv['test'] / piv['train']
    piv






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th>base</th>
          <th>test</th>
          <th>train</th>
          <th>ratio</th>
        </tr>
        <tr>
          <th>y</th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>3</th>
          <td>10</td>
          <td>20</td>
          <td>0.500000</td>
        </tr>
        <tr>
          <th>4</th>
          <td>71</td>
          <td>145</td>
          <td>0.489655</td>
        </tr>
        <tr>
          <th>5</th>
          <td>706</td>
          <td>1432</td>
          <td>0.493017</td>
        </tr>
        <tr>
          <th>6</th>
          <td>936</td>
          <td>1900</td>
          <td>0.492632</td>
        </tr>
        <tr>
          <th>7</th>
          <td>356</td>
          <td>723</td>
          <td>0.492393</td>
        </tr>
        <tr>
          <th>8</th>
          <td>64</td>
          <td>129</td>
          <td>0.496124</td>
        </tr>
        <tr>
          <th>9</th>
          <td>2</td>
          <td>3</td>
          <td>0.666667</td>
        </tr>
      </tbody>
    </table>
    </div>



Le ratio entre les classes est identique, la classe test contient deux
fois moins d’invidivu et c’est vrai pour toutes les classes excepté pour
la classe 9 qui contient si peu d’éléments que c’est impossible.

.. code:: ipython3

    from sklearn.neighbors import KNeighborsRegressor
    knn = KNeighborsRegressor(n_neighbors=1)
    knn.fit(X_train, y_train)




.. parsed-literal::
    KNeighborsRegressor(algorithm='auto', leaf_size=30, metric='minkowski',
              metric_params=None, n_jobs=1, n_neighbors=1, p=2,
              weights='uniform')



.. code:: ipython3

    prediction = knn.predict(X_test)

.. code:: ipython3

    from sklearn.metrics import r2_score
    r2_score(y_test, prediction)




.. parsed-literal::
    -0.1007330402006481



Cela n’améliore pas la qualité du modèle mais on est sûr que les classes
sous-représentées sont mieux gérées par cette répartition aléatoire
stratifiée.