.. _2019-01-25linregrst:

========================================
Un exemple simple de régression linéaire
========================================


.. only:: html

    **Links:** :download:`notebook <2019-01-25_linreg.ipynb>`, :downloadlink:`html <2019-01-25_linreg2html.html>`, :download:`PDF <2019-01-25_linreg.pdf>`, :download:`python <2019-01-25_linreg.py>`, :downloadlink:`slides <2019-01-25_linreg.slides.html>`, :githublink:`GitHub|_doc/notebooks/encours/2019-01-25_linreg.ipynb|*`


Un jeu de données, une régression linéaire et quelques trucs bizarres.
Le jeu de données est `Wine Data
Set <https://archive.ics.uci.edu/ml/datasets/wine>`__.

.. code:: ipython3

    %matplotlib inline

.. code:: ipython3

    from jyquickhelper import add_notebook_menu
    add_notebook_menu()






.. contents::
    :local:





Récupération des données
------------------------

Le premier code récupère les données depuis Internet.

.. code:: ipython3

    import pandas

.. code:: ipython3

    df = pandas.read_csv("https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data", 
                         header=None)
    df.head(n=1)






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>0</th>
          <th>1</th>
          <th>2</th>
          <th>3</th>
          <th>4</th>
          <th>5</th>
          <th>6</th>
          <th>7</th>
          <th>8</th>
          <th>9</th>
          <th>10</th>
          <th>11</th>
          <th>12</th>
          <th>13</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>1</td>
          <td>14.23</td>
          <td>1.71</td>
          <td>2.43</td>
          <td>15.6</td>
          <td>127</td>
          <td>2.8</td>
          <td>3.06</td>
          <td>0.28</td>
          <td>2.29</td>
          <td>5.64</td>
          <td>1.04</td>
          <td>3.92</td>
          <td>1065</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    df.shape




.. parsed-literal::
    (178, 14)



On récupère le nom des colonnes.

.. code:: ipython3

    s = "index Alcohol Malicacid Ash Alcalinityofash Magnesium Totalphenols Flavanoids"
    s += " Nonflavanoidphenols Proanthocyanins Colorintensity Hue"
    s += " OD280/OD315 Proline"
    df.columns = s.split()
    df.head().T






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>0</th>
          <th>1</th>
          <th>2</th>
          <th>3</th>
          <th>4</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>index</th>
          <td>1.00</td>
          <td>1.00</td>
          <td>1.00</td>
          <td>1.00</td>
          <td>1.00</td>
        </tr>
        <tr>
          <th>Alcohol</th>
          <td>14.23</td>
          <td>13.20</td>
          <td>13.16</td>
          <td>14.37</td>
          <td>13.24</td>
        </tr>
        <tr>
          <th>Malicacid</th>
          <td>1.71</td>
          <td>1.78</td>
          <td>2.36</td>
          <td>1.95</td>
          <td>2.59</td>
        </tr>
        <tr>
          <th>Ash</th>
          <td>2.43</td>
          <td>2.14</td>
          <td>2.67</td>
          <td>2.50</td>
          <td>2.87</td>
        </tr>
        <tr>
          <th>Alcalinityofash</th>
          <td>15.60</td>
          <td>11.20</td>
          <td>18.60</td>
          <td>16.80</td>
          <td>21.00</td>
        </tr>
        <tr>
          <th>Magnesium</th>
          <td>127.00</td>
          <td>100.00</td>
          <td>101.00</td>
          <td>113.00</td>
          <td>118.00</td>
        </tr>
        <tr>
          <th>Totalphenols</th>
          <td>2.80</td>
          <td>2.65</td>
          <td>2.80</td>
          <td>3.85</td>
          <td>2.80</td>
        </tr>
        <tr>
          <th>Flavanoids</th>
          <td>3.06</td>
          <td>2.76</td>
          <td>3.24</td>
          <td>3.49</td>
          <td>2.69</td>
        </tr>
        <tr>
          <th>Nonflavanoidphenols</th>
          <td>0.28</td>
          <td>0.26</td>
          <td>0.30</td>
          <td>0.24</td>
          <td>0.39</td>
        </tr>
        <tr>
          <th>Proanthocyanins</th>
          <td>2.29</td>
          <td>1.28</td>
          <td>2.81</td>
          <td>2.18</td>
          <td>1.82</td>
        </tr>
        <tr>
          <th>Colorintensity</th>
          <td>5.64</td>
          <td>4.38</td>
          <td>5.68</td>
          <td>7.80</td>
          <td>4.32</td>
        </tr>
        <tr>
          <th>Hue</th>
          <td>1.04</td>
          <td>1.05</td>
          <td>1.03</td>
          <td>0.86</td>
          <td>1.04</td>
        </tr>
        <tr>
          <th>OD280/OD315</th>
          <td>3.92</td>
          <td>3.40</td>
          <td>3.17</td>
          <td>3.45</td>
          <td>2.93</td>
        </tr>
        <tr>
          <th>Proline</th>
          <td>1065.00</td>
          <td>1050.00</td>
          <td>1185.00</td>
          <td>1480.00</td>
          <td>735.00</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    df.describe().T






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>count</th>
          <th>mean</th>
          <th>std</th>
          <th>min</th>
          <th>25%</th>
          <th>50%</th>
          <th>75%</th>
          <th>max</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>index</th>
          <td>178.0</td>
          <td>1.938202</td>
          <td>0.775035</td>
          <td>1.00</td>
          <td>1.0000</td>
          <td>2.000</td>
          <td>3.0000</td>
          <td>3.00</td>
        </tr>
        <tr>
          <th>Alcohol</th>
          <td>178.0</td>
          <td>13.000618</td>
          <td>0.811827</td>
          <td>11.03</td>
          <td>12.3625</td>
          <td>13.050</td>
          <td>13.6775</td>
          <td>14.83</td>
        </tr>
        <tr>
          <th>Malicacid</th>
          <td>178.0</td>
          <td>2.336348</td>
          <td>1.117146</td>
          <td>0.74</td>
          <td>1.6025</td>
          <td>1.865</td>
          <td>3.0825</td>
          <td>5.80</td>
        </tr>
        <tr>
          <th>Ash</th>
          <td>178.0</td>
          <td>2.366517</td>
          <td>0.274344</td>
          <td>1.36</td>
          <td>2.2100</td>
          <td>2.360</td>
          <td>2.5575</td>
          <td>3.23</td>
        </tr>
        <tr>
          <th>Alcalinityofash</th>
          <td>178.0</td>
          <td>19.494944</td>
          <td>3.339564</td>
          <td>10.60</td>
          <td>17.2000</td>
          <td>19.500</td>
          <td>21.5000</td>
          <td>30.00</td>
        </tr>
        <tr>
          <th>Magnesium</th>
          <td>178.0</td>
          <td>99.741573</td>
          <td>14.282484</td>
          <td>70.00</td>
          <td>88.0000</td>
          <td>98.000</td>
          <td>107.0000</td>
          <td>162.00</td>
        </tr>
        <tr>
          <th>Totalphenols</th>
          <td>178.0</td>
          <td>2.295112</td>
          <td>0.625851</td>
          <td>0.98</td>
          <td>1.7425</td>
          <td>2.355</td>
          <td>2.8000</td>
          <td>3.88</td>
        </tr>
        <tr>
          <th>Flavanoids</th>
          <td>178.0</td>
          <td>2.029270</td>
          <td>0.998859</td>
          <td>0.34</td>
          <td>1.2050</td>
          <td>2.135</td>
          <td>2.8750</td>
          <td>5.08</td>
        </tr>
        <tr>
          <th>Nonflavanoidphenols</th>
          <td>178.0</td>
          <td>0.361854</td>
          <td>0.124453</td>
          <td>0.13</td>
          <td>0.2700</td>
          <td>0.340</td>
          <td>0.4375</td>
          <td>0.66</td>
        </tr>
        <tr>
          <th>Proanthocyanins</th>
          <td>178.0</td>
          <td>1.590899</td>
          <td>0.572359</td>
          <td>0.41</td>
          <td>1.2500</td>
          <td>1.555</td>
          <td>1.9500</td>
          <td>3.58</td>
        </tr>
        <tr>
          <th>Colorintensity</th>
          <td>178.0</td>
          <td>5.058090</td>
          <td>2.318286</td>
          <td>1.28</td>
          <td>3.2200</td>
          <td>4.690</td>
          <td>6.2000</td>
          <td>13.00</td>
        </tr>
        <tr>
          <th>Hue</th>
          <td>178.0</td>
          <td>0.957449</td>
          <td>0.228572</td>
          <td>0.48</td>
          <td>0.7825</td>
          <td>0.965</td>
          <td>1.1200</td>
          <td>1.71</td>
        </tr>
        <tr>
          <th>OD280/OD315</th>
          <td>178.0</td>
          <td>2.611685</td>
          <td>0.709990</td>
          <td>1.27</td>
          <td>1.9375</td>
          <td>2.780</td>
          <td>3.1700</td>
          <td>4.00</td>
        </tr>
        <tr>
          <th>Proline</th>
          <td>178.0</td>
          <td>746.893258</td>
          <td>314.907474</td>
          <td>278.00</td>
          <td>500.5000</td>
          <td>673.500</td>
          <td>985.0000</td>
          <td>1680.00</td>
        </tr>
      </tbody>
    </table>
    </div>



Corrélations
------------

On utilise le module `seaborn <https://seaborn.pydata.org/index.html>`__
pour regarder visuellement les corrélations avec le graphe `Scatterplot
Matrix <https://seaborn.pydata.org/examples/scatterplot_matrix.html>`__.

.. code:: ipython3

    import seaborn as sns
    sns.set(style="ticks")
    
    sns.pairplot(df);



.. image:: 2019-01-25_linreg_11_0.png


Il y a quelques corrélations.

Première régression
-------------------

On essaye de prédire le taux d’alcool du vin à partir des autres
variables.

.. code:: ipython3

    X = df.drop(["Alcohol", "index"], axis=1)
    y = df["Alcohol"]

On divise entre train et test pour éviter l’overfitting.

.. code:: ipython3

    from sklearn.model_selection import train_test_split

.. code:: ipython3

    X_train, X_test, y_train, y_test = train_test_split(X, y)

.. code:: ipython3

    from sklearn.linear_model import LinearRegression

.. code:: ipython3

    model = LinearRegression()

.. code:: ipython3

    model.fit(X_train, y_train)




.. parsed-literal::
    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,
             normalize=False)



.. code:: ipython3

    model.coef_




.. parsed-literal::
    array([ 1.63363895e-01,  3.94203469e-01, -6.00949512e-02, -1.19824047e-04,
            1.58697845e-01, -9.56728506e-02, -3.06298297e-01,  5.07946728e-02,
            1.38811432e-01,  1.59459210e-02,  1.64240691e-01,  8.15269173e-04])



.. code:: ipython3

    from sklearn.metrics import r2_score

.. code:: ipython3

    y_pred = model.predict(X_test)
    r2_score(y_test, y_pred)




.. parsed-literal::
    0.4649938412573541



.. code:: ipython3

    y_predt = model.predict(X_train)
    r2_score(y_train, y_predt)




.. parsed-literal::
    0.6136769355983627



Même chose avec un arbre de décision.

.. code:: ipython3

    from sklearn.tree import DecisionTreeRegressor

.. code:: ipython3

    dt = DecisionTreeRegressor(min_samples_leaf=10)

.. code:: ipython3

    dt.fit(X_train, y_train)




.. parsed-literal::
    DecisionTreeRegressor(criterion='mse', max_depth=None, max_features=None,
               max_leaf_nodes=None, min_impurity_decrease=0.0,
               min_impurity_split=None, min_samples_leaf=10,
               min_samples_split=2, min_weight_fraction_leaf=0.0,
               presort=False, random_state=None, splitter='best')



.. code:: ipython3

    y_pred = dt.predict(X_test)
    r2_score(y_test, y_pred)




.. parsed-literal::
    0.6694565403724826



.. code:: ipython3

    y_predt = dt.predict(X_train)
    r2_score(y_train, y_predt)




.. parsed-literal::
    0.7172799827093439



.. code:: ipython3

    X_train.shape




.. parsed-literal::
    (133, 12)



On vérifie avec une validation croisée.

.. code:: ipython3

    from sklearn.model_selection import cross_val_score

.. code:: ipython3

    cross_val_score(model, X, y, cv=5)




.. parsed-literal::
    array([-0.51847929,  0.40456596, -0.73716381, -0.46995099, -0.47213708])



Ca oscille trop pour que cela soit vraisemblable. C’est probablement dû
à l’ordre dans les données.

.. code:: ipython3

    try:
        from sklearn.utils import shuffle
        df2 = shuffle(df).reset_index()
    except Exception:
        import random
        index = list(df.index)
        df2 = df.iloc[index].reset_index()

.. code:: ipython3

    X2 = df2.drop("Alcohol", axis=1)
    y2 = df2.Alcohol
    cross_val_score(model, X2, y2, cv=5)




.. parsed-literal::
    array([0.40197927, 0.55886103, 0.57939265, 0.49028122, 0.62212365])



Beaucoup plus stable.

Feature importance
------------------

.. code:: ipython3

    dt.feature_importances_




.. parsed-literal::
    array([0.03523663, 0.        , 0.00776046, 0.        , 0.        ,
           0.        , 0.02238126, 0.        , 0.6849213 , 0.        ,
           0.0124994 , 0.23720095])



.. code:: ipython3

    ft = pandas.DataFrame(dict(name=X_train.columns, fi=dt.feature_importances_))
    ft = ft.sort_values("fi", ascending=False)
    ft






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>name</th>
          <th>fi</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>8</th>
          <td>Colorintensity</td>
          <td>0.684921</td>
        </tr>
        <tr>
          <th>11</th>
          <td>Proline</td>
          <td>0.237201</td>
        </tr>
        <tr>
          <th>0</th>
          <td>Malicacid</td>
          <td>0.035237</td>
        </tr>
        <tr>
          <th>6</th>
          <td>Nonflavanoidphenols</td>
          <td>0.022381</td>
        </tr>
        <tr>
          <th>10</th>
          <td>OD280/OD315</td>
          <td>0.012499</td>
        </tr>
        <tr>
          <th>2</th>
          <td>Alcalinityofash</td>
          <td>0.007760</td>
        </tr>
        <tr>
          <th>1</th>
          <td>Ash</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>3</th>
          <td>Magnesium</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>4</th>
          <td>Totalphenols</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>5</th>
          <td>Flavanoids</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>7</th>
          <td>Proanthocyanins</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>9</th>
          <td>Hue</td>
          <td>0.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    ft.set_index("name").plot(kind="bar");



.. image:: 2019-01-25_linreg_42_0.png


L’intensité de la couleur a l’air de jouer un grand rôle mais dans quel
sens ?

.. code:: ipython3

    ax = df[["Colorintensity", "Alcohol"]].plot(x="Colorintensity", y="Alcohol", kind="scatter")
    ax.set_title("Lien entre le degré d'alcool\net l'intensité de la couleur");


.. parsed-literal::
    'c' argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with 'x' & 'y'.  Please use a 2-D array with a single row if you really want to specify the same RGB or RGBA value for all points.



.. image:: 2019-01-25_linreg_44_1.png


La relation n’est pas linéaire car le degré d’alcool montre rarement
au-dessus de 14.5 mais elle paraît linéaire par morceaux.