.. _adultcatrst:

==================================
Jeu de données avec des catégories
==================================


.. only:: html

    **Links:** :download:`notebook <adult_cat.ipynb>`, :downloadlink:`html <adult_cat2html.html>`, :download:`PDF <adult_cat.pdf>`, :download:`python <adult_cat.py>`, :downloadlink:`slides <adult_cat.slides.html>`, :githublink:`GitHub|_doc/notebooks/lectures/adult_cat.ipynb|*`


Le jeu de données `Adult Data
Set <https://archive.ics.uci.edu/ml/datasets/adult>`__ ne contient
presque que des catégories. Ce notebook explore différentes moyens de
les traiter.

.. code:: ipython3

    %matplotlib inline

.. code:: ipython3

    from jyquickhelper import add_notebook_menu
    add_notebook_menu()






.. contents::
    :local:





.. code:: ipython3

    import fiona

données
-------

.. code:: ipython3

    from papierstat.datasets import load_adult_dataset
    train, test = load_adult_dataset(url="copy")
    train.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>age</th>
          <th>workclass</th>
          <th>fnlwgt</th>
          <th>education</th>
          <th>education_num</th>
          <th>marital_status</th>
          <th>occupation</th>
          <th>relationship</th>
          <th>race</th>
          <th>sex</th>
          <th>capital_gain</th>
          <th>capital_loss</th>
          <th>hours_per_week</th>
          <th>native_country</th>
          <th>&lt;=50K</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>39</td>
          <td>State-gov</td>
          <td>77516</td>
          <td>Bachelors</td>
          <td>13</td>
          <td>Never-married</td>
          <td>Adm-clerical</td>
          <td>Not-in-family</td>
          <td>White</td>
          <td>Male</td>
          <td>2174</td>
          <td>0</td>
          <td>40</td>
          <td>United-States</td>
          <td>&lt;=50K</td>
        </tr>
        <tr>
          <th>1</th>
          <td>50</td>
          <td>Self-emp-not-inc</td>
          <td>83311</td>
          <td>Bachelors</td>
          <td>13</td>
          <td>Married-civ-spouse</td>
          <td>Exec-managerial</td>
          <td>Husband</td>
          <td>White</td>
          <td>Male</td>
          <td>0</td>
          <td>0</td>
          <td>13</td>
          <td>United-States</td>
          <td>&lt;=50K</td>
        </tr>
        <tr>
          <th>2</th>
          <td>38</td>
          <td>Private</td>
          <td>215646</td>
          <td>HS-grad</td>
          <td>9</td>
          <td>Divorced</td>
          <td>Handlers-cleaners</td>
          <td>Not-in-family</td>
          <td>White</td>
          <td>Male</td>
          <td>0</td>
          <td>0</td>
          <td>40</td>
          <td>United-States</td>
          <td>&lt;=50K</td>
        </tr>
        <tr>
          <th>3</th>
          <td>53</td>
          <td>Private</td>
          <td>234721</td>
          <td>11th</td>
          <td>7</td>
          <td>Married-civ-spouse</td>
          <td>Handlers-cleaners</td>
          <td>Husband</td>
          <td>Black</td>
          <td>Male</td>
          <td>0</td>
          <td>0</td>
          <td>40</td>
          <td>United-States</td>
          <td>&lt;=50K</td>
        </tr>
        <tr>
          <th>4</th>
          <td>28</td>
          <td>Private</td>
          <td>338409</td>
          <td>Bachelors</td>
          <td>13</td>
          <td>Married-civ-spouse</td>
          <td>Prof-specialty</td>
          <td>Wife</td>
          <td>Black</td>
          <td>Female</td>
          <td>0</td>
          <td>0</td>
          <td>40</td>
          <td>Cuba</td>
          <td>&lt;=50K</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    label = '<=50K'
    set(train[label])




.. parsed-literal::
    {'<=50K', '>50K'}



.. code:: ipython3

    set(test[label])




.. parsed-literal::
    {'<=50K', '>50K'}



.. code:: ipython3

    import numpy
    import pandas
    X_train = train.drop(label, axis=1)
    y_train = train[label] == '>50K'
    y_train = pandas.Series(numpy.array([1.0 if y else 0.0 for y in y_train]))
    X_test = test.drop(label, axis=1)
    y_test = test[label] == '>50K'
    y_test = pandas.Series(numpy.array([1.0 if y else 0.0 for y in y_test]))

.. code:: ipython3

    train.dtypes




.. parsed-literal::
    age                int64
    workclass         object
    fnlwgt             int64
    education         object
    education_num      int64
    marital_status    object
    occupation        object
    relationship      object
    race              object
    sex               object
    capital_gain       int64
    capital_loss       int64
    hours_per_week     int64
    native_country    object
    <=50K             object
    dtype: object



La variable *fnlwgt* représente une forme de pondération : le nombre
d’individus que chaque observation représente. Elle ne doit pas servir à
la prédiction, comme pondération qu’on ignorera : cette pondération
n’est pas liée aux données mais à l’échantillon et elle est impossible à
construire. Il faut s’en passer.

.. code:: ipython3

    X_train = X_train.drop(['fnlwgt'], axis=1).copy()
    X_test = X_test.drop(['fnlwgt'], axis=1).copy()

catégories
----------

On garde la liste des variables catégorielles.

.. code:: ipython3

    cat_col = list(_ for _ in X_train.select_dtypes("object").columns)
    cat_col




.. parsed-literal::
    ['workclass',
     'education',
     'marital_status',
     'occupation',
     'relationship',
     'race',
     'sex',
     'native_country']



La fonction
`get_dummies <https://pandas.pydata.org/pandas-docs/stable/generated/pandas.get_dummies.html>`__
est pratique mais problématique si les modalités de la base
d’apprentissage et la base de test sont différentes, ce qui est
fréquemment le cas pour les catégories peu fréquentes. Il faudrait
regrouper les deux bases, l’appliquer puis sépareer à nouveau. Trop
long. On veut utiliser
`OneHotEncoder <http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html>`__
et
`LabelEncoder <http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html>`__
mais ce n’est pas très pratique parce que
`OneHotEncoder <http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html>`__
n’accepte que les colonnes entières et
`LabelEncoder <http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html>`__
et peut traiter qu’une colonne à la fois.

.. code:: ipython3

    from sklearn.pipeline import make_pipeline
    from sklearn.preprocessing import LabelEncoder, OneHotEncoder
    pipe = make_pipeline(LabelEncoder(), OneHotEncoder())
    try:
        pipe.fit(X_train[cat_col[0]], y_train)
    except Exception as e:
        print(e)


.. parsed-literal::
    fit_transform() takes 2 positional arguments but 3 were given


On utilise
`OneHotEncoder <http://contrib.scikit-learn.org/categorical-encoding/>`__
du module
`category_encoders <http://contrib.scikit-learn.org/categorical-encoding/>`__.

.. code:: ipython3

    from category_encoders import OneHotEncoder
    ce = OneHotEncoder(cols=cat_col, handle_missing='value',
                       drop_invariant=False, handle_unknown='value')
    X_train_cat = ce.fit_transform(X_train)
    X_train_cat.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>age</th>
          <th>workclass_1</th>
          <th>workclass_2</th>
          <th>workclass_3</th>
          <th>workclass_4</th>
          <th>workclass_5</th>
          <th>workclass_6</th>
          <th>workclass_7</th>
          <th>workclass_8</th>
          <th>workclass_9</th>
          <th>...</th>
          <th>native_country_33</th>
          <th>native_country_34</th>
          <th>native_country_35</th>
          <th>native_country_36</th>
          <th>native_country_37</th>
          <th>native_country_38</th>
          <th>native_country_39</th>
          <th>native_country_40</th>
          <th>native_country_41</th>
          <th>native_country_42</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>39</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>1</th>
          <td>50</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>2</th>
          <td>38</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>3</th>
          <td>53</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>4</th>
          <td>28</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    <p>5 rows × 107 columns</p>
    </div>



C’est assez compliqué à lire. On renomme les colonnes avec les noms des
catégories originaux. Il y a probablement mieux mais ce code fonctionne.

.. code:: ipython3

    def rename_columns(df, ce):
        rev_mapping = {r['col']: r['mapping'] for r in ce.category_mapping}
        cols = []
        for c in df.columns:
            if '_' not in c:
                cols.append(c)
                continue
            spl = c.split('_')
            col = "_".join(spl[:-1])
            try:
                nb = int(spl[-1])    
                mapping  = rev_mapping[col]        
                cols.append(str(col) + "__" + str(mapping.index[nb]))
            except ValueError:
                cols.append(c)
        df.columns = cols + list(df.columns)[len(cols):]
    
    rename_columns(X_train_cat, ce)
    X_train_cat.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>age</th>
          <th>workclass__Self-emp-not-inc</th>
          <th>workclass__Private</th>
          <th>workclass__Federal-gov</th>
          <th>workclass__Local-gov</th>
          <th>workclass__?</th>
          <th>workclass__Self-emp-inc</th>
          <th>workclass__Without-pay</th>
          <th>workclass__Never-worked</th>
          <th>workclass__nan</th>
          <th>...</th>
          <th>native_country__Scotland</th>
          <th>native_country__Trinadad&amp;Tobago</th>
          <th>native_country__Greece</th>
          <th>native_country__Nicaragua</th>
          <th>native_country__Vietnam</th>
          <th>native_country__Hong</th>
          <th>native_country__Ireland</th>
          <th>native_country__Hungary</th>
          <th>native_country__Holand-Netherlands</th>
          <th>native_country__nan</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>39</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>1</th>
          <td>50</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>2</th>
          <td>38</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>3</th>
          <td>53</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>4</th>
          <td>28</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    <p>5 rows × 107 columns</p>
    </div>



C’est plus clair. Une dernière remarque sur le paramètre
*handle_missing* et *handle_unknown*. Le premier définit la façon de
gérer les valeurs manquantes, le premier la façon dont le modèle doit
gérer les catégories nouvelles, c’est-à-dire les catégories non vues
dans la base d’apprentissage.

premier jet
-----------

On construit un
`pipeline <http://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html>`__.

.. code:: ipython3

    from sklearn.linear_model import LogisticRegression
    pipe = make_pipeline(
                OneHotEncoder(cols=cat_col, handle_missing='value',
                              drop_invariant=False, handle_unknown='value'), 
                LogisticRegression())
    pipe.fit(X_train, y_train)


.. parsed-literal::
    C:\xavierdupre\__home_\github_fork\scikit-learn\sklearn\linear_model\_logistic.py:764: ConvergenceWarning: lbfgs failed to converge (status=1):
    STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
    Increase the number of iterations (max_iter) or scale the data as shown in:
        https://scikit-learn.org/stable/modules/preprocessing.html
    Please also refer to the documentation for alternative solver options:
        https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
      extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)




.. parsed-literal::

    Pipeline(memory=None,
             steps=[('onehotencoder',
                     OneHotEncoder(cols=['workclass', 'education', 'marital_status',
                                         'occupation', 'relationship', 'race',
                                         'sex', 'native_country'],
                                   drop_invariant=False, handle_missing='value',
                                   handle_unknown='value', return_df=True,
                                   use_cat_names=False, verbose=0)),
                    ('logisticregression',
                     LogisticRegression(C=1.0, class_weight=None, dual=False,
                                        fit_intercept=True, intercept_scaling=1,
                                        l1_ratio=None, max_iter=100,
                                        multi_class='auto', n_jobs=None,
                                        penalty='l2', random_state=None,
                                        solver='lbfgs', tol=0.0001, verbose=0,
                                        warm_start=False))],
             verbose=False)



.. code:: ipython3

    pipe.score(X_test, y_test)




.. parsed-literal::
    0.8426386585590566



On essaye avec une
`RandomForest <http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html>`__.

.. code:: ipython3

    from sklearn.ensemble import RandomForestClassifier
    pipe2 = make_pipeline(ce, RandomForestClassifier(n_estimators=100))
    pipe2.fit(X_train, y_train)




.. parsed-literal::
    Pipeline(memory=None,
             steps=[('onehotencoder',
                     OneHotEncoder(cols=['workclass', 'education', 'marital_status',
                                         'occupation', 'relationship', 'race',
                                         'sex', 'native_country'],
                                   drop_invariant=False, handle_missing='value',
                                   handle_unknown='value', return_df=True,
                                   use_cat_names=False, verbose=0)),
                    ('randomforestclassifier',
                     RandomForestClassifier(bootstrap=True, ccp_alpha=0.0,
                                            class_weight=None, criterion='gini',
                                            max_depth=None, max_features='auto',
                                            max_leaf_nodes=None, max_samples=None,
                                            min_impurity_decrease=0.0,
                                            min_impurity_split=None,
                                            min_samples_leaf=1, min_samples_split=2,
                                            min_weight_fraction_leaf=0.0,
                                            n_estimators=100, n_jobs=None,
                                            oob_score=False, random_state=None,
                                            verbose=0, warm_start=False))],
             verbose=False)



.. code:: ipython3

    pipe2.score(X_test, y_test)




.. parsed-literal::
    0.8448498249493275



.. code:: ipython3

    pipe2.steps[-1][-1].feature_importances_[:5]




.. parsed-literal::
    array([0.22779679, 0.00529596, 0.00965327, 0.01187745, 0.00584904])



On regarde l’\ `importance des
features <http://scikit-learn.org/stable/auto_examples/ensemble/plot_forest_importances.html>`__.

.. code:: ipython3

    import pandas
    df = pandas.DataFrame(dict(name=X_train_cat.columns, 
                               importance=pipe2.steps[-1][-1].feature_importances_))
    df = df.sort_values("importance", ascending=False).reset_index(drop=True)
    df = df.set_index('name')
    df.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>importance</th>
        </tr>
        <tr>
          <th>name</th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>age</th>
          <td>0.227797</td>
        </tr>
        <tr>
          <th>hours_per_week</th>
          <td>0.113923</td>
        </tr>
        <tr>
          <th>capital_gain</th>
          <td>0.110967</td>
        </tr>
        <tr>
          <th>marital_status__Divorced</th>
          <td>0.067712</td>
        </tr>
        <tr>
          <th>education_num</th>
          <td>0.064527</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    import matplotlib.pyplot as plt
    fig, ax = plt.subplots(1, 1, figsize=(6, 4))
    df[:10].plot.barh(ax=ax)
    ax.set_title('Importance des variables - RandomForest');



.. image:: adult_cat_30_0.png


On compare avec
`XGBoost <http://xgboost.readthedocs.io/en/latest/model.html>`__. L’âge
semble la variable plus importante. Cela dit, si ce graphique donne
quelques pistes, ce n’est pas la vérité car les variables peuvent être
corrélées. Deux variables corrélées sont interchangeables.

.. code:: ipython3

    from xgboost import XGBClassifier
    pipe3 = make_pipeline(ce, XGBClassifier())
    pipe3.fit(X_train, y_train)




.. parsed-literal::
    Pipeline(memory=None,
             steps=[('onehotencoder',
                     OneHotEncoder(cols=['workclass', 'education', 'marital_status',
                                         'occupation', 'relationship', 'race',
                                         'sex', 'native_country'],
                                   drop_invariant=False, handle_missing='value',
                                   handle_unknown='value', return_df=True,
                                   use_cat_names=False, verbose=0)),
                    ('xgbclassifier',
                     XGBClassifier(base_score=0.5, booster='gbtree',
                                   colsample_bylevel=1, colsample_bynode=1,
                                   colsample_bytree=1, gamma=0, learning_rate=0.1,
                                   max_delta_step=0, max_depth=3,
                                   min_child_weight=1, missing=None,
                                   n_estimators=100, n_jobs=1, nthread=None,
                                   objective='binary:logistic', random_state=0,
                                   reg_alpha=0, reg_lambda=1, scale_pos_weight=1,
                                   seed=None, silent=None, subsample=1,
                                   verbosity=1))],
             verbose=False)



.. code:: ipython3

    pipe3.score(X_test, y_test)




.. parsed-literal::
    0.869172655242307



.. code:: ipython3

    df = pandas.DataFrame(dict(name=X_train_cat.columns, 
                               importance=pipe3.steps[-1][-1].feature_importances_))
    df = df.sort_values("importance", ascending=False).reset_index(drop=True)
    df = df.set_index('name')
    fig, ax = plt.subplots(1, 1, figsize=(6, 4))
    df[:10].plot.barh(ax=ax)
    ax.set_title('Importance des variables - XGBoost');



.. image:: adult_cat_34_0.png


On retrouve presque les mêmes variables mais pas dans le même ordre. On
essaye un dernier module
`catboost <https://tech.yandex.com/catboost/>`__.

.. code:: ipython3

    from catboost import CatBoostClassifier
    pipe4 = make_pipeline(ce, CatBoostClassifier(iterations=100, verbose=False))
    pipe4.fit(X_train, y_train)




.. parsed-literal::
    Pipeline(memory=None,
             steps=[('onehotencoder',
                     OneHotEncoder(cols=['workclass', 'education', 'marital_status',
                                         'occupation', 'relationship', 'race',
                                         'sex', 'native_country'],
                                   drop_invariant=False, handle_missing='value',
                                   handle_unknown='value', return_df=True,
                                   use_cat_names=False, verbose=0)),
                    ('catboostclassifier',
                     <catboost.core.CatBoostClassifier object at 0x000001E0DF767AC8>)],
             verbose=False)



.. code:: ipython3

    pipe4.score(X_test, y_test)




.. parsed-literal::
    0.0



.. code:: ipython3

    df = pandas.DataFrame(dict(name=X_train_cat.columns, 
                               importance=pipe4.steps[-1][-1].feature_importances_))
    df = df.sort_values("importance", ascending=False).reset_index(drop=True)
    df = df.set_index('name')
    fig, ax = plt.subplots(1, 1, figsize=(6, 4))
    df[:10].plot.barh(ax=ax)
    ax.set_title('Importance des variables - CatBoost');



.. image:: adult_cat_38_0.png


Les modèles sont à peu près d’accord sur la performance mais pas
vraiment sur l’ordre des features les plus importantes. Comme ce sont
tous des random forests, même apprises différemment, on peut supposer
qu’il existe des corrélations entre les variables. Des corrélations au
sens du modèle donc pas nécessairement linéaires.

Courbe ROC
----------

.. code:: ipython3

    from sklearn.metrics import roc_curve
    import warnings
    if len(pipe2.steps[-1][-1].classes_) != len(pipe3.steps[-1][-1].classes_):
        raise Exception("Mésentente classificatoire pipe2 {0} != pipe3 {1}".format(
                        pipe2.steps[-1][-1].classes_, pipe3.steps[-1][-1].classes_))
    if len(pipe2.steps[-1][-1].classes_) != len(pipe4.steps[-1][-1].classes_):
        if not pipe4.steps[-1][-1].classes_:
            # Probably a bug (happens on circleci).
            # Assuming classes are in the same order.
            # See https://github.com/catboost/catboost/blob/master/catboost/python-package/catboost/core.py#L1994
            warnings.warn("pipe4.steps[-1][-1].classes_ is empty.")
            pipe4.steps[-1][-1]._classes = pipe2.steps[-1][-1].classes_
    if len(pipe2.steps[-1][-1].classes_) != len(pipe4.steps[-1][-1].classes_):
        print("Mésentente classificatoire pipe2 {0} != pipe4 {1}".format(
                        pipe2.steps[-1][-1].classes_, pipe4.steps[-1][-1].classes_))
    
    index2 = pipe2.steps[-1][-1].classes_[1]
    index3 = pipe3.steps[-1][-1].classes_[1]
    index4 = pipe4.steps[-1][-1].classes_[1]
    fpr2, tpr2, th2 = roc_curve(y_test, pipe2.predict_proba(X_test)[:, 1], 
                                pos_label=index2, drop_intermediate=False)
    fpr3, tpr3, th3 = roc_curve(y_test, pipe3.predict_proba(X_test)[:, 1], 
                                pos_label=index3, drop_intermediate=False)
    if len(pipe4.steps[-1][-1].classes_) >= 2:
        fpr4, tpr4, th4 = roc_curve(y_test, pipe4.predict_proba(X_test)[:, 1], 
                                    pos_label=index4, drop_intermediate=False)
    else:
        fpr4 = None

.. code:: ipython3

    from sklearn.metrics import auc
    fig, ax = plt.subplots(1, 1, figsize=(4, 4))
    ax.plot(fpr2, tpr2, label='%1.3f RandomForest' % auc(fpr2, tpr2), color='y')
    ax.plot(fpr3, tpr3, label='%1.3f XGBoost' % auc(fpr3, tpr3))
    if fpr4 is not None:
        ax.plot(fpr4, tpr4, label='%1.3f CatBoost' % auc(fpr4, tpr4))
    ax.legend()
    ax.set_title('Courbe ROC pour trois modèles');



.. image:: adult_cat_42_0.png


GridSearch
----------

On cherche à optimiser les hyperparamètres sur la base d’apprentissage.
On vérifie d’abord que les données sont bien identiquement distribuées.
La validation croisée considère des parties contigües de la base de
données. Il arrive que les données ne soient pas tout-à-fait homogènes
et qu’il faille les mélanger. On compare les performances avant et après
mélange pour vérifier que l’ordre des données n’a pas d’incidence.

.. code:: ipython3

    from sklearn.model_selection import cross_val_score
    cross_val_score(pipe2, X_train, y_train, cv=5)




.. parsed-literal::
    array([0.84707508, 0.84152334, 0.84259828, 0.85288698, 0.84766585])



.. code:: ipython3

    from pandas_streaming.df import dataframe_shuffle
    from numpy.random import permutation
    index = permutation(X_train.index)
    X_train_shuffled = X_train.iloc[index, :]
    y_train_shuffled = y_train[index]
    cross_val_score(pipe2, X_train_shuffled, y_train_shuffled, cv=5)




.. parsed-literal::
    array([0.84277599, 0.84797297, 0.84659091, 0.84981572, 0.85089066])



L’ordre de la base n’a pas d’incidence.

.. code:: ipython3

    from sklearn.model_selection import GridSearchCV
    param_grid = {'randomforestclassifier__n_estimators':[10, 20, 50],
                  'randomforestclassifier__min_samples_leaf': [2, 10]}
    cvgrid = GridSearchCV(estimator=pipe2, param_grid=param_grid, verbose=2)
    cvgrid.fit(X_train, y_train)


.. parsed-literal::
    Fitting 5 folds for each of 6 candidates, totalling 30 fits
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=10 


.. parsed-literal::

    [Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.


.. parsed-literal::
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=10, total=   1.2s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=10 


.. parsed-literal::

    [Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    1.1s remaining:    0.0s


.. parsed-literal::
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=10, total=   1.2s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=10 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=10, total=   1.2s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=10 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=10, total=   1.2s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=10 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=10, total=   1.2s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=20 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=20, total=   1.6s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=20 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=20, total=   1.6s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=20 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=20, total=   1.6s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=20 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=20, total=   1.5s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=20 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=20, total=   1.6s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=50 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=50, total=   2.6s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=50 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=50, total=   3.2s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=50 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=50, total=   3.4s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=50 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=50, total=   3.6s
    [CV] randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=50 
    [CV]  randomforestclassifier__min_samples_leaf=2, randomforestclassifier__n_estimators=50, total=   2.8s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=10 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=10, total=   1.4s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=10 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=10, total=   1.2s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=10 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=10, total=   1.1s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=10 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=10, total=   1.1s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=10 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=10, total=   1.1s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=20 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=20, total=   1.4s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=20 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=20, total=   1.4s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=20 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=20, total=   1.5s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=20 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=20, total=   1.6s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=20 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=20, total=   1.6s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=50 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=50, total=   2.4s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=50 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=50, total=   2.8s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=50 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=50, total=   2.9s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=50 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=50, total=   3.9s
    [CV] randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=50 
    [CV]  randomforestclassifier__min_samples_leaf=10, randomforestclassifier__n_estimators=50, total=   3.1s


.. parsed-literal::

    [Parallel(n_jobs=1)]: Done  30 out of  30 | elapsed:   58.3s finished




.. parsed-literal::
    GridSearchCV(cv=None, error_score=nan,
                 estimator=Pipeline(memory=None,
                                    steps=[('onehotencoder',
                                            OneHotEncoder(cols=['workclass',
                                                                'education',
                                                                'marital_status',
                                                                'occupation',
                                                                'relationship',
                                                                'race', 'sex',
                                                                'native_country'],
                                                          drop_invariant=False,
                                                          handle_missing='value',
                                                          handle_unknown='value',
                                                          return_df=True,
                                                          use_cat_names=False,
                                                          verbose=0)),
                                           ('randomforestclassifier...
                                                                   min_weight_fraction_leaf=0.0,
                                                                   n_estimators=100,
                                                                   n_jobs=None,
                                                                   oob_score=False,
                                                                   random_state=None,
                                                                   verbose=0,
                                                                   warm_start=False))],
                                    verbose=False),
                 iid='deprecated', n_jobs=None,
                 param_grid={'randomforestclassifier__min_samples_leaf': [2, 10],
                             'randomforestclassifier__n_estimators': [10, 20, 50]},
                 pre_dispatch='2*n_jobs', refit=True, return_train_score=False,
                 scoring=None, verbose=2)



.. code:: ipython3

    import pandas
    df = pandas.DataFrame(cvgrid.cv_results_['params'])
    df['mean_fit_time'] = cvgrid.cv_results_['mean_fit_time']
    df['mean_test_score'] = cvgrid.cv_results_['mean_test_score']
    df.sort_values('mean_test_score')






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>randomforestclassifier__min_samples_leaf</th>
          <th>randomforestclassifier__n_estimators</th>
          <th>mean_fit_time</th>
          <th>mean_test_score</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>3</th>
          <td>10</td>
          <td>10</td>
          <td>1.069937</td>
          <td>0.857437</td>
        </tr>
        <tr>
          <th>4</th>
          <td>10</td>
          <td>20</td>
          <td>1.351183</td>
          <td>0.858635</td>
        </tr>
        <tr>
          <th>5</th>
          <td>10</td>
          <td>50</td>
          <td>2.862142</td>
          <td>0.859034</td>
        </tr>
        <tr>
          <th>0</th>
          <td>2</td>
          <td>10</td>
          <td>1.094900</td>
          <td>0.861092</td>
        </tr>
        <tr>
          <th>1</th>
          <td>2</td>
          <td>20</td>
          <td>1.456303</td>
          <td>0.863641</td>
        </tr>
        <tr>
          <th>2</th>
          <td>2</td>
          <td>50</td>
          <td>2.944120</td>
          <td>0.863764</td>
        </tr>
      </tbody>
    </table>
    </div>



Il faudrait continuer à explorer les hyperparamètres et confirmer sur la
base de test. A priori, cela marche mieux avec plus d’arbres.

Features polynômiales
---------------------

On essaye même si cela a peu de chance d’aboutir compte tenu des
variables, principalement catégorielles, et du fait qu’on utilise une
forêt aléatoire.

.. code:: ipython3

    from sklearn.preprocessing import PolynomialFeatures
    pipe5 = make_pipeline(ce, PolynomialFeatures(), RandomForestClassifier())
    pipe5.fit(X_train, y_train)




.. parsed-literal::
    Pipeline(memory=None,
             steps=[('onehotencoder',
                     OneHotEncoder(cols=['workclass', 'education', 'marital_status',
                                         'occupation', 'relationship', 'race',
                                         'sex', 'native_country'],
                                   drop_invariant=False, handle_missing='value',
                                   handle_unknown='value', return_df=True,
                                   use_cat_names=False, verbose=0)),
                    ('polynomialfeatures',
                     PolynomialFeatures(degree=2, include_bias=True,
                                        int...
                     RandomForestClassifier(bootstrap=True, ccp_alpha=0.0,
                                            class_weight=None, criterion='gini',
                                            max_depth=None, max_features='auto',
                                            max_leaf_nodes=None, max_samples=None,
                                            min_impurity_decrease=0.0,
                                            min_impurity_split=None,
                                            min_samples_leaf=1, min_samples_split=2,
                                            min_weight_fraction_leaf=0.0,
                                            n_estimators=100, n_jobs=None,
                                            oob_score=False, random_state=None,
                                            verbose=0, warm_start=False))],
             verbose=False)



.. code:: ipython3

    pipe5.score(X_test, y_test)




.. parsed-literal::
    0.8463853571647934



Ca n’améliore pas.

Interprétation
--------------

On souhaite en savoir plus sur les variables.

.. code:: ipython3

    conc = pandas.concat([X_train_cat, pandas.Series(y_train)], axis=1)
    conc.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>age</th>
          <th>workclass__Self-emp-not-inc</th>
          <th>workclass__Private</th>
          <th>workclass__Federal-gov</th>
          <th>workclass__Local-gov</th>
          <th>workclass__?</th>
          <th>workclass__Self-emp-inc</th>
          <th>workclass__Without-pay</th>
          <th>workclass__Never-worked</th>
          <th>workclass__nan</th>
          <th>...</th>
          <th>native_country__Trinadad&amp;Tobago</th>
          <th>native_country__Greece</th>
          <th>native_country__Nicaragua</th>
          <th>native_country__Vietnam</th>
          <th>native_country__Hong</th>
          <th>native_country__Ireland</th>
          <th>native_country__Hungary</th>
          <th>native_country__Holand-Netherlands</th>
          <th>native_country__nan</th>
          <th>0</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>39</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>1</th>
          <td>50</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>2</th>
          <td>38</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>3</th>
          <td>53</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>4</th>
          <td>28</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0.0</td>
        </tr>
      </tbody>
    </table>
    <p>5 rows × 108 columns</p>
    </div>



.. code:: ipython3

    corr = conc.corr()

.. code:: ipython3

    from seaborn import clustermap
    
    clustermap(corr, center=0, cmap="vlag", linewidths=.75, figsize=(15, 15));


.. parsed-literal::
    c:\python372_x64\lib\site-packages\statsmodels\tools\_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
      import pandas.util.testing as tm
    c:\python372_x64\lib\site-packages\seaborn\matrix.py:624: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance.
      warnings.warn(msg)



.. image:: adult_cat_57_1.png


Ce n’est pas facile à voir. Il faudrait essayer avec
`bokeh <https://bokeh.pydata.org/en/latest/docs/gallery/les_mis.html>`__
ou essayer de procéder autrement.

ACM
---

Ce qui suit n’est pas tout-à-fait une
`ACM <https://fr.wikipedia.org/wiki/Analyse_des_correspondances_multiples>`__
mais cela s’en inspire. On considère les variables comme des
observations et on les projette sur des plans définis par les axes d’une
`ACP <https://fr.wikipedia.org/wiki/Analyse_en_composantes_principales>`__.
On normalise également car on mélange variables continues et variables
binaires d’ordre de grandeur différents. Les calculs sont plus précis
lorsque les matrices ont des coefficients de même ordre. Le dernier
exercice de cet examen `Programmation ENSAE
2006 <http://www.xavierdupre.fr/site2013/enseignements/tdnote/ecrit_2006.pdf>`__
achèvera de vous convaincre.

.. code:: ipython3

    from sklearn.preprocessing import StandardScaler
    import pandas
    rows_cat = pandas.DataFrame(StandardScaler().fit_transform(X_train_cat))
    rows_cat.columns = X_train_cat.columns
    rows_cat = rows_cat.T
    rows_cat.head(n=2)






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>0</th>
          <th>1</th>
          <th>2</th>
          <th>3</th>
          <th>4</th>
          <th>5</th>
          <th>6</th>
          <th>7</th>
          <th>8</th>
          <th>9</th>
          <th>...</th>
          <th>32551</th>
          <th>32552</th>
          <th>32553</th>
          <th>32554</th>
          <th>32555</th>
          <th>32556</th>
          <th>32557</th>
          <th>32558</th>
          <th>32559</th>
          <th>32560</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>age</th>
          <td>0.030671</td>
          <td>0.837109</td>
          <td>-0.042642</td>
          <td>1.057047</td>
          <td>-0.775768</td>
          <td>-0.115955</td>
          <td>0.763796</td>
          <td>0.983734</td>
          <td>-0.555830</td>
          <td>0.250608</td>
          <td>...</td>
          <td>-0.482518</td>
          <td>0.323921</td>
          <td>-0.482518</td>
          <td>1.057047</td>
          <td>-1.215643</td>
          <td>-0.849080</td>
          <td>0.103983</td>
          <td>1.423610</td>
          <td>-1.215643</td>
          <td>0.983734</td>
        </tr>
        <tr>
          <th>workclass__Self-emp-not-inc</th>
          <td>4.907700</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>...</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
          <td>-0.203761</td>
        </tr>
      </tbody>
    </table>
    <p>2 rows × 32561 columns</p>
    </div>



.. code:: ipython3

    from sklearn.decomposition import PCA
    pca = PCA(n_components=3)
    pca.fit(rows_cat)




.. parsed-literal::
    PCA(copy=True, iterated_power='auto', n_components=3, random_state=None,
        svd_solver='auto', tol=0.0, whiten=False)



.. code:: ipython3

    import pandas
    tr = pandas.DataFrame(pca.transform(rows_cat))
    tr.columns = ['axe1', 'axe2', 'axe3']
    tr.index = rows_cat.index
    tr.sort_values('axe1').head(n=2)






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>axe1</th>
          <th>axe2</th>
          <th>axe3</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>sex__nan</th>
          <td>-129.499567</td>
          <td>60.907917</td>
          <td>29.421935</td>
        </tr>
        <tr>
          <th>marital_status__Married-civ-spouse</th>
          <td>-106.135296</td>
          <td>-11.383937</td>
          <td>-54.355587</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    ax = tr.plot(x='axe1', y='axe2', kind='scatter', figsize=(10, 10))
    for t, (x, y, z) in tr.iterrows():
        ax.text(x, y, t, fontsize=10, rotation=10)
    ax.set_title("ACP sur les variables - axe 1, 2");



.. image:: adult_cat_63_0.png


.. code:: ipython3

    ax = tr.plot(x='axe1', y='axe3', kind='scatter', figsize=(10, 10))
    for t, (x, y, z) in tr.iterrows():
        ax.text(x, z, t, fontsize=10, rotation=10)
    ax.set_title("ACP sur les variables - axe 1, 3");



.. image:: adult_cat_64_0.png


On voit quelques variables à supprimer car très corrélées comme la
relation ou la situation maritale. On voit aussi que les deux genres
homme/femme sont opposés. On voit aussi que certaines catégories sont
très proches comme Prof, Masters ou diplômés. Il est probable que le
modèle de prédiction ne pâtisse pas du regroupement de ces trois
catégories. On utilise `bokeh <https://bokeh.pydata.org/en/latest/>`__
pour pouvoir zoomer.

.. code:: ipython3

    import bokeh, bokeh.io as bio
    bio.output_notebook()





.. raw:: html


    <div class="bk-root">
        <a href="https://bokeh.org" target="_blank" class="bk-logo bk-logo-small bk-logo-notebook"></a>
        <span id="1001">Loading BokehJS ...</span>
    </div>




.. code:: ipython3

    from bokeh.plotting import figure, show
    p = figure(title="ACP sur les variables - axe 1, 2")
    p.circle(tr["axe1"], tr["axe2"])
    p.text(tr["axe1"], tr["axe2"], tr.index,
           text_font_size="8pt", text_baseline="middle", angle=0.1)
    show(p)





.. raw:: html







    <div class="bk-root" id="ea645201-cb7b-4b13-b5ac-791d7836e501" data-root-id="1002"></div>





Analyse d’erreur
----------------

On recherche les erreurs les plus flagrantes, celles dont le score est
élevé.

.. code:: ipython3

    pred = pipe.predict(X_test)
    proba = pipe.predict_proba(X_test)
    pred2 = pipe2.predict(X_test)
    proba2 = pipe2.predict_proba(X_test)
    pred3 = pipe3.predict(X_test)
    proba3 = pipe3.predict_proba(X_test)
    pred4 = pipe4.predict(X_test)
    proba4 = pipe4.predict_proba(X_test)

.. code:: ipython3

    data = pandas.concat([
                pandas.DataFrame(y_test.astype(float).values, columns=['y_test']),
                pandas.DataFrame(pred, columns=['pred1']),
                pandas.DataFrame(proba[:,1], columns=['P1(>=50K)']),
                pandas.DataFrame(pred2, columns=['pred2']),
                pandas.DataFrame(proba2[:,1], columns=['P2(>=50K)']),
                pandas.DataFrame(pred3, columns=['pred3']),
                pandas.DataFrame(proba3[:,1], columns=['P3(>=50K)']),
                pandas.DataFrame(pred4, columns=['pred4']),
                pandas.DataFrame(proba4[:,1], columns=['P4(>=50K)']),
                X_test,
                ], axis=1)
    data.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>y_test</th>
          <th>pred1</th>
          <th>P1(&gt;=50K)</th>
          <th>pred2</th>
          <th>P2(&gt;=50K)</th>
          <th>pred3</th>
          <th>P3(&gt;=50K)</th>
          <th>pred4</th>
          <th>P4(&gt;=50K)</th>
          <th>age</th>
          <th>...</th>
          <th>education_num</th>
          <th>marital_status</th>
          <th>occupation</th>
          <th>relationship</th>
          <th>race</th>
          <th>sex</th>
          <th>capital_gain</th>
          <th>capital_loss</th>
          <th>hours_per_week</th>
          <th>native_country</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0.0</td>
          <td>0.0</td>
          <td>0.010831</td>
          <td>0.0</td>
          <td>0.000000</td>
          <td>0.0</td>
          <td>0.007642</td>
          <td>0.0</td>
          <td>0.000384</td>
          <td>25</td>
          <td>...</td>
          <td>7</td>
          <td>Never-married</td>
          <td>Machine-op-inspct</td>
          <td>Own-child</td>
          <td>Black</td>
          <td>Male</td>
          <td>0</td>
          <td>0</td>
          <td>40</td>
          <td>United-States</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0.0</td>
          <td>0.0</td>
          <td>0.179896</td>
          <td>0.0</td>
          <td>0.030000</td>
          <td>0.0</td>
          <td>0.203497</td>
          <td>0.0</td>
          <td>0.207168</td>
          <td>38</td>
          <td>...</td>
          <td>9</td>
          <td>Married-civ-spouse</td>
          <td>Farming-fishing</td>
          <td>Husband</td>
          <td>White</td>
          <td>Male</td>
          <td>0</td>
          <td>0</td>
          <td>50</td>
          <td>United-States</td>
        </tr>
        <tr>
          <th>2</th>
          <td>1.0</td>
          <td>1.0</td>
          <td>0.526712</td>
          <td>1.0</td>
          <td>0.526667</td>
          <td>0.0</td>
          <td>0.277167</td>
          <td>0.0</td>
          <td>0.471523</td>
          <td>28</td>
          <td>...</td>
          <td>12</td>
          <td>Married-civ-spouse</td>
          <td>Protective-serv</td>
          <td>Husband</td>
          <td>White</td>
          <td>Male</td>
          <td>0</td>
          <td>0</td>
          <td>40</td>
          <td>United-States</td>
        </tr>
        <tr>
          <th>3</th>
          <td>1.0</td>
          <td>1.0</td>
          <td>0.789027</td>
          <td>1.0</td>
          <td>0.930000</td>
          <td>1.0</td>
          <td>0.984138</td>
          <td>1.0</td>
          <td>0.987516</td>
          <td>44</td>
          <td>...</td>
          <td>10</td>
          <td>Married-civ-spouse</td>
          <td>Machine-op-inspct</td>
          <td>Husband</td>
          <td>Black</td>
          <td>Male</td>
          <td>7688</td>
          <td>0</td>
          <td>40</td>
          <td>United-States</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0.0</td>
          <td>0.0</td>
          <td>0.009680</td>
          <td>0.0</td>
          <td>0.000000</td>
          <td>0.0</td>
          <td>0.002210</td>
          <td>0.0</td>
          <td>0.000151</td>
          <td>18</td>
          <td>...</td>
          <td>10</td>
          <td>Never-married</td>
          <td>?</td>
          <td>Own-child</td>
          <td>White</td>
          <td>Female</td>
          <td>0</td>
          <td>0</td>
          <td>30</td>
          <td>United-States</td>
        </tr>
      </tbody>
    </table>
    <p>5 rows × 22 columns</p>
    </div>



.. code:: ipython3

    data[data.y_test != data.pred4].sort_values('P4(>=50K)', ascending=False).head().T






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>3605</th>
          <th>2926</th>
          <th>13783</th>
          <th>2247</th>
          <th>13128</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>y_test</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>pred1</th>
          <td>1</td>
          <td>1</td>
          <td>1</td>
          <td>1</td>
          <td>1</td>
        </tr>
        <tr>
          <th>P1(&gt;=50K)</th>
          <td>1</td>
          <td>0.793878</td>
          <td>0.764859</td>
          <td>0.788413</td>
          <td>0.832539</td>
        </tr>
        <tr>
          <th>pred2</th>
          <td>1</td>
          <td>1</td>
          <td>1</td>
          <td>1</td>
          <td>1</td>
        </tr>
        <tr>
          <th>P2(&gt;=50K)</th>
          <td>0.74</td>
          <td>0.74</td>
          <td>0.937333</td>
          <td>0.94</td>
          <td>0.65</td>
        </tr>
        <tr>
          <th>pred3</th>
          <td>1</td>
          <td>1</td>
          <td>1</td>
          <td>1</td>
          <td>1</td>
        </tr>
        <tr>
          <th>P3(&gt;=50K)</th>
          <td>0.972606</td>
          <td>0.900088</td>
          <td>0.933867</td>
          <td>0.796695</td>
          <td>0.913922</td>
        </tr>
        <tr>
          <th>pred4</th>
          <td>1</td>
          <td>1</td>
          <td>1</td>
          <td>1</td>
          <td>1</td>
        </tr>
        <tr>
          <th>P4(&gt;=50K)</th>
          <td>0.998917</td>
          <td>0.989239</td>
          <td>0.980988</td>
          <td>0.944513</td>
          <td>0.931411</td>
        </tr>
        <tr>
          <th>age</th>
          <td>36</td>
          <td>65</td>
          <td>51</td>
          <td>55</td>
          <td>48</td>
        </tr>
        <tr>
          <th>workclass</th>
          <td>Self-emp-not-inc</td>
          <td>Self-emp-not-inc</td>
          <td>Private</td>
          <td>Self-emp-inc</td>
          <td>Local-gov</td>
        </tr>
        <tr>
          <th>education</th>
          <td>HS-grad</td>
          <td>Masters</td>
          <td>Some-college</td>
          <td>Prof-school</td>
          <td>Bachelors</td>
        </tr>
        <tr>
          <th>education_num</th>
          <td>9</td>
          <td>14</td>
          <td>10</td>
          <td>15</td>
          <td>13</td>
        </tr>
        <tr>
          <th>marital_status</th>
          <td>Married-civ-spouse</td>
          <td>Married-spouse-absent</td>
          <td>Married-civ-spouse</td>
          <td>Married-civ-spouse</td>
          <td>Separated</td>
        </tr>
        <tr>
          <th>occupation</th>
          <td>Exec-managerial</td>
          <td>Prof-specialty</td>
          <td>Exec-managerial</td>
          <td>Prof-specialty</td>
          <td>Prof-specialty</td>
        </tr>
        <tr>
          <th>relationship</th>
          <td>Husband</td>
          <td>Not-in-family</td>
          <td>Husband</td>
          <td>Husband</td>
          <td>Unmarried</td>
        </tr>
        <tr>
          <th>race</th>
          <td>Asian-Pac-Islander</td>
          <td>White</td>
          <td>White</td>
          <td>White</td>
          <td>White</td>
        </tr>
        <tr>
          <th>sex</th>
          <td>Male</td>
          <td>Female</td>
          <td>Male</td>
          <td>Male</td>
          <td>Female</td>
        </tr>
        <tr>
          <th>capital_gain</th>
          <td>41310</td>
          <td>7978</td>
          <td>0</td>
          <td>0</td>
          <td>7443</td>
        </tr>
        <tr>
          <th>capital_loss</th>
          <td>0</td>
          <td>0</td>
          <td>1902</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>hours_per_week</th>
          <td>90</td>
          <td>40</td>
          <td>40</td>
          <td>55</td>
          <td>45</td>
        </tr>
        <tr>
          <th>native_country</th>
          <td>South</td>
          <td>United-States</td>
          <td>United-States</td>
          <td>United-States</td>
          <td>United-States</td>
        </tr>
      </tbody>
    </table>
    </div>



Tous les modèles font l’erreur sur ces cinq exemples. Le modèle a toutes
les raisons de décider que les personnes gagnent plus de 50k par an,
beaucoup d’études, plutôt âge ou travaillant beaucoup.

.. code:: ipython3

    wrong_study = data[data.y_test != data.pred4].sort_values('P4(>=50K)', ascending=True).head(n=3).T
    wrong_study






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>10408</th>
          <th>5953</th>
          <th>3059</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>y_test</th>
          <td>1</td>
          <td>1</td>
          <td>1</td>
        </tr>
        <tr>
          <th>pred1</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>P1(&gt;=50K)</th>
          <td>0.0146811</td>
          <td>0.00972505</td>
          <td>0.0145646</td>
        </tr>
        <tr>
          <th>pred2</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>P2(&gt;=50K)</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>pred3</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>P3(&gt;=50K)</th>
          <td>0.00288558</td>
          <td>0.00299048</td>
          <td>0.00561586</td>
        </tr>
        <tr>
          <th>pred4</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>P4(&gt;=50K)</th>
          <td>0.000149386</td>
          <td>0.000395875</td>
          <td>0.000863302</td>
        </tr>
        <tr>
          <th>age</th>
          <td>22</td>
          <td>20</td>
          <td>22</td>
        </tr>
        <tr>
          <th>workclass</th>
          <td>?</td>
          <td>Private</td>
          <td>Private</td>
        </tr>
        <tr>
          <th>education</th>
          <td>Some-college</td>
          <td>12th</td>
          <td>Some-college</td>
        </tr>
        <tr>
          <th>education_num</th>
          <td>10</td>
          <td>8</td>
          <td>10</td>
        </tr>
        <tr>
          <th>marital_status</th>
          <td>Never-married</td>
          <td>Never-married</td>
          <td>Never-married</td>
        </tr>
        <tr>
          <th>occupation</th>
          <td>?</td>
          <td>Other-service</td>
          <td>Sales</td>
        </tr>
        <tr>
          <th>relationship</th>
          <td>Own-child</td>
          <td>Own-child</td>
          <td>Not-in-family</td>
        </tr>
        <tr>
          <th>race</th>
          <td>White</td>
          <td>Black</td>
          <td>White</td>
        </tr>
        <tr>
          <th>sex</th>
          <td>Male</td>
          <td>Male</td>
          <td>Male</td>
        </tr>
        <tr>
          <th>capital_gain</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>capital_loss</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>hours_per_week</th>
          <td>15</td>
          <td>35</td>
          <td>25</td>
        </tr>
        <tr>
          <th>native_country</th>
          <td>?</td>
          <td>United-States</td>
          <td>United-States</td>
        </tr>
      </tbody>
    </table>
    </div>



Ceux-ci sont probablement étudiants et déjà aisés. Il faudrait avoir
quelques informations sur les parents pour confirmer. On recherche les
plus proches voisins dans la base pour voir ce que le modèle répond. Il
faut néanmoins appliquer cela sur la base une fois les catégories
transformées.

.. code:: ipython3

    from sklearn.neighbors import NearestNeighbors
    knn = NearestNeighbors()
    knn.fit(X_train_cat)




.. parsed-literal::
    NearestNeighbors(algorithm='auto', leaf_size=30, metric='minkowski',
                     metric_params=None, n_jobs=None, n_neighbors=5, p=2,
                     radius=1.0)



.. code:: ipython3

    X_test_cat = pipe2.steps[0][-1].transform(X_test)
    X_test_cat.columns = X_train_cat.columns

.. code:: ipython3

    wrong = data[data.y_test != data.pred4].sort_values('P4(>=50K)', ascending=True).head()
    wrong.head(n=2)






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>y_test</th>
          <th>pred1</th>
          <th>P1(&gt;=50K)</th>
          <th>pred2</th>
          <th>P2(&gt;=50K)</th>
          <th>pred3</th>
          <th>P3(&gt;=50K)</th>
          <th>pred4</th>
          <th>P4(&gt;=50K)</th>
          <th>age</th>
          <th>...</th>
          <th>education_num</th>
          <th>marital_status</th>
          <th>occupation</th>
          <th>relationship</th>
          <th>race</th>
          <th>sex</th>
          <th>capital_gain</th>
          <th>capital_loss</th>
          <th>hours_per_week</th>
          <th>native_country</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>10408</th>
          <td>1.0</td>
          <td>0.0</td>
          <td>0.014681</td>
          <td>0.0</td>
          <td>0.0</td>
          <td>0.0</td>
          <td>0.002886</td>
          <td>0.0</td>
          <td>0.000149</td>
          <td>22</td>
          <td>...</td>
          <td>10</td>
          <td>Never-married</td>
          <td>?</td>
          <td>Own-child</td>
          <td>White</td>
          <td>Male</td>
          <td>0</td>
          <td>0</td>
          <td>15</td>
          <td>?</td>
        </tr>
        <tr>
          <th>5953</th>
          <td>1.0</td>
          <td>0.0</td>
          <td>0.009725</td>
          <td>0.0</td>
          <td>0.0</td>
          <td>0.0</td>
          <td>0.002990</td>
          <td>0.0</td>
          <td>0.000396</td>
          <td>20</td>
          <td>...</td>
          <td>8</td>
          <td>Never-married</td>
          <td>Other-service</td>
          <td>Own-child</td>
          <td>Black</td>
          <td>Male</td>
          <td>0</td>
          <td>0</td>
          <td>35</td>
          <td>United-States</td>
        </tr>
      </tbody>
    </table>
    <p>2 rows × 22 columns</p>
    </div>



.. code:: ipython3

    wrong_cat = X_test_cat.iloc[wrong.index, :]
    wrong_cat.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>age</th>
          <th>workclass__Self-emp-not-inc</th>
          <th>workclass__Private</th>
          <th>workclass__Federal-gov</th>
          <th>workclass__Local-gov</th>
          <th>workclass__?</th>
          <th>workclass__Self-emp-inc</th>
          <th>workclass__Without-pay</th>
          <th>workclass__Never-worked</th>
          <th>workclass__nan</th>
          <th>...</th>
          <th>native_country__Scotland</th>
          <th>native_country__Trinadad&amp;Tobago</th>
          <th>native_country__Greece</th>
          <th>native_country__Nicaragua</th>
          <th>native_country__Vietnam</th>
          <th>native_country__Hong</th>
          <th>native_country__Ireland</th>
          <th>native_country__Hungary</th>
          <th>native_country__Holand-Netherlands</th>
          <th>native_country__nan</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>10408</th>
          <td>22</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>5953</th>
          <td>20</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>3059</th>
          <td>22</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>11821</th>
          <td>24</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>12808</th>
          <td>67</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    <p>5 rows × 107 columns</p>
    </div>



.. code:: ipython3

    dist, index = knn.kneighbors(wrong_cat)
    dist




.. parsed-literal::
    array([[2.        , 2.        , 2.23606798, 2.44948974, 2.44948974],
           [1.41421356, 1.41421356, 2.23606798, 2.23606798, 2.23606798],
           [1.        , 1.41421356, 1.41421356, 1.73205081, 1.73205081],
           [1.41421356, 1.73205081, 1.73205081, 1.73205081, 2.        ],
           [2.64575131, 3.31662479, 3.46410162, 3.46410162, 3.74165739]])



.. code:: ipython3

    index




.. parsed-literal::
    array([[18056, 18362,  8757, 13177,   314],
           [24878, 30060, 27902,  8008, 23771],
           [  920, 31176,  5206,  7415, 15308],
           [17303, 18592,  2019, 20325,  7542],
           [  346, 14421,  8214, 11050, 18902]], dtype=int64)



.. code:: ipython3

    train_nn = pandas.concat([X_train, y_train], axis=1).iloc[[24878, 18056, 920], :].T
    train_nn.columns = ['TR-' + str(_) for _ in train_nn.columns]
    train_nn






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>TR-24878</th>
          <th>TR-18056</th>
          <th>TR-920</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>age</th>
          <td>20</td>
          <td>21</td>
          <td>23</td>
        </tr>
        <tr>
          <th>workclass</th>
          <td>Private</td>
          <td>?</td>
          <td>Private</td>
        </tr>
        <tr>
          <th>education</th>
          <td>12th</td>
          <td>Some-college</td>
          <td>Some-college</td>
        </tr>
        <tr>
          <th>education_num</th>
          <td>8</td>
          <td>10</td>
          <td>10</td>
        </tr>
        <tr>
          <th>marital_status</th>
          <td>Never-married</td>
          <td>Never-married</td>
          <td>Never-married</td>
        </tr>
        <tr>
          <th>occupation</th>
          <td>Other-service</td>
          <td>?</td>
          <td>Sales</td>
        </tr>
        <tr>
          <th>relationship</th>
          <td>Own-child</td>
          <td>Own-child</td>
          <td>Not-in-family</td>
        </tr>
        <tr>
          <th>race</th>
          <td>White</td>
          <td>White</td>
          <td>White</td>
        </tr>
        <tr>
          <th>sex</th>
          <td>Male</td>
          <td>Male</td>
          <td>Male</td>
        </tr>
        <tr>
          <th>capital_gain</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>capital_loss</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>hours_per_week</th>
          <td>35</td>
          <td>16</td>
          <td>25</td>
        </tr>
        <tr>
          <th>native_country</th>
          <td>United-States</td>
          <td>United-States</td>
          <td>United-States</td>
        </tr>
        <tr>
          <th>0</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    </div>



Il faut comparer la première colonne avec la quatrième, la seconde avec
la ciinquième et la troisème avec la sixième. Ces exemples sont voisins.
On voit que les exemples sont très proches. Il n’y a qu’une seule valeur
qui change à chaque fois et il est difficile d’expliquer les erreurs.

.. code:: ipython3

    pandas.concat([train_nn, wrong_study], axis=1, sort=True)






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>TR-24878</th>
          <th>TR-18056</th>
          <th>TR-920</th>
          <th>10408</th>
          <th>5953</th>
          <th>3059</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>age</th>
          <td>20</td>
          <td>21</td>
          <td>23</td>
          <td>22</td>
          <td>20</td>
          <td>22</td>
        </tr>
        <tr>
          <th>capital_gain</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>capital_loss</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>education</th>
          <td>12th</td>
          <td>Some-college</td>
          <td>Some-college</td>
          <td>Some-college</td>
          <td>12th</td>
          <td>Some-college</td>
        </tr>
        <tr>
          <th>education_num</th>
          <td>8</td>
          <td>10</td>
          <td>10</td>
          <td>10</td>
          <td>8</td>
          <td>10</td>
        </tr>
        <tr>
          <th>hours_per_week</th>
          <td>35</td>
          <td>16</td>
          <td>25</td>
          <td>15</td>
          <td>35</td>
          <td>25</td>
        </tr>
        <tr>
          <th>marital_status</th>
          <td>Never-married</td>
          <td>Never-married</td>
          <td>Never-married</td>
          <td>Never-married</td>
          <td>Never-married</td>
          <td>Never-married</td>
        </tr>
        <tr>
          <th>native_country</th>
          <td>United-States</td>
          <td>United-States</td>
          <td>United-States</td>
          <td>?</td>
          <td>United-States</td>
          <td>United-States</td>
        </tr>
        <tr>
          <th>occupation</th>
          <td>Other-service</td>
          <td>?</td>
          <td>Sales</td>
          <td>?</td>
          <td>Other-service</td>
          <td>Sales</td>
        </tr>
        <tr>
          <th>race</th>
          <td>White</td>
          <td>White</td>
          <td>White</td>
          <td>White</td>
          <td>Black</td>
          <td>White</td>
        </tr>
        <tr>
          <th>relationship</th>
          <td>Own-child</td>
          <td>Own-child</td>
          <td>Not-in-family</td>
          <td>Own-child</td>
          <td>Own-child</td>
          <td>Not-in-family</td>
        </tr>
        <tr>
          <th>sex</th>
          <td>Male</td>
          <td>Male</td>
          <td>Male</td>
          <td>Male</td>
          <td>Male</td>
          <td>Male</td>
        </tr>
        <tr>
          <th>workclass</th>
          <td>Private</td>
          <td>?</td>
          <td>Private</td>
          <td>?</td>
          <td>Private</td>
          <td>Private</td>
        </tr>
        <tr>
          <th>0</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>y_test</th>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>1</td>
          <td>1</td>
          <td>1</td>
        </tr>
        <tr>
          <th>pred1</th>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>P1(&gt;=50K)</th>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>0.0146811</td>
          <td>0.00972505</td>
          <td>0.0145646</td>
        </tr>
        <tr>
          <th>pred2</th>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>P2(&gt;=50K)</th>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>pred3</th>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>P3(&gt;=50K)</th>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>0.00288558</td>
          <td>0.00299048</td>
          <td>0.00561586</td>
        </tr>
        <tr>
          <th>pred4</th>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>P4(&gt;=50K)</th>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>0.000149386</td>
          <td>0.000395875</td>
          <td>0.000863302</td>
        </tr>
      </tbody>
    </table>
    </div>



Ethique
-------

Le modèle qu’on a appris est-il éthique ? Il n’y a pas de réponse
simples à ce problème car il est difficile de transcrire
mathématiquement le caractère éthique d’un modèle. Dans le cas présent,
admettons que l’on souhaite vérifier que le modèle ne retourne pas une
réponse biaisée en fonction du genre. Une première idée consiste à
enlever la variable pour être sûr sur le modèle n’en tienne pas compte
mais cela ne garantit que la variable genre n’est une variable corrélée
aux autres. Et la corrélation implique ici au sens du modèle ce qui a un
sens plus fort si le modèle n’est pas linéaire. On aura tout-à-faire
enlevé la variable genre si elle ne peut être prédite à partir des
autres.

.. code:: ipython3

    X_train_sex = train.drop([label, 'sex'], axis=1)
    y_train_sex = train['sex'] == 'Male'
    X_test_sex = test.drop([label, 'sex'], axis=1)
    y_test_sex = test['sex'] == 'Male'

.. code:: ipython3

    ce_sex = OneHotEncoder(cols=[_ for _ in cat_col if _ != 'sex'], 
                           handle_missing='value', drop_invariant=False,
                           handle_unknown='value')
    model_sex = make_pipeline(ce_sex, RandomForestClassifier(n_estimators=100))
    model_sex.fit(X_train_sex, y_train_sex)




.. parsed-literal::
    Pipeline(memory=None,
             steps=[('onehotencoder',
                     OneHotEncoder(cols=['workclass', 'education', 'marital_status',
                                         'occupation', 'relationship', 'race',
                                         'native_country'],
                                   drop_invariant=False, handle_missing='value',
                                   handle_unknown='value', return_df=True,
                                   use_cat_names=False, verbose=0)),
                    ('randomforestclassifier',
                     RandomForestClassifier(bootstrap=True, ccp_alpha=0.0,
                                            class_weight=None, criterion='gini',
                                            max_depth=None, max_features='auto',
                                            max_leaf_nodes=None, max_samples=None,
                                            min_impurity_decrease=0.0,
                                            min_impurity_split=None,
                                            min_samples_leaf=1, min_samples_split=2,
                                            min_weight_fraction_leaf=0.0,
                                            n_estimators=100, n_jobs=None,
                                            oob_score=False, random_state=None,
                                            verbose=0, warm_start=False))],
             verbose=False)



.. code:: ipython3

    model_sex.score(X_test_sex, y_test_sex)




.. parsed-literal::
    0.8368650574289048



Il est possible de prédire le genre en fonction des autres variables à
83% près. Ce n’est pas en enlevant la variable qu’on peut empêcher le
modèle d’être biaisé par rapport à cette information. Il n’est pas
évident de retirer toute influence de ce paramètre. On peut choisir de
la garder en supposant que le modèle la choisira plutôt qu’une autre
pour prédire si elle s’avère pertinente. Dans ce cas, on pourrait
comparer combien de fois le modèle change de prédiction si on inverse la
variable *sex*. On rappelle la performance du modèle.

.. code:: ipython3

    pipe2.score(X_test, y_test)




.. parsed-literal::
    0.8448498249493275



On remplace la variable *sex* par la prédiction de l’autre modèle.

.. code:: ipython3

    X_test_modified = X_test.copy()
    X_test_modified['sex'] = model_sex.predict(X_test_sex)
    X_test_modified['sex'] = X_test_modified['sex'].apply(lambda x: 'Male' if x else 'Female')

.. code:: ipython3

    pipe2.score(X_test_modified, y_test)




.. parsed-literal::
    0.843928505620048



Quasiment aucun changement. Inversons la colonne maintenant.

.. code:: ipython3

    X_test_inv = X_test.copy()
    X_test_inv['sex'] = X_test_inv['sex'].apply(lambda x: 'Female' if x == 'Male' else 'Male')

.. code:: ipython3

    pipe2.score(X_test_inv, y_test)




.. parsed-literal::
    0.8505005835022419



Encore mieux. Regardons les différences maintenant.

.. code:: ipython3

    diff1 = X_test['sex'] != X_test_inv['sex']
    diff2 = pipe2.predict(X_test) != pipe2.predict(X_test_inv)
    diff2.sum(), diff1.sum(), diff2.sum() / diff1.sum()




.. parsed-literal::
    (1024, 16281, 0.06289539954548247)



La prédiction change dans 5% des cas. On est sûr que pour ces
observations et ce modèle, le genre a un impact, cela ne veut rien dire
pour les autres. Regardons les cinq premières.

.. code:: ipython3

    look = X_test.copy()
    look['y'] = y_test
    look['prediction_sex'] = model_sex.predict(X_test_sex)
    look[diff2].head().T






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>2</th>
          <th>14</th>
          <th>17</th>
          <th>19</th>
          <th>28</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>age</th>
          <td>28</td>
          <td>48</td>
          <td>43</td>
          <td>40</td>
          <td>54</td>
        </tr>
        <tr>
          <th>workclass</th>
          <td>Local-gov</td>
          <td>Private</td>
          <td>Private</td>
          <td>Private</td>
          <td>Private</td>
        </tr>
        <tr>
          <th>education</th>
          <td>Assoc-acdm</td>
          <td>HS-grad</td>
          <td>HS-grad</td>
          <td>Doctorate</td>
          <td>HS-grad</td>
        </tr>
        <tr>
          <th>education_num</th>
          <td>12</td>
          <td>9</td>
          <td>9</td>
          <td>16</td>
          <td>9</td>
        </tr>
        <tr>
          <th>marital_status</th>
          <td>Married-civ-spouse</td>
          <td>Married-civ-spouse</td>
          <td>Married-civ-spouse</td>
          <td>Married-civ-spouse</td>
          <td>Married-civ-spouse</td>
        </tr>
        <tr>
          <th>occupation</th>
          <td>Protective-serv</td>
          <td>Machine-op-inspct</td>
          <td>Adm-clerical</td>
          <td>Prof-specialty</td>
          <td>Craft-repair</td>
        </tr>
        <tr>
          <th>relationship</th>
          <td>Husband</td>
          <td>Husband</td>
          <td>Wife</td>
          <td>Husband</td>
          <td>Husband</td>
        </tr>
        <tr>
          <th>race</th>
          <td>White</td>
          <td>White</td>
          <td>White</td>
          <td>Asian-Pac-Islander</td>
          <td>White</td>
        </tr>
        <tr>
          <th>sex</th>
          <td>Male</td>
          <td>Male</td>
          <td>Female</td>
          <td>Male</td>
          <td>Male</td>
        </tr>
        <tr>
          <th>capital_gain</th>
          <td>0</td>
          <td>3103</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>capital_loss</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>hours_per_week</th>
          <td>40</td>
          <td>48</td>
          <td>30</td>
          <td>45</td>
          <td>35</td>
        </tr>
        <tr>
          <th>native_country</th>
          <td>United-States</td>
          <td>United-States</td>
          <td>United-States</td>
          <td>?</td>
          <td>United-States</td>
        </tr>
        <tr>
          <th>y</th>
          <td>1</td>
          <td>1</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
        </tr>
        <tr>
          <th>prediction_sex</th>
          <td>True</td>
          <td>True</td>
          <td>False</td>
          <td>True</td>
          <td>True</td>
        </tr>
      </tbody>
    </table>
    </div>



Il existe visible quelques observations à vérifier où la relation
*relationship* et le genre *sex* semble être en contradiction ou plutôt
ne pas prendre en compte tous les types de relations possibles.

.. code:: ipython3

    X_train[['sex', 'relationship', 'age']].groupby(['sex', 'relationship'], as_index=False)\
            .count().pivot('sex', 'relationship', 'age')






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th>relationship</th>
          <th>Husband</th>
          <th>Not-in-family</th>
          <th>Other-relative</th>
          <th>Own-child</th>
          <th>Unmarried</th>
          <th>Wife</th>
        </tr>
        <tr>
          <th>sex</th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>Female</th>
          <td>1</td>
          <td>3875</td>
          <td>430</td>
          <td>2245</td>
          <td>2654</td>
          <td>1566</td>
        </tr>
        <tr>
          <th>Male</th>
          <td>13192</td>
          <td>4430</td>
          <td>551</td>
          <td>2823</td>
          <td>792</td>
          <td>2</td>
        </tr>
      </tbody>
    </table>
    </div>



Pas de conclusion à ce stade. Il faut poursuivre l’exploration `Machine
Learning
éthique <http://www.xavierdupre.fr/app/ensae_teaching_cs/helpsphinx/td_2a_mlplus.html#machine-learning-ethique>`__.

Sélection des variables
-----------------------

Il n’y a pas de méthode optimale pour sélectionner les variables. Il
existe différentes options comme celles proposées par
`scikit-learn/feature_selection <http://scikit-learn.org/stable/modules/classes.html#module-sklearn.feature_selection>`__.
Certaines partent des features brutes, d’autres utilisent le modèle qui
doit être appris. Mais ce n’est pas toujours évident de faire marcher
ces méthodes sur n’importe quel modèle.

.. code:: ipython3

    from sklearn.feature_selection import RFE
    try:
        model = RFE(pipe2)
        model.fit(X_train, y_train)
    except Exception as e:
        print(e)


.. parsed-literal::
    could not convert string to float: 'State-gov'


Dans notre cas, on retire les variables une à une en fonction de
l’indicateur ``feature_importance`` ce qui n’est pas facile car les
variables sont des modalités. Il faut en faire la somme…

.. code:: ipython3

    def grouped_feature_importance(model, datas, cat_col):
        ce = model.steps[0][-1]
        data_cat = ce.fit_transform(datas)    
        rename_columns(data_cat, ce)
        df = pandas.DataFrame(dict(name=data_cat.columns, 
                                   importance=model.steps[-1][-1].feature_importances_))
        df = df.sort_values("importance", ascending=False).reset_index(drop=True)
        df['raw_var'] = df['name'].apply(lambda x: x.split('__')[0])
        gr = df.groupby('raw_var').sum().sort_values('importance', ascending=True).reset_index(drop=False).copy()
        return gr
    
    fi_global = grouped_feature_importance(pipe2, X_train, cat_col)
    fi_global






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>raw_var</th>
          <th>importance</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>race</td>
          <td>0.018372</td>
        </tr>
        <tr>
          <th>1</th>
          <td>sex</td>
          <td>0.019006</td>
        </tr>
        <tr>
          <th>2</th>
          <td>native_country</td>
          <td>0.028188</td>
        </tr>
        <tr>
          <th>3</th>
          <td>capital_loss</td>
          <td>0.034305</td>
        </tr>
        <tr>
          <th>4</th>
          <td>workclass</td>
          <td>0.047340</td>
        </tr>
        <tr>
          <th>5</th>
          <td>education</td>
          <td>0.059821</td>
        </tr>
        <tr>
          <th>6</th>
          <td>education_num</td>
          <td>0.064527</td>
        </tr>
        <tr>
          <th>7</th>
          <td>relationship</td>
          <td>0.083855</td>
        </tr>
        <tr>
          <th>8</th>
          <td>occupation</td>
          <td>0.091443</td>
        </tr>
        <tr>
          <th>9</th>
          <td>marital_status</td>
          <td>0.100458</td>
        </tr>
        <tr>
          <th>10</th>
          <td>capital_gain</td>
          <td>0.110967</td>
        </tr>
        <tr>
          <th>11</th>
          <td>hours_per_week</td>
          <td>0.113923</td>
        </tr>
        <tr>
          <th>12</th>
          <td>age</td>
          <td>0.227797</td>
        </tr>
      </tbody>
    </table>
    </div>



On fait une boucle.

.. code:: ipython3

    kept = list(fi_global.raw_var)
    res = []
    last_removed = None
    while len(kept) > 0:
        cat_col_red = set()
        for col in kept:
            if "__" in col:
                col = "__".join(col.split('__')[:-1])
            cat_col_red.add(col)
        cat_col_red = list(cat_col_red)
        X_train_reduced = X_train[cat_col_red]
        X_test_reduced = X_test[cat_col_red]    
        ce = OneHotEncoder(cols=cat_col_red, handle_missing='value',
                           drop_invariant=False, handle_unknown='value')    
        model = make_pipeline(ce, RandomForestClassifier(n_estimators=5))
        model.fit(X_train_reduced, y_train)
        score = model.score(X_test_reduced, y_test)
        fi = grouped_feature_importance(model, X_train_reduced, cat_col_red)
        r = dict(score=score, features=kept.copy(), nb=len(kept), model=model,
                 removed=last_removed, next_remove=fi.iloc[0,0], score_remove=fi.iloc[0,1])
        print(r['nb'], r['score'], last_removed, list(fi.iloc[0,:]), X_train_reduced.shape)
        last_removed = fi.iloc[0,0]
        kept = [_ for _ in kept if _ != last_removed]
        res.append(r)


.. parsed-literal::
    13 0.8345310484613967 None ['race', 0.017876461431527595] (32561, 13)
    12 0.8328112523800749 race ['native_country', 0.030328521242080998] (32561, 12)
    11 0.8377863767581843 native_country ['sex', 0.020085260379222935] (32561, 11)
    10 0.8366193722744303 sex ['capital_loss', 0.04176572757064676] (32561, 10)
    9 0.8286960260426264 capital_loss ['workclass', 0.051745962436835796] (32561, 9)
    8 0.8289417111971009 workclass ['education', 0.06708986825516294] (32561, 8)
    7 0.8292488176401941 education ['marital_status', 0.10470565098684484] (32561, 7)
    6 0.8299858731036177 marital_status ['education_num', 0.12482378792123464] (32561, 6)
    5 0.8271604938271605 education_num ['capital_gain', 0.15606710618140648] (32561, 5)
    4 0.8043731957496468 capital_gain ['hours_per_week', 0.1952915897722717] (32561, 4)
    3 0.8129721761562557 hours_per_week ['age', 0.2703491413657654] (32561, 3)
    2 0.8207726798108225 age ['occupation', 0.3492816128792312] (32561, 2)
    1 0.7637737239727289 occupation ['relationship', 0.9999999999999999] (32561, 1)


Il faut se rappeler qu’un classifieur constant retournerait environ 76%
de bonne classification.

.. code:: ipython3

    1 - y_test.sum() / len(y_test)




.. parsed-literal::
    0.7637737239727289