.. _td2asentimentanalysisrst:

=============================
2A.ml - Analyse de sentiments
=============================


.. only:: html

    **Links:** :download:`notebook <td2a_sentiment_analysis.ipynb>`, :downloadlink:`html <td2a_sentiment_analysis2html.html>`, :download:`python <td2a_sentiment_analysis.py>`, :downloadlink:`slides <td2a_sentiment_analysis.slides.html>`, :githublink:`GitHub|_doc/notebooks/td2a_ml/td2a_sentiment_analysis.ipynb|*`


C’est désormais un problème classique de machine learning. D’un côté, du
texte, de l’autre une appréciation, le plus souvent binaire, positive ou
négative mais qui pourrait être graduelle.

.. code:: ipython3

    %matplotlib inline

.. code:: ipython3

    from jyquickhelper import add_notebook_menu
    add_notebook_menu()






.. contents::
    :local:





Les données
-----------

On récupère les données depuis le site UCI `Sentiment Labelled Sentences
Data
Set <https://archive.ics.uci.edu/ml/datasets/Sentiment+Labelled+Sentences>`__
où on utilise la fonction ``load_sentiment_dataset``.

.. code:: ipython3

    from ensae_teaching_cs.data import load_sentiment_dataset
    df = load_sentiment_dataset()
    df.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentance</th>
          <th>sentiment</th>
          <th>source</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>So there is no way for me to plug it in here i...</td>
          <td>0</td>
          <td>amazon_cells_labelled</td>
        </tr>
        <tr>
          <th>1</th>
          <td>Good case, Excellent value.</td>
          <td>1</td>
          <td>amazon_cells_labelled</td>
        </tr>
        <tr>
          <th>2</th>
          <td>Great for the jawbone.</td>
          <td>1</td>
          <td>amazon_cells_labelled</td>
        </tr>
        <tr>
          <th>3</th>
          <td>Tied to charger for conversations lasting more...</td>
          <td>0</td>
          <td>amazon_cells_labelled</td>
        </tr>
        <tr>
          <th>4</th>
          <td>The mic is great.</td>
          <td>1</td>
          <td>amazon_cells_labelled</td>
        </tr>
      </tbody>
    </table>
    </div>



Exercice 1 : approche td-idf
----------------------------

La cible est la colonne *sentiment*, les deux autres colonnes sont les
features. Il faudra utiliser les prétraitements
`LabelEncoder <http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html>`__,
`OneHotEncoder <http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html>`__,
`TF-IDF <http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfTransformer.html>`__.
L’un d’entre eux n’est pas nécessaire depuis la version
`0.20.0 <http://scikit-learn.org/stable/whats_new.html#sklearn-preprocessing>`__
de *scikit-learn*.


Exercice 2 : word2vec
---------------------

On utilise l’approche
`word2vec <https://en.wikipedia.org/wiki/Word2vec>`__ du module
`gensim <https://radimrehurek.com/gensim/models/word2vec.html>`__ ou
`spacy <https://spacy.io/usage/vectors-similarity>`__.


Exercice 3 : comparer les deux approches
----------------------------------------

Avec une courbe
`ROC <http://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html>`__
par exemple.