{"cells": [{"cell_type": "markdown", "metadata": {}, "source": ["# 2A.ml - Classification binaire avec features textuelles\n", "\n", "Ce notebook propose de voir comment incorporer des features pour voir l'am\u00e9lioration des performances sur une classification binaire. "]}, {"cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [{"data": {"text/html": ["<div id=\"my_id_menu_nb\">run previous cell, wait for 2 seconds</div>\n", "<script>\n", "function repeat_indent_string(n){\n", "    var a = \"\" ;\n", "    for ( ; n > 0 ; --n)\n", "        a += \"    \";\n", "    return a;\n", "}\n", "var update_menu_string = function(begin, lfirst, llast, sformat, send, keep_item, begin_format, end_format) {\n", "    var anchors = document.getElementsByClassName(\"section\");\n", "    if (anchors.length == 0) {\n", "        anchors = document.getElementsByClassName(\"text_cell_render rendered_html\");\n", "    }\n", "    var i,t;\n", "    var text_menu = begin;\n", "    var text_memo = \"<pre>\\nlength:\" + anchors.length + \"\\n\";\n", "    var ind = \"\";\n", "    var memo_level = 1;\n", "    var href;\n", "    var tags = [];\n", "    var main_item = 0;\n", "    var format_open = 0;\n", "    for (i = 0; i <= llast; i++)\n", "        tags.push(\"h\" + i);\n", "\n", "    for (i = 0; i < anchors.length; i++) {\n", "        text_memo += \"**\" + anchors[i].id + \"--\\n\";\n", "\n", "        var child = null;\n", "        for(t = 0; t < tags.length; t++) {\n", "            var r = anchors[i].getElementsByTagName(tags[t]);\n", "            if (r.length > 0) {\n", "child = r[0];\n", "break;\n", "            }\n", "        }\n", "        if (child == null) {\n", "            text_memo += \"null\\n\";\n", "            continue;\n", "        }\n", "        if (anchors[i].hasAttribute(\"id\")) {\n", "            // when converted in RST\n", "            href = anchors[i].id;\n", "            text_memo += \"#1-\" + href;\n", "            // passer \u00e0 child suivant (le chercher)\n", "        }\n", "        else if (child.hasAttribute(\"id\")) {\n", "            // in a notebook\n", "            href = child.id;\n", "            text_memo += \"#2-\" + href;\n", "        }\n", "        else {\n", "            text_memo += \"#3-\" + \"*\" + \"\\n\";\n", "            continue;\n", "        }\n", "        var title = child.textContent;\n", "        var level = parseInt(child.tagName.substring(1,2));\n", "\n", "        text_memo += \"--\" + level + \"?\" + lfirst + \"--\" + title + \"\\n\";\n", "\n", "        if ((level < lfirst) || (level > llast)) {\n", "            continue ;\n", "        }\n", "        if (title.endsWith('\u00b6')) {\n", "            title = title.substring(0,title.length-1).replace(\"<\", \"&lt;\")\n", "         .replace(\">\", \"&gt;\").replace(\"&\", \"&amp;\");\n", "        }\n", "        if (title.length == 0) {\n", "            continue;\n", "        }\n", "\n", "        while (level < memo_level) {\n", "            text_menu += end_format + \"</ul>\\n\";\n", "            format_open -= 1;\n", "            memo_level -= 1;\n", "        }\n", "        if (level == lfirst) {\n", "            main_item += 1;\n", "        }\n", "        if (keep_item != -1 && main_item != keep_item + 1) {\n", "            // alert(main_item + \" - \" + level + \" - \" + keep_item);\n", "            continue;\n", "        }\n", "        while (level > memo_level) {\n", "            text_menu += \"<ul>\\n\";\n", "            memo_level += 1;\n", "        }\n", "        text_menu += repeat_indent_string(level-2);\n", "        text_menu += begin_format + sformat.replace(\"__HREF__\", href).replace(\"__TITLE__\", title);\n", "        format_open += 1;\n", "    }\n", "    while (1 < memo_level) {\n", "        text_menu += end_format + \"</ul>\\n\";\n", "        memo_level -= 1;\n", "        format_open -= 1;\n", "    }\n", "    text_menu += send;\n", "    //text_menu += \"\\n\" + text_memo;\n", "\n", "    while (format_open > 0) {\n", "        text_menu += end_format;\n", "        format_open -= 1;\n", "    }\n", "    return text_menu;\n", "};\n", "var update_menu = function() {\n", "    var sbegin = \"\";\n", "    var sformat = '<a href=\"#__HREF__\">__TITLE__</a>';\n", "    var send = \"\";\n", "    var begin_format = '<li>';\n", "    var end_format = '</li>';\n", "    var keep_item = -1;\n", "    var text_menu = update_menu_string(sbegin, 2, 4, sformat, send, keep_item,\n", "       begin_format, end_format);\n", "    var menu = document.getElementById(\"my_id_menu_nb\");\n", "    menu.innerHTML=text_menu;\n", "};\n", "window.setTimeout(update_menu,2000);\n", "            </script>"], "text/plain": ["<IPython.core.display.HTML object>"]}, "execution_count": 2, "metadata": {}, "output_type": "execute_result"}], "source": ["from jyquickhelper import add_notebook_menu\n", "add_notebook_menu()"]}, {"cell_type": "markdown", "metadata": {"collapsed": true}, "source": ["## R\u00e9cup\u00e9rer les donn\u00e9es\n", "\n", "Les donn\u00e9es sont t\u00e9l\u00e9chargeables [Comp\u00e9tition 2017 - additifs alimentaires](http://www.xavierdupre.fr/app/ensae_teaching_cs/helpsphinx/questions/competition_2A.html#id1) ou encore avec le code :"]}, {"cell_type": "code", "execution_count": 2, "metadata": {"collapsed": true}, "outputs": [], "source": ["from pyensae.datasource import download_data\n", "data_train = download_data(\"off_train_all.zip\", \n", "                           url=\"https://raw.githubusercontent.com/sdpython/data/master/OpenFoodFacts/\")"]}, {"cell_type": "code", "execution_count": 3, "metadata": {"collapsed": true}, "outputs": [], "source": ["data_test = download_data(\"off_test_all.zip\", \n", "                           url=\"https://raw.githubusercontent.com/sdpython/data/master/OpenFoodFacts/\")"]}, {"cell_type": "code", "execution_count": 4, "metadata": {"collapsed": true}, "outputs": [], "source": ["import pandas\n", "df = pandas.read_csv(\"off_test_all.txt\", sep=\"\\t\", encoding=\"utf8\", low_memory=False)"]}, {"cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [{"data": {"text/html": ["<div>\n", "<style>\n", "    .dataframe thead tr:only-child th {\n", "        text-align: right;\n", "    }\n", "\n", "    .dataframe thead th {\n", "        text-align: left;\n", "    }\n", "\n", "    .dataframe tbody tr th {\n", "        vertical-align: top;\n", "    }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", "  <thead>\n", "    <tr style=\"text-align: right;\">\n", "      <th></th>\n", "      <th>code</th>\n", "      <th>url</th>\n", "      <th>creator</th>\n", "      <th>created_t</th>\n", "      <th>created_datetime</th>\n", "      <th>last_modified_t</th>\n", "      <th>last_modified_datetime</th>\n", "      <th>product_name</th>\n", "      <th>generic_name</th>\n", "      <th>quantity</th>\n", "      <th>...</th>\n", "      <th>collagen-meat-protein-ratio_100g</th>\n", "      <th>cocoa_100g</th>\n", "      <th>chlorophyl_100g</th>\n", "      <th>carbon-footprint_100g</th>\n", "      <th>nutrition-score-fr_100g</th>\n", "      <th>nutrition-score-uk_100g</th>\n", "      <th>glycemic-index_100g</th>\n", "      <th>water-hardness_100g</th>\n", "      <th>hasE</th>\n", "      <th>s100</th>\n", "    </tr>\n", "  </thead>\n", "  <tbody>\n", "    <tr>\n", "      <th>0</th>\n", "      <td>1.008255e+10</td>\n", "      <td>http://world-fr.openfoodfacts.org/produit/0010...</td>\n", "      <td>usda-ndb-import</td>\n", "      <td>1489064583</td>\n", "      <td>2017-03-09T13:03:03Z</td>\n", "      <td>1489064583</td>\n", "      <td>2017-03-09T13:03:03Z</td>\n", "      <td>Golden Island, Pork Jerky, Grilled Barbecue</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>...</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>23.0</td>\n", "      <td>23.0</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>False</td>\n", "      <td>17.0</td>\n", "    </tr>\n", "    <tr>\n", "      <th>1</th>\n", "      <td>1.182204e+10</td>\n", "      <td>http://world-fr.openfoodfacts.org/produit/0011...</td>\n", "      <td>usda-ndb-import</td>\n", "      <td>1489070197</td>\n", "      <td>2017-03-09T14:36:37Z</td>\n", "      <td>1489070197</td>\n", "      <td>2017-03-09T14:36:37Z</td>\n", "      <td>Big Fizz, Soda, Orange</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>...</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>True</td>\n", "      <td>7.0</td>\n", "    </tr>\n", "    <tr>\n", "      <th>2</th>\n", "      <td>2.548401e+10</td>\n", "      <td>http://world-fr.openfoodfacts.org/produit/0025...</td>\n", "      <td>usda-ndb-import</td>\n", "      <td>1489052024</td>\n", "      <td>2017-03-09T09:33:44Z</td>\n", "      <td>1489052024</td>\n", "      <td>2017-03-09T09:33:44Z</td>\n", "      <td>Tofubaked Marinated Baked Tofu, Sesame Ginger</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>...</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>-2.0</td>\n", "      <td>-2.0</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>True</td>\n", "      <td>17.0</td>\n", "    </tr>\n", "    <tr>\n", "      <th>3</th>\n", "      <td>1.229250e+10</td>\n", "      <td>http://world-fr.openfoodfacts.org/produit/0012...</td>\n", "      <td>usda-ndb-import</td>\n", "      <td>1489133493</td>\n", "      <td>2017-03-10T08:11:33Z</td>\n", "      <td>1489133493</td>\n", "      <td>2017-03-10T08:11:33Z</td>\n", "      <td>Milk Chocolate Eggs</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>...</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>23.0</td>\n", "      <td>23.0</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>True</td>\n", "      <td>17.0</td>\n", "    </tr>\n", "    <tr>\n", "      <th>4</th>\n", "      <td>1.115054e+10</td>\n", "      <td>http://world-fr.openfoodfacts.org/produit/0011...</td>\n", "      <td>usda-ndb-import</td>\n", "      <td>1489052892</td>\n", "      <td>2017-03-09T09:48:12Z</td>\n", "      <td>1489052892</td>\n", "      <td>2017-03-09T09:48:12Z</td>\n", "      <td>Fresh Polish Sausage</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>...</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>22.0</td>\n", "      <td>22.0</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>True</td>\n", "      <td>17.0</td>\n", "    </tr>\n", "  </tbody>\n", "</table>\n", "<p>5 rows \u00d7 165 columns</p>\n", "</div>"], "text/plain": ["           code                                                url  \\\n", "0  1.008255e+10  http://world-fr.openfoodfacts.org/produit/0010...   \n", "1  1.182204e+10  http://world-fr.openfoodfacts.org/produit/0011...   \n", "2  2.548401e+10  http://world-fr.openfoodfacts.org/produit/0025...   \n", "3  1.229250e+10  http://world-fr.openfoodfacts.org/produit/0012...   \n", "4  1.115054e+10  http://world-fr.openfoodfacts.org/produit/0011...   \n", "\n", "           creator   created_t      created_datetime last_modified_t  \\\n", "0  usda-ndb-import  1489064583  2017-03-09T13:03:03Z      1489064583   \n", "1  usda-ndb-import  1489070197  2017-03-09T14:36:37Z      1489070197   \n", "2  usda-ndb-import  1489052024  2017-03-09T09:33:44Z      1489052024   \n", "3  usda-ndb-import  1489133493  2017-03-10T08:11:33Z      1489133493   \n", "4  usda-ndb-import  1489052892  2017-03-09T09:48:12Z      1489052892   \n", "\n", "  last_modified_datetime                                   product_name  \\\n", "0   2017-03-09T13:03:03Z    Golden Island, Pork Jerky, Grilled Barbecue   \n", "1   2017-03-09T14:36:37Z                         Big Fizz, Soda, Orange   \n", "2   2017-03-09T09:33:44Z  Tofubaked Marinated Baked Tofu, Sesame Ginger   \n", "3   2017-03-10T08:11:33Z                            Milk Chocolate Eggs   \n", "4   2017-03-09T09:48:12Z                           Fresh Polish Sausage   \n", "\n", "  generic_name quantity  ...  collagen-meat-protein-ratio_100g cocoa_100g  \\\n", "0          NaN      NaN  ...                               NaN        NaN   \n", "1          NaN      NaN  ...                               NaN        NaN   \n", "2          NaN      NaN  ...                               NaN        NaN   \n", "3          NaN      NaN  ...                               NaN        NaN   \n", "4          NaN      NaN  ...                               NaN        NaN   \n", "\n", "  chlorophyl_100g carbon-footprint_100g nutrition-score-fr_100g  \\\n", "0             NaN                   NaN                    23.0   \n", "1             NaN                   NaN                     NaN   \n", "2             NaN                   NaN                    -2.0   \n", "3             NaN                   NaN                    23.0   \n", "4             NaN                   NaN                    22.0   \n", "\n", "  nutrition-score-uk_100g glycemic-index_100g water-hardness_100g   hasE  s100  \n", "0                    23.0                 NaN                 NaN  False  17.0  \n", "1                     NaN                 NaN                 NaN   True   7.0  \n", "2                    -2.0                 NaN                 NaN   True  17.0  \n", "3                    23.0                 NaN                 NaN   True  17.0  \n", "4                    22.0                 NaN                 NaN   True  17.0  \n", "\n", "[5 rows x 165 columns]"]}, "execution_count": 6, "metadata": {}, "output_type": "execute_result"}], "source": ["df.head()"]}, {"cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [{"data": {"text/html": ["<div>\n", "<style>\n", "    .dataframe thead tr:only-child th {\n", "        text-align: right;\n", "    }\n", "\n", "    .dataframe thead th {\n", "        text-align: left;\n", "    }\n", "\n", "    .dataframe tbody tr th {\n", "        vertical-align: top;\n", "    }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", "  <thead>\n", "    <tr style=\"text-align: right;\">\n", "      <th></th>\n", "      <th>0</th>\n", "      <th>1</th>\n", "    </tr>\n", "  </thead>\n", "  <tbody>\n", "    <tr>\n", "      <th>code</th>\n", "      <td>1.00826e+10</td>\n", "      <td>1.1822e+10</td>\n", "    </tr>\n", "    <tr>\n", "      <th>url</th>\n", "      <td>http://world-fr.openfoodfacts.org/produit/0010...</td>\n", "      <td>http://world-fr.openfoodfacts.org/produit/0011...</td>\n", "    </tr>\n", "    <tr>\n", "      <th>creator</th>\n", "      <td>usda-ndb-import</td>\n", "      <td>usda-ndb-import</td>\n", "    </tr>\n", "    <tr>\n", "      <th>created_t</th>\n", "      <td>1489064583</td>\n", "      <td>1489070197</td>\n", "    </tr>\n", "    <tr>\n", "      <th>created_datetime</th>\n", "      <td>2017-03-09T13:03:03Z</td>\n", "      <td>2017-03-09T14:36:37Z</td>\n", "    </tr>\n", "    <tr>\n", "      <th>last_modified_t</th>\n", "      <td>1489064583</td>\n", "      <td>1489070197</td>\n", "    </tr>\n", "    <tr>\n", "      <th>last_modified_datetime</th>\n", "      <td>2017-03-09T13:03:03Z</td>\n", "      <td>2017-03-09T14:36:37Z</td>\n", "    </tr>\n", "    <tr>\n", "      <th>product_name</th>\n", "      <td>Golden Island, Pork Jerky, Grilled Barbecue</td>\n", "      <td>Big Fizz, Soda, Orange</td>\n", "    </tr>\n", "    <tr>\n", "      <th>generic_name</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>quantity</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>packaging</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>packaging_tags</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>brands</th>\n", "      <td>Golden Island Jerky Inc.</td>\n", "      <td>Rite Aid Corporation</td>\n", "    </tr>\n", "    <tr>\n", "      <th>brands_tags</th>\n", "      <td>golden-island-jerky-inc</td>\n", "      <td>rite-aid-corporation</td>\n", "    </tr>\n", "    <tr>\n", "      <th>categories</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>categories_tags</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>categories_fr</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>origins</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>origins_tags</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>manufacturing_places</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>manufacturing_places_tags</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>labels</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>labels_tags</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>labels_fr</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>emb_codes</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>emb_codes_tags</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>first_packaging_code_geo</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>cities</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>cities_tags</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>purchase_places</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>stores</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>countries</th>\n", "      <td>US</td>\n", "      <td>US</td>\n", "    </tr>\n", "    <tr>\n", "      <th>countries_tags</th>\n", "      <td>en:united-states</td>\n", "      <td>en:united-states</td>\n", "    </tr>\n", "    <tr>\n", "      <th>countries_fr</th>\n", "      <td>\u00c9tats-Unis</td>\n", "      <td>\u00c9tats-Unis</td>\n", "    </tr>\n", "    <tr>\n", "      <th>ingredients_text</th>\n", "      <td>Pork, sugar, water, brown sugar, gluten free s...</td>\n", "      <td>Carbonated water, high fructose corn syrup, ci...</td>\n", "    </tr>\n", "    <tr>\n", "      <th>allergens</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>allergens_fr</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>traces</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>traces_tags</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>traces_fr</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>serving_size</th>\n", "      <td>28 g (1 oz)</td>\n", "      <td>240 ml (8 fl oz)</td>\n", "    </tr>\n", "    <tr>\n", "      <th>no_nutriments</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>additives_n</th>\n", "      <td>0</td>\n", "      <td>6</td>\n", "    </tr>\n", "    <tr>\n", "      <th>additives</th>\n", "      <td>en:2-or-less;en:brown-sugar;en:contain-rice;en...</td>\n", "      <td>en:and-brominated-vegetable-oil;en:carbonated-...</td>\n", "    </tr>\n", "    <tr>\n", "      <th>additives_tags</th>\n", "      <td>NaN</td>\n", "      <td>en:e110,en:e211,en:e330,en:e414,en:e443,en:e445</td>\n", "    </tr>\n", "    <tr>\n", "      <th>additives_fr</th>\n", "      <td>NaN</td>\n", "      <td>E110 - Jaune orang\u00e9 S,E211 - Benzoate de sodiu...</td>\n", "    </tr>\n", "    <tr>\n", "      <th>ingredients_from_palm_oil_n</th>\n", "      <td>0</td>\n", "      <td>0</td>\n", "    </tr>\n", "    <tr>\n", "      <th>ingredients_from_palm_oil</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>ingredients_from_palm_oil_tags</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>ingredients_that_may_be_from_palm_oil_n</th>\n", "      <td>0</td>\n", "      <td>0</td>\n", "    </tr>\n", "  </tbody>\n", "</table>\n", "</div>"], "text/plain": ["                                                                                         0  \\\n", "code                                                                           1.00826e+10   \n", "url                                      http://world-fr.openfoodfacts.org/produit/0010...   \n", "creator                                                                    usda-ndb-import   \n", "created_t                                                                       1489064583   \n", "created_datetime                                                      2017-03-09T13:03:03Z   \n", "last_modified_t                                                                 1489064583   \n", "last_modified_datetime                                                2017-03-09T13:03:03Z   \n", "product_name                                   Golden Island, Pork Jerky, Grilled Barbecue   \n", "generic_name                                                                           NaN   \n", "quantity                                                                               NaN   \n", "packaging                                                                              NaN   \n", "packaging_tags                                                                         NaN   \n", "brands                                                            Golden Island Jerky Inc.   \n", "brands_tags                                                        golden-island-jerky-inc   \n", "categories                                                                             NaN   \n", "categories_tags                                                                        NaN   \n", "categories_fr                                                                          NaN   \n", "origins                                                                                NaN   \n", "origins_tags                                                                           NaN   \n", "manufacturing_places                                                                   NaN   \n", "manufacturing_places_tags                                                              NaN   \n", "labels                                                                                 NaN   \n", "labels_tags                                                                            NaN   \n", "labels_fr                                                                              NaN   \n", "emb_codes                                                                              NaN   \n", "emb_codes_tags                                                                         NaN   \n", "first_packaging_code_geo                                                               NaN   \n", "cities                                                                                 NaN   \n", "cities_tags                                                                            NaN   \n", "purchase_places                                                                        NaN   \n", "stores                                                                                 NaN   \n", "countries                                                                               US   \n", "countries_tags                                                            en:united-states   \n", "countries_fr                                                                    \u00c9tats-Unis   \n", "ingredients_text                         Pork, sugar, water, brown sugar, gluten free s...   \n", "allergens                                                                              NaN   \n", "allergens_fr                                                                           NaN   \n", "traces                                                                                 NaN   \n", "traces_tags                                                                            NaN   \n", "traces_fr                                                                              NaN   \n", "serving_size                                                                   28 g (1 oz)   \n", "no_nutriments                                                                          NaN   \n", "additives_n                                                                              0   \n", "additives                                en:2-or-less;en:brown-sugar;en:contain-rice;en...   \n", "additives_tags                                                                         NaN   \n", "additives_fr                                                                           NaN   \n", "ingredients_from_palm_oil_n                                                              0   \n", "ingredients_from_palm_oil                                                              NaN   \n", "ingredients_from_palm_oil_tags                                                         NaN   \n", "ingredients_that_may_be_from_palm_oil_n                                                  0   \n", "\n", "                                                                                         1  \n", "code                                                                            1.1822e+10  \n", "url                                      http://world-fr.openfoodfacts.org/produit/0011...  \n", "creator                                                                    usda-ndb-import  \n", "created_t                                                                       1489070197  \n", "created_datetime                                                      2017-03-09T14:36:37Z  \n", "last_modified_t                                                                 1489070197  \n", "last_modified_datetime                                                2017-03-09T14:36:37Z  \n", "product_name                                                        Big Fizz, Soda, Orange  \n", "generic_name                                                                           NaN  \n", "quantity                                                                               NaN  \n", "packaging                                                                              NaN  \n", "packaging_tags                                                                         NaN  \n", "brands                                                                Rite Aid Corporation  \n", "brands_tags                                                           rite-aid-corporation  \n", "categories                                                                             NaN  \n", "categories_tags                                                                        NaN  \n", "categories_fr                                                                          NaN  \n", "origins                                                                                NaN  \n", "origins_tags                                                                           NaN  \n", "manufacturing_places                                                                   NaN  \n", "manufacturing_places_tags                                                              NaN  \n", "labels                                                                                 NaN  \n", "labels_tags                                                                            NaN  \n", "labels_fr                                                                              NaN  \n", "emb_codes                                                                              NaN  \n", "emb_codes_tags                                                                         NaN  \n", "first_packaging_code_geo                                                               NaN  \n", "cities                                                                                 NaN  \n", "cities_tags                                                                            NaN  \n", "purchase_places                                                                        NaN  \n", "stores                                                                                 NaN  \n", "countries                                                                               US  \n", "countries_tags                                                            en:united-states  \n", "countries_fr                                                                    \u00c9tats-Unis  \n", "ingredients_text                         Carbonated water, high fructose corn syrup, ci...  \n", "allergens                                                                              NaN  \n", "allergens_fr                                                                           NaN  \n", "traces                                                                                 NaN  \n", "traces_tags                                                                            NaN  \n", "traces_fr                                                                              NaN  \n", "serving_size                                                              240 ml (8 fl oz)  \n", "no_nutriments                                                                          NaN  \n", "additives_n                                                                              6  \n", "additives                                en:and-brominated-vegetable-oil;en:carbonated-...  \n", "additives_tags                             en:e110,en:e211,en:e330,en:e414,en:e443,en:e445  \n", "additives_fr                             E110 - Jaune orang\u00e9 S,E211 - Benzoate de sodiu...  \n", "ingredients_from_palm_oil_n                                                              0  \n", "ingredients_from_palm_oil                                                              NaN  \n", "ingredients_from_palm_oil_tags                                                         NaN  \n", "ingredients_that_may_be_from_palm_oil_n                                                  0  "]}, "execution_count": 7, "metadata": {}, "output_type": "execute_result"}], "source": ["df.head(n=2).T[:50]"]}, {"cell_type": "code", "execution_count": 7, "metadata": {"scrolled": false}, "outputs": [{"data": {"text/html": ["<div>\n", "<style>\n", "    .dataframe thead tr:only-child th {\n", "        text-align: right;\n", "    }\n", "\n", "    .dataframe thead th {\n", "        text-align: left;\n", "    }\n", "\n", "    .dataframe tbody tr th {\n", "        vertical-align: top;\n", "    }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", "  <thead>\n", "    <tr style=\"text-align: right;\">\n", "      <th></th>\n", "      <th>0</th>\n", "      <th>1</th>\n", "    </tr>\n", "  </thead>\n", "  <tbody>\n", "    <tr>\n", "      <th>ingredients_that_may_be_from_palm_oil</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>ingredients_that_may_be_from_palm_oil_tags</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>nutrition_grade_uk</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>nutrition_grade_fr</th>\n", "      <td>e</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>pnns_groups_1</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>pnns_groups_2</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>states</th>\n", "      <td>en:to-be-completed, en:nutrition-facts-complet...</td>\n", "      <td>en:to-be-completed, en:nutrition-facts-complet...</td>\n", "    </tr>\n", "    <tr>\n", "      <th>states_tags</th>\n", "      <td>en:to-be-completed,en:nutrition-facts-complete...</td>\n", "      <td>en:to-be-completed,en:nutrition-facts-complete...</td>\n", "    </tr>\n", "    <tr>\n", "      <th>states_fr</th>\n", "      <td>A compl\u00e9ter,Informations nutritionnelles compl...</td>\n", "      <td>A compl\u00e9ter,Informations nutritionnelles compl...</td>\n", "    </tr>\n", "    <tr>\n", "      <th>main_category</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>main_category_fr</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>image_url</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>image_small_url</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>energy_100g</th>\n", "      <td>1494</td>\n", "      <td>226</td>\n", "    </tr>\n", "    <tr>\n", "      <th>energy-from-fat_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>fat_100g</th>\n", "      <td>7.14</td>\n", "      <td>0</td>\n", "    </tr>\n", "    <tr>\n", "      <th>saturated-fat_100g</th>\n", "      <td>1.79</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>butyric-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>caproic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>caprylic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>capric-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>lauric-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>myristic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>palmitic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>stearic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>arachidic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>behenic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>lignoceric-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>cerotic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>montanic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>melissic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>monounsaturated-fat_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>polyunsaturated-fat_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>omega-3-fat_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>alpha-linolenic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>eicosapentaenoic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>docosahexaenoic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>omega-6-fat_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>linoleic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>arachidonic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>gamma-linolenic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>dihomo-gamma-linolenic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>omega-9-fat_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>oleic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>elaidic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>gondoic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>mead-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>erucic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>nervonic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>trans-fat_100g</th>\n", "      <td>0</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "  </tbody>\n", "</table>\n", "</div>"], "text/plain": ["                                                                                            0  \\\n", "ingredients_that_may_be_from_palm_oil                                                     NaN   \n", "ingredients_that_may_be_from_palm_oil_tags                                                NaN   \n", "nutrition_grade_uk                                                                        NaN   \n", "nutrition_grade_fr                                                                          e   \n", "pnns_groups_1                                                                             NaN   \n", "pnns_groups_2                                                                             NaN   \n", "states                                      en:to-be-completed, en:nutrition-facts-complet...   \n", "states_tags                                 en:to-be-completed,en:nutrition-facts-complete...   \n", "states_fr                                   A compl\u00e9ter,Informations nutritionnelles compl...   \n", "main_category                                                                             NaN   \n", "main_category_fr                                                                          NaN   \n", "image_url                                                                                 NaN   \n", "image_small_url                                                                           NaN   \n", "energy_100g                                                                              1494   \n", "energy-from-fat_100g                                                                      NaN   \n", "fat_100g                                                                                 7.14   \n", "saturated-fat_100g                                                                       1.79   \n", "butyric-acid_100g                                                                         NaN   \n", "caproic-acid_100g                                                                         NaN   \n", "caprylic-acid_100g                                                                        NaN   \n", "capric-acid_100g                                                                          NaN   \n", "lauric-acid_100g                                                                          NaN   \n", "myristic-acid_100g                                                                        NaN   \n", "palmitic-acid_100g                                                                        NaN   \n", "stearic-acid_100g                                                                         NaN   \n", "arachidic-acid_100g                                                                       NaN   \n", "behenic-acid_100g                                                                         NaN   \n", "lignoceric-acid_100g                                                                      NaN   \n", "cerotic-acid_100g                                                                         NaN   \n", "montanic-acid_100g                                                                        NaN   \n", "melissic-acid_100g                                                                        NaN   \n", "monounsaturated-fat_100g                                                                  NaN   \n", "polyunsaturated-fat_100g                                                                  NaN   \n", "omega-3-fat_100g                                                                          NaN   \n", "alpha-linolenic-acid_100g                                                                 NaN   \n", "eicosapentaenoic-acid_100g                                                                NaN   \n", "docosahexaenoic-acid_100g                                                                 NaN   \n", "omega-6-fat_100g                                                                          NaN   \n", "linoleic-acid_100g                                                                        NaN   \n", "arachidonic-acid_100g                                                                     NaN   \n", "gamma-linolenic-acid_100g                                                                 NaN   \n", "dihomo-gamma-linolenic-acid_100g                                                          NaN   \n", "omega-9-fat_100g                                                                          NaN   \n", "oleic-acid_100g                                                                           NaN   \n", "elaidic-acid_100g                                                                         NaN   \n", "gondoic-acid_100g                                                                         NaN   \n", "mead-acid_100g                                                                            NaN   \n", "erucic-acid_100g                                                                          NaN   \n", "nervonic-acid_100g                                                                        NaN   \n", "trans-fat_100g                                                                              0   \n", "\n", "                                                                                            1  \n", "ingredients_that_may_be_from_palm_oil                                                     NaN  \n", "ingredients_that_may_be_from_palm_oil_tags                                                NaN  \n", "nutrition_grade_uk                                                                        NaN  \n", "nutrition_grade_fr                                                                        NaN  \n", "pnns_groups_1                                                                             NaN  \n", "pnns_groups_2                                                                             NaN  \n", "states                                      en:to-be-completed, en:nutrition-facts-complet...  \n", "states_tags                                 en:to-be-completed,en:nutrition-facts-complete...  \n", "states_fr                                   A compl\u00e9ter,Informations nutritionnelles compl...  \n", "main_category                                                                             NaN  \n", "main_category_fr                                                                          NaN  \n", "image_url                                                                                 NaN  \n", "image_small_url                                                                           NaN  \n", "energy_100g                                                                               226  \n", "energy-from-fat_100g                                                                      NaN  \n", "fat_100g                                                                                    0  \n", "saturated-fat_100g                                                                        NaN  \n", "butyric-acid_100g                                                                         NaN  \n", "caproic-acid_100g                                                                         NaN  \n", "caprylic-acid_100g                                                                        NaN  \n", "capric-acid_100g                                                                          NaN  \n", "lauric-acid_100g                                                                          NaN  \n", "myristic-acid_100g                                                                        NaN  \n", "palmitic-acid_100g                                                                        NaN  \n", "stearic-acid_100g                                                                         NaN  \n", "arachidic-acid_100g                                                                       NaN  \n", "behenic-acid_100g                                                                         NaN  \n", "lignoceric-acid_100g                                                                      NaN  \n", "cerotic-acid_100g                                                                         NaN  \n", "montanic-acid_100g                                                                        NaN  \n", "melissic-acid_100g                                                                        NaN  \n", "monounsaturated-fat_100g                                                                  NaN  \n", "polyunsaturated-fat_100g                                                                  NaN  \n", "omega-3-fat_100g                                                                          NaN  \n", "alpha-linolenic-acid_100g                                                                 NaN  \n", "eicosapentaenoic-acid_100g                                                                NaN  \n", "docosahexaenoic-acid_100g                                                                 NaN  \n", "omega-6-fat_100g                                                                          NaN  \n", "linoleic-acid_100g                                                                        NaN  \n", "arachidonic-acid_100g                                                                     NaN  \n", "gamma-linolenic-acid_100g                                                                 NaN  \n", "dihomo-gamma-linolenic-acid_100g                                                          NaN  \n", "omega-9-fat_100g                                                                          NaN  \n", "oleic-acid_100g                                                                           NaN  \n", "elaidic-acid_100g                                                                         NaN  \n", "gondoic-acid_100g                                                                         NaN  \n", "mead-acid_100g                                                                            NaN  \n", "erucic-acid_100g                                                                          NaN  \n", "nervonic-acid_100g                                                                        NaN  \n", "trans-fat_100g                                                                            NaN  "]}, "execution_count": 8, "metadata": {}, "output_type": "execute_result"}], "source": ["df.head(n=2).T[50:100]"]}, {"cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [{"data": {"text/html": ["<div>\n", "<style>\n", "    .dataframe thead tr:only-child th {\n", "        text-align: right;\n", "    }\n", "\n", "    .dataframe thead th {\n", "        text-align: left;\n", "    }\n", "\n", "    .dataframe tbody tr th {\n", "        vertical-align: top;\n", "    }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", "  <thead>\n", "    <tr style=\"text-align: right;\">\n", "      <th></th>\n", "      <th>0</th>\n", "      <th>1</th>\n", "    </tr>\n", "  </thead>\n", "  <tbody>\n", "    <tr>\n", "      <th>cholesterol_100g</th>\n", "      <td>0.089</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>carbohydrates_100g</th>\n", "      <td>39.29</td>\n", "      <td>13.75</td>\n", "    </tr>\n", "    <tr>\n", "      <th>sugars_100g</th>\n", "      <td>39.29</td>\n", "      <td>13.75</td>\n", "    </tr>\n", "    <tr>\n", "      <th>sucrose_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>glucose_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>fructose_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>lactose_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>maltose_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>maltodextrins_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>starch_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>polyols_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>fiber_100g</th>\n", "      <td>0</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>proteins_100g</th>\n", "      <td>28.57</td>\n", "      <td>0</td>\n", "    </tr>\n", "    <tr>\n", "      <th>casein_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>serum-proteins_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>nucleotides_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>salt_100g</th>\n", "      <td>2.81178</td>\n", "      <td>0.04826</td>\n", "    </tr>\n", "    <tr>\n", "      <th>sodium_100g</th>\n", "      <td>1.107</td>\n", "      <td>0.019</td>\n", "    </tr>\n", "    <tr>\n", "      <th>alcohol_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-a_100g</th>\n", "      <td>0</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>beta-carotene_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-d_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-e_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-k_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-c_100g</th>\n", "      <td>0</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-b1_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-b2_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-pp_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-b6_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-b9_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>folates_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>vitamin-b12_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>biotin_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>pantothenic-acid_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>silica_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>bicarbonate_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>potassium_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>chloride_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>calcium_100g</th>\n", "      <td>0</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>phosphorus_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>iron_100g</th>\n", "      <td>0.00257</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>magnesium_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>zinc_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>copper_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>manganese_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>fluoride_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>selenium_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>chromium_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>molybdenum_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>iodine_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "  </tbody>\n", "</table>\n", "</div>"], "text/plain": ["                             0        1\n", "cholesterol_100g         0.089      NaN\n", "carbohydrates_100g       39.29    13.75\n", "sugars_100g              39.29    13.75\n", "sucrose_100g               NaN      NaN\n", "glucose_100g               NaN      NaN\n", "fructose_100g              NaN      NaN\n", "lactose_100g               NaN      NaN\n", "maltose_100g               NaN      NaN\n", "maltodextrins_100g         NaN      NaN\n", "starch_100g                NaN      NaN\n", "polyols_100g               NaN      NaN\n", "fiber_100g                   0      NaN\n", "proteins_100g            28.57        0\n", "casein_100g                NaN      NaN\n", "serum-proteins_100g        NaN      NaN\n", "nucleotides_100g           NaN      NaN\n", "salt_100g              2.81178  0.04826\n", "sodium_100g              1.107    0.019\n", "alcohol_100g               NaN      NaN\n", "vitamin-a_100g               0      NaN\n", "beta-carotene_100g         NaN      NaN\n", "vitamin-d_100g             NaN      NaN\n", "vitamin-e_100g             NaN      NaN\n", "vitamin-k_100g             NaN      NaN\n", "vitamin-c_100g               0      NaN\n", "vitamin-b1_100g            NaN      NaN\n", "vitamin-b2_100g            NaN      NaN\n", "vitamin-pp_100g            NaN      NaN\n", "vitamin-b6_100g            NaN      NaN\n", "vitamin-b9_100g            NaN      NaN\n", "folates_100g               NaN      NaN\n", "vitamin-b12_100g           NaN      NaN\n", "biotin_100g                NaN      NaN\n", "pantothenic-acid_100g      NaN      NaN\n", "silica_100g                NaN      NaN\n", "bicarbonate_100g           NaN      NaN\n", "potassium_100g             NaN      NaN\n", "chloride_100g              NaN      NaN\n", "calcium_100g                 0      NaN\n", "phosphorus_100g            NaN      NaN\n", "iron_100g              0.00257      NaN\n", "magnesium_100g             NaN      NaN\n", "zinc_100g                  NaN      NaN\n", "copper_100g                NaN      NaN\n", "manganese_100g             NaN      NaN\n", "fluoride_100g              NaN      NaN\n", "selenium_100g              NaN      NaN\n", "chromium_100g              NaN      NaN\n", "molybdenum_100g            NaN      NaN\n", "iodine_100g                NaN      NaN"]}, "execution_count": 9, "metadata": {}, "output_type": "execute_result"}], "source": ["df.head(n=2).T[100:150]"]}, {"cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [{"data": {"text/html": ["<div>\n", "<style>\n", "    .dataframe thead tr:only-child th {\n", "        text-align: right;\n", "    }\n", "\n", "    .dataframe thead th {\n", "        text-align: left;\n", "    }\n", "\n", "    .dataframe tbody tr th {\n", "        vertical-align: top;\n", "    }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", "  <thead>\n", "    <tr style=\"text-align: right;\">\n", "      <th></th>\n", "      <th>0</th>\n", "      <th>1</th>\n", "    </tr>\n", "  </thead>\n", "  <tbody>\n", "    <tr>\n", "      <th>caffeine_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>taurine_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>ph_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>fruits-vegetables-nuts_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>fruits-vegetables-nuts-estimate_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>collagen-meat-protein-ratio_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>cocoa_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>chlorophyl_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>carbon-footprint_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>nutrition-score-fr_100g</th>\n", "      <td>23</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>nutrition-score-uk_100g</th>\n", "      <td>23</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>glycemic-index_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>water-hardness_100g</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>hasE</th>\n", "      <td>False</td>\n", "      <td>True</td>\n", "    </tr>\n", "    <tr>\n", "      <th>s100</th>\n", "      <td>17</td>\n", "      <td>7</td>\n", "    </tr>\n", "  </tbody>\n", "</table>\n", "</div>"], "text/plain": ["                                          0     1\n", "caffeine_100g                           NaN   NaN\n", "taurine_100g                            NaN   NaN\n", "ph_100g                                 NaN   NaN\n", "fruits-vegetables-nuts_100g             NaN   NaN\n", "fruits-vegetables-nuts-estimate_100g    NaN   NaN\n", "collagen-meat-protein-ratio_100g        NaN   NaN\n", "cocoa_100g                              NaN   NaN\n", "chlorophyl_100g                         NaN   NaN\n", "carbon-footprint_100g                   NaN   NaN\n", "nutrition-score-fr_100g                  23   NaN\n", "nutrition-score-uk_100g                  23   NaN\n", "glycemic-index_100g                     NaN   NaN\n", "water-hardness_100g                     NaN   NaN\n", "hasE                                  False  True\n", "s100                                     17     7"]}, "execution_count": 10, "metadata": {}, "output_type": "execute_result"}], "source": ["df.head(n=2).T[150:]"]}, {"cell_type": "markdown", "metadata": {}, "source": ["## Exercice 1 : extraire un \u00e9chantillon al\u00e9atoire\n", "\n", "Les donn\u00e9es sont volumineuses. Prenons un \u00e9chantillon pour aller plus vite."]}, {"cell_type": "code", "execution_count": 10, "metadata": {"collapsed": true}, "outputs": [], "source": []}, {"cell_type": "markdown", "metadata": {}, "source": ["## Exercice 2 : caler un premier mod\u00e8le de classification binaire\n", "\n", "Avec les variables num\u00e9riques uniquement. La cible est la variable ``hasE``."]}, {"cell_type": "code", "execution_count": 11, "metadata": {"collapsed": true}, "outputs": [], "source": []}, {"cell_type": "markdown", "metadata": {}, "source": ["## Exercice 3 : tracer une courbe ROC"]}, {"cell_type": "code", "execution_count": 12, "metadata": {"collapsed": true}, "outputs": [], "source": []}, {"cell_type": "markdown", "metadata": {}, "source": ["## Exercice 4 : utiliser la variable qui contient le pays\n", "\n", "Est-ce que le mod\u00e8le est plus performant ?"]}, {"cell_type": "code", "execution_count": 13, "metadata": {"collapsed": true}, "outputs": [], "source": []}, {"cell_type": "markdown", "metadata": {}, "source": ["## Exercice 5 : utiliser la variable additives_tags ?\n", "\n", "Les r\u00e9sultats ne vous paraissent-ils pas \u00e9tranges ?"]}, {"cell_type": "code", "execution_count": 14, "metadata": {"collapsed": true}, "outputs": [], "source": []}, {"cell_type": "markdown", "metadata": {}, "source": ["## Exercice 6 : utiliser la variable ingredients_text ?\n", "\n", "Que sugg\u00e9rez vous *embedding* ?"]}, {"cell_type": "code", "execution_count": 15, "metadata": {"collapsed": true}, "outputs": [], "source": []}, {"cell_type": "markdown", "metadata": {}, "source": ["## Exercice 7 : utiliser la variable product_name ?"]}, {"cell_type": "code", "execution_count": 16, "metadata": {"collapsed": true}, "outputs": [], "source": []}, {"cell_type": "markdown", "metadata": {}, "source": ["## Exercice 8 : utiliser GridSearch pour optimiser un hyper-param\u00e8tre du mod\u00e8le ?"]}, {"cell_type": "code", "execution_count": 17, "metadata": {"collapsed": true}, "outputs": [], "source": []}, {"cell_type": "markdown", "metadata": {}, "source": ["## Exercice 9 : faites tourner une cross validation ?"]}, {"cell_type": "code", "execution_count": 18, "metadata": {"collapsed": true}, "outputs": [], "source": []}], "metadata": {"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5"}}, "nbformat": 4, "nbformat_minor": 2}