.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "gyexamples/plot_gexternal_lightgbm_reg.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_gyexamples_plot_gexternal_lightgbm_reg.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_gyexamples_plot_gexternal_lightgbm_reg.py:


.. _example-lightgbm-reg:

Convert a pipeline with a LightGBM regressor
============================================

.. index:: LightGBM

The discrepancies observed when using float and TreeEnsemble operator
(see :ref:`l-example-discrepencies-float-double`)
explains why the converter for *LGBMRegressor* may introduce significant
discrepancies even when it is used with float tensors.

Library *lightgbm* is implemented with double. A random forest regressor
with multiple trees computes its prediction by adding the prediction of
every tree. After being converting into ONNX, this summation becomes
:math:`\left[\sum\right]_{i=1}^F float(T_i(x))`,
where *F* is the number of trees in the forest,
:math:`T_i(x)` the output of tree *i* and :math:`\left[\sum\right]`
a float addition. The discrepancy can be expressed as
:math:`D(x) = |\left[\sum\right]_{i=1}^F float(T_i(x)) -
\sum_{i=1}^F T_i(x)|`.
This grows with the number of trees in the forest.

To reduce the impact, an option was added to split the node
*TreeEnsembleRegressor* into multiple ones and to do a summation
with double this time. If we assume the node if split into *a* nodes,
the discrepancies then become
:math:`D'(x) = |\sum_{k=1}^a \left[\sum\right]_{i=1}^{F/a}
float(T_{ak + i}(x)) - \sum_{i=1}^F T_i(x)|`.

In 2022, :epkg:`onnx` and :epkg:`onnxruntime` updated the specifications
of TreeEnsemble operators and they can now support double thresholds
(see `TreeEnsembleRegressor v3
<https://onnx.ai/onnx/operators/onnx_aionnxml_TreeEnsembleRegressor.html>`_).
That would be the recommended option to reduce the discrepancies.

.. contents::
    :local:

Train a LGBMRegressor
+++++++++++++++++++++

.. GENERATED FROM PYTHON SOURCE LINES 44-69

.. code-block:: default

    import warnings
    import time
    import timeit
    from packaging.version import Version
    import numpy
    from pandas import DataFrame
    import matplotlib.pyplot as plt
    from tqdm import tqdm
    from lightgbm import LGBMRegressor
    from onnxruntime import InferenceSession
    from skl2onnx import update_registered_converter
    from skl2onnx.common.shape_calculator import calculate_linear_regressor_output_shapes  # noqa
    from mlprodict.onnx_conv import to_onnx
    from onnxmltools import __version__ as oml_version
    from onnxmltools.convert.lightgbm.operator_converters.LightGbm import convert_lightgbm  # noqa


    N = 1000
    X = numpy.random.randn(N, 20)
    y = (numpy.random.randn(N) +
         numpy.random.randn(N) * 100 * numpy.random.randint(0, 1, 1000))

    reg = LGBMRegressor(n_estimators=1000)
    reg.fit(X, y)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <style>#sk-container-id-9 {color: black;background-color: white;}#sk-container-id-9 pre{padding: 0;}#sk-container-id-9 div.sk-toggleable {background-color: white;}#sk-container-id-9 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-9 label.sk-toggleable__label-arrow:before {content: "▸";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-9 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-9 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-9 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-9 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-9 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-9 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: "▾";}#sk-container-id-9 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-9 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-9 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-9 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-9 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-9 div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-9 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-9 div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-9 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-9 div.sk-item {position: relative;z-index: 1;}#sk-container-id-9 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-9 div.sk-item::before, #sk-container-id-9 div.sk-parallel-item::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-9 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-9 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-9 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-9 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-9 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-9 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-9 div.sk-label-container {text-align: center;}#sk-container-id-9 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-9 div.sk-text-repr-fallback {display: none;}</style><div id="sk-container-id-9" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>LGBMRegressor(n_estimators=1000)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class="sk-container" hidden><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-28" type="checkbox" checked><label for="sk-estimator-id-28" class="sk-toggleable__label sk-toggleable__label-arrow">LGBMRegressor</label><div class="sk-toggleable__content"><pre>LGBMRegressor(n_estimators=1000)</pre></div></div></div></div></div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 70-81

Register the converter for LGBMRegressor
++++++++++++++++++++++++++++++++++++++++

The converter is implemented in :epkg:`onnxmltools`:
`onnxmltools...LightGbm.py
<https://github.com/onnx/onnxmltools/blob/master/onnxmltools/convert/
lightgbm/operator_converters/LightGbm.py>`_.
and the shape calculator:
`onnxmltools...Regressor.py
<https://github.com/onnx/onnxmltools/blob/master/onnxmltools/convert/
lightgbm/shape_calculators/Regressor.py>`_.

.. GENERATED FROM PYTHON SOURCE LINES 81-102

.. code-block:: default



    def skl2onnx_convert_lightgbm(scope, operator, container):
        options = scope.get_options(operator.raw_operator)
        if 'split' in options:
            if Version(oml_version) < Version('1.9.2'):
                warnings.warn(
                    "Option split was released in version 1.9.2 but %s is "
                    "installed. It will be ignored." % oml_version)
            operator.split = options['split']
        else:
            operator.split = None
        convert_lightgbm(scope, operator, container)


    update_registered_converter(
        LGBMRegressor, 'LightGbmLGBMRegressor',
        calculate_linear_regressor_output_shapes,
        skl2onnx_convert_lightgbm,
        options={'split': None})








.. GENERATED FROM PYTHON SOURCE LINES 103-109

Convert
+++++++

We convert the same model following the two scenarios, one single
TreeEnsembleRegressor node, or more. *split* parameter is the number of
trees per node TreeEnsembleRegressor.

.. GENERATED FROM PYTHON SOURCE LINES 109-117

.. code-block:: default


    model_onnx = to_onnx(reg, X[:1].astype(numpy.float32),
                         target_opset={'': 17, 'ai.onnx.ml': 3})

    model_onnx_split = to_onnx(reg, X[:1].astype(numpy.float32),
                               target_opset={'': 17, 'ai.onnx.ml': 3},
                               options={'split': 100})








.. GENERATED FROM PYTHON SOURCE LINES 118-120

We create another model using the `ai.onnx.ml == 3`.
Node thresholds are stored in doubles and not in floats anymore.

.. GENERATED FROM PYTHON SOURCE LINES 120-125

.. code-block:: default


    model_onnx_64 = to_onnx(reg, X[:1].astype(numpy.float64),
                            target_opset={'': 17, 'ai.onnx.ml': 3},
                            rewrite_ops=True)








.. GENERATED FROM PYTHON SOURCE LINES 126-128

Discrepancies
+++++++++++++

.. GENERATED FROM PYTHON SOURCE LINES 128-146

.. code-block:: default


    sess = InferenceSession(model_onnx.SerializeToString(),
                            providers=['CPUExecutionProvider'])
    sess_split = InferenceSession(model_onnx_split.SerializeToString(),
                                  providers=['CPUExecutionProvider'])

    X32 = X.astype(numpy.float32)[:500]
    expected = reg.predict(X32)
    got = sess.run(None, {'X': X32})[0].ravel()
    got_split = sess_split.run(None, {'X': X32})[0].ravel()

    disp = numpy.abs(got - expected).sum()
    disc_split = numpy.abs(got_split - expected).sum()

    print(f"sum of discrepancies 1 node: {disp}")
    print(f"sum of discrepancies split node: {disc_split}, "
          f"ratio: {disp / disc_split}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    sum of discrepancies 1 node: 6.853524745783303e-05
    sum of discrepancies split node: 2.0830457483485063e-05, ratio: 3.2901460523452055




.. GENERATED FROM PYTHON SOURCE LINES 147-149

The sum of the discrepancies were reduced 4, 5 times.
The maximum is much better too.

.. GENERATED FROM PYTHON SOURCE LINES 149-156

.. code-block:: default


    disc = numpy.abs(got - expected).max()
    disc_split = numpy.abs(got_split - expected).max()

    print("max discrepancies 1 node", disc)
    print("max discrepancies split node", disc_split, "ratio:", disc / disc_split)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    max discrepancies 1 node 1.3420434359368016e-06
    max discrepancies split node 3.822674381481761e-07 ratio: 3.5107448398903207




.. GENERATED FROM PYTHON SOURCE LINES 157-160

Let's compare with the double thresholds.
We compare the inputs into float first and then in double
to make sure they are the same.

.. GENERATED FROM PYTHON SOURCE LINES 160-171

.. code-block:: default


    sess_64 = InferenceSession(model_onnx_64.SerializeToString(),
                               providers=['CPUExecutionProvider'])

    X64 = X32.astype(numpy.float64)
    expected_64 = reg.predict(X64)
    got_64 = sess_64.run(None, {'X': X64})[0].ravel()
    disc_64 = numpy.abs(got_64 - expected_64).sum()
    disc_max64 = numpy.abs(got_64 - expected_64).max()
    print(f"sum of discrepancies with doubles: sum={disc_64}, max={disc_max64}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    sum of discrepancies with doubles: sum=9.069053242884979e-06, max=1.1771164043494764e-07




.. GENERATED FROM PYTHON SOURCE LINES 172-176

Processing time
+++++++++++++++

The processing time is slower but not much.

.. GENERATED FROM PYTHON SOURCE LINES 176-187

.. code-block:: default


    print("processing time no split",
          timeit.timeit(
              lambda: sess.run(None, {'X': X32})[0], number=150))
    print("processing time no split with double",
          timeit.timeit(
              lambda: sess_64.run(None, {'X': X64})[0], number=150))
    print("processing time split",
          timeit.timeit(
              lambda: sess_split.run(None, {'X': X32})[0], number=150))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    processing time no split 0.773850008030422
    processing time no split with double 0.7656510029919446
    processing time split 0.8690498230280355




.. GENERATED FROM PYTHON SOURCE LINES 188-193

Split influence
+++++++++++++++

Let's see how the sum of the discrepancies moves against
the parameter *split*.

.. GENERATED FROM PYTHON SOURCE LINES 193-215

.. code-block:: default


    res = []
    for i in tqdm(list(range(20, 170, 20)) + [200, 300, 400, 500]):
        model_onnx_split = to_onnx(reg, X[:1].astype(numpy.float32),
                                   target_opset={'': 17, 'ai.onnx.ml': 3},
                                   options={'split': i})
        times = []
        for _ in range(0, 4):
            begin = time.perf_counter()
            sess_split = InferenceSession(model_onnx_split.SerializeToString(),
                                          providers=['CPUExecutionProvider'])
            times.append(time.perf_counter() - begin)
        times.sort()
        got_split = sess_split.run(None, {'X': X32})[0].ravel()
        disc_split = numpy.abs(got_split - expected).max()
        res.append(dict(split=i, max_diff=disc_split, time=sum(times[1:3]) / 2))

    df = DataFrame(res).set_index('split')
    df["baseline"] = disc
    df["baseline_64"] = disc_max64
    print(df)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


      0%|          | 0/12 [00:00<?, ?it/s]
      8%|8         | 1/12 [00:11<02:02, 11.09s/it]
     17%|#6        | 2/12 [00:21<01:47, 10.75s/it]
     25%|##5       | 3/12 [00:32<01:38, 10.97s/it]
     33%|###3      | 4/12 [00:43<01:25, 10.71s/it]
     42%|####1     | 5/12 [00:53<01:13, 10.56s/it]
     50%|#####     | 6/12 [01:03<01:02, 10.47s/it]
     58%|#####8    | 7/12 [01:14<00:53, 10.70s/it]
     67%|######6   | 8/12 [01:25<00:42, 10.58s/it]
     75%|#######5  | 9/12 [01:35<00:31, 10.47s/it]
     83%|########3 | 10/12 [01:45<00:20, 10.40s/it]
     92%|#########1| 11/12 [01:55<00:10, 10.34s/it]
    100%|##########| 12/12 [02:06<00:00, 10.54s/it]
    100%|##########| 12/12 [02:06<00:00, 10.58s/it]
               max_diff      time  baseline   baseline_64
    split                                                
    20     1.499505e-07  0.245719  0.000001  1.177116e-07
    40     3.415094e-07  0.253975  0.000001  1.177116e-07
    60     2.809303e-07  0.275046  0.000001  1.177116e-07
    80     3.464011e-07  0.281518  0.000001  1.177116e-07
    100    3.822674e-07  0.287719  0.000001  1.177116e-07
    120    3.544410e-07  0.292886  0.000001  1.177116e-07
    140    3.822674e-07  0.301454  0.000001  1.177116e-07
    160    5.536555e-07  0.305995  0.000001  1.177116e-07
    200    6.395834e-07  0.297995  0.000001  1.177116e-07
    300    3.822674e-07  0.304245  0.000001  1.177116e-07
    400    6.267877e-07  0.301312  0.000001  1.177116e-07
    500    7.292194e-07  0.311266  0.000001  1.177116e-07




.. GENERATED FROM PYTHON SOURCE LINES 216-217

Graph.

.. GENERATED FROM PYTHON SOURCE LINES 217-227

.. code-block:: default


    fig, ax = plt.subplots(1, 2, figsize=(10, 4))
    df[["max_diff", "baseline", "baseline_64"]].plot(
        title="Sum of discrepancies against split\n"
              "split = numbers of tree per node",
        ax=ax[0])
    df[["time"]].plot(title="Processing time against split\n"
                            "split = numbers of tree per node",
                      ax=ax[1])




.. image-sg:: /gyexamples/images/sphx_glr_plot_gexternal_lightgbm_reg_001.png
   :alt: Sum of discrepancies against split split = numbers of tree per node, Processing time against split split = numbers of tree per node
   :srcset: /gyexamples/images/sphx_glr_plot_gexternal_lightgbm_reg_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <AxesSubplot: title={'center': 'Processing time against split\nsplit = numbers of tree per node'}, xlabel='split'>



.. GENERATED FROM PYTHON SOURCE LINES 228-237

Conclusion
++++++++++

The time curve is too noisy to conclude.
More measures should be made.
The double sum reduces the discrepancies
but increases the processing time. It is a tradeoff.
The best option is using double for threshold and summation
but it requires the latest definition of TreeEnsemble `ai.onnx.ml=3`.

.. GENERATED FROM PYTHON SOURCE LINES 237-240

.. code-block:: default



    # plt.show()








.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 2 minutes  46.095 seconds)


.. _sphx_glr_download_gyexamples_plot_gexternal_lightgbm_reg.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_gexternal_lightgbm_reg.py <plot_gexternal_lightgbm_reg.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_gexternal_lightgbm_reg.ipynb <plot_gexternal_lightgbm_reg.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_