.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "gyexamples/plot_profile_ort.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_gyexamples_plot_profile_ort.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_gyexamples_plot_profile_ort.py:


.. _l-profile-ort-api:

Profile onnxruntime execution
=============================

The following examples converts a model into :epkg:`ONNX` and runs it
with :epkg:`onnxruntime`. This one is then uses to profile the execution
by looking the time spent in each operator. This analysis gives some
hints on how to optimize the processing time by looking the nodes
consuming most of the ressources.

.. contents::
    :local:

Neareast Neighbours
+++++++++++++++++++

.. GENERATED FROM PYTHON SOURCE LINES 20-35

.. code-block:: default

    import json
    import numpy
    import matplotlib.pyplot as plt
    from mpl_toolkits.axes_grid1.axes_divider import make_axes_area_auto_adjustable
    import pandas
    from onnxruntime import InferenceSession, SessionOptions, get_device
    from onnxruntime.capi._pybind_state import (  # pylint: disable=E0611
        SessionIOBinding, OrtDevice as C_OrtDevice, OrtValue as C_OrtValue)
    from sklearn.neighbors import RadiusNeighborsRegressor
    from skl2onnx import to_onnx
    from tqdm import tqdm
    from mlprodict.testing.experimental_c_impl.experimental_c import code_optimisation
    from mlprodict.plotting.plotting import onnx_simple_text_plot, plot_onnx
    from mlprodict.onnxrt.ops_whole.session import OnnxWholeSession








.. GENERATED FROM PYTHON SOURCE LINES 36-37

Available optimisation on this machine.

.. GENERATED FROM PYTHON SOURCE LINES 37-41

.. code-block:: default


    print(code_optimisation())






.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    AVX-omp=8




.. GENERATED FROM PYTHON SOURCE LINES 42-44

Building the model
++++++++++++++++++

.. GENERATED FROM PYTHON SOURCE LINES 44-51

.. code-block:: default


    X = numpy.random.randn(1000, 10).astype(numpy.float64)
    y = X.sum(axis=1).reshape((-1, 1))

    model = RadiusNeighborsRegressor()
    model.fit(X, y)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <style>#sk-container-id-7 {color: black;background-color: white;}#sk-container-id-7 pre{padding: 0;}#sk-container-id-7 div.sk-toggleable {background-color: white;}#sk-container-id-7 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-7 label.sk-toggleable__label-arrow:before {content: "▸";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-7 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-7 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-7 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-7 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-7 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-7 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: "▾";}#sk-container-id-7 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-7 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-7 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-7 div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-7 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-7 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-7 div.sk-item {position: relative;z-index: 1;}#sk-container-id-7 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-7 div.sk-item::before, #sk-container-id-7 div.sk-parallel-item::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-7 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-7 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-7 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-7 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-7 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-7 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-7 div.sk-label-container {text-align: center;}#sk-container-id-7 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-7 div.sk-text-repr-fallback {display: none;}</style><div id="sk-container-id-7" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>RadiusNeighborsRegressor()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class="sk-container" hidden><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-24" type="checkbox" checked><label for="sk-estimator-id-24" class="sk-toggleable__label sk-toggleable__label-arrow">RadiusNeighborsRegressor</label><div class="sk-toggleable__content"><pre>RadiusNeighborsRegressor()</pre></div></div></div></div></div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 52-54

Conversion to ONNX
++++++++++++++++++

.. GENERATED FROM PYTHON SOURCE LINES 54-60

.. code-block:: default


    onx = to_onnx(model, X, options={'optim': 'cdist'},
                  target_opset=17)

    print(onnx_simple_text_plot(onx))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    opset: domain='' version=17
    opset: domain='ai.onnx.ml' version=1
    opset: domain='com.microsoft' version=1
    input: name='X' type=dtype('float64') shape=[None, 10]
    init: name='knny_ArrayFeatureExtractorcst' type=dtype('float64') shape=(1000,)
    init: name='cond_CDistcst' type=dtype('float64') shape=(1000, 10)
    init: name='cond_Lesscst' type=dtype('float64') shape=(1,) -- array([1.])
    init: name='arange_CumSumcst' type=dtype('int64') shape=(1,) -- array([1])
    init: name='knny_Reshapecst' type=dtype('int64') shape=(2,) -- array([  -1, 1000])
    init: name='Re_Reshapecst' type=dtype('int64') shape=(2,) -- array([-1,  1])
    CDist[com.microsoft](X, cond_CDistcst, metric=b'euclidean') -> cond_dist
      Less(cond_dist, cond_Lesscst) -> cond_C0
        Cast(cond_C0, to=11) -> nnbin_output0
          ReduceSum(nnbin_output0, arange_CumSumcst, keepdims=0) -> norm_reduced0
      Shape(cond_dist) -> arange_shape0
        ConstantOfShape(arange_shape0, value=[-1.0]) -> arange_output01
          Cast(arange_output01, to=7) -> arange_output0
            CumSum(arange_output0, arange_CumSumcst) -> arange_y0
              Neg(arange_y0) -> arange_Y0
            Add(arange_Y0, arange_output0) -> arange_C0
        Where(cond_C0, arange_C0, arange_output0) -> nnind_output0
          Flatten(nnind_output0) -> knny_output0
            ArrayFeatureExtractor(knny_ArrayFeatureExtractorcst, knny_output0) -> knny_Z0
              Reshape(knny_Z0, knny_Reshapecst, allowzero=0) -> knny_reshaped0
                Cast(knny_reshaped0, to=11) -> final_output0
          Mul(final_output0, nnbin_output0) -> final_C0
            ReduceSum(final_C0, arange_CumSumcst, keepdims=0) -> final_reduced0
              Shape(final_reduced0) -> normr_shape0
            Reshape(norm_reduced0, normr_shape0, allowzero=0) -> normr_reshaped0
              Div(final_reduced0, normr_reshaped0) -> Di_C0
                Reshape(Di_C0, Re_Reshapecst, allowzero=0) -> variable
    output: name='variable' type=dtype('float64') shape=[None, 1]




.. GENERATED FROM PYTHON SOURCE LINES 61-62

The ONNX graph looks like the following.

.. GENERATED FROM PYTHON SOURCE LINES 62-67

.. code-block:: default


    _, ax = plt.subplots(1, 1, figsize=(8, 15))
    plot_onnx(onx, ax=ax)





.. image-sg:: /gyexamples/images/sphx_glr_plot_profile_ort_001.png
   :alt: plot profile ort
   :srcset: /gyexamples/images/sphx_glr_plot_profile_ort_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <AxesSubplot: >



.. GENERATED FROM PYTHON SOURCE LINES 68-74

Profiling
+++++++++

The profiling is enabled by setting attribute `enable_profling`
in :epkg:`SessionOptions`. Method *end_profiling* collects
all the results and stores it on disk in :epkg:`JSON` format.

.. GENERATED FROM PYTHON SOURCE LINES 74-87

.. code-block:: default


    so = SessionOptions()
    so.enable_profiling = True
    sess = InferenceSession(onx.SerializeToString(), so,
                            providers=['CPUExecutionProvider'])
    feeds = {'X': X[:100]}

    for i in tqdm(range(0, 10)):
        sess.run(None, feeds)

    prof = sess.end_profiling()
    print(prof)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


      0%|          | 0/10 [00:00<?, ?it/s]
     90%|######### | 9/10 [00:00<00:00, 86.09it/s]
    100%|##########| 10/10 [00:00<00:00, 85.23it/s]
    onnxruntime_profile__2023-01-17_01-56-07.json




.. GENERATED FROM PYTHON SOURCE LINES 88-90

Better rendering
++++++++++++++++

.. GENERATED FROM PYTHON SOURCE LINES 90-97

.. code-block:: default



    with open(prof, "r") as f:
        js = json.load(f)
    df = pandas.DataFrame(OnnxWholeSession.process_profiling(js))
    df






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>cat</th>
          <th>pid</th>
          <th>tid</th>
          <th>dur</th>
          <th>ts</th>
          <th>ph</th>
          <th>name</th>
          <th>args_op_name</th>
          <th>args_thread_scheduling_stats</th>
          <th>args_input_type_shape</th>
          <th>args_activation_size</th>
          <th>args_parameter_size</th>
          <th>args_graph_index</th>
          <th>args_output_size</th>
          <th>args_provider</th>
          <th>args_output_type_shape</th>
          <th>args_exec_plan_index</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>Session</td>
          <td>32082</td>
          <td>32082</td>
          <td>5242</td>
          <td>6</td>
          <td>X</td>
          <td>model_loading_array</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>1</th>
          <td>Session</td>
          <td>32082</td>
          <td>32082</td>
          <td>7317</td>
          <td>5340</td>
          <td>X</td>
          <td>session_initialization</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>2</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>2</td>
          <td>14974</td>
          <td>X</td>
          <td>cond_CDist_fence_before</td>
          <td>CDist</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>3</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>2881</td>
          <td>14988</td>
          <td>X</td>
          <td>cond_CDist_kernel_time</td>
          <td>CDist</td>
          <td>{'main_thread': {'thread_pool_name': 'session-...</td>
          <td>[{'double': [100, 10]}, {'double': [1000, 10]}]</td>
          <td>8000</td>
          <td>80000</td>
          <td>0</td>
          <td>800000</td>
          <td>CPUExecutionProvider</td>
          <td>[{'double': [100, 1000]}]</td>
          <td>0</td>
        </tr>
        <tr>
          <th>4</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>0</td>
          <td>17895</td>
          <td>X</td>
          <td>cond_CDist_fence_after</td>
          <td>CDist</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>617</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>0</td>
          <td>131850</td>
          <td>X</td>
          <td>Re_Reshape_fence_before</td>
          <td>Reshape</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>618</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>40</td>
          <td>131853</td>
          <td>X</td>
          <td>Re_Reshape_kernel_time</td>
          <td>Reshape</td>
          <td>{'main_thread': {'thread_pool_name': 'session-...</td>
          <td>[{'double': [100]}, {'int64': [2]}]</td>
          <td>800</td>
          <td>16</td>
          <td>20</td>
          <td>800</td>
          <td>CPUExecutionProvider</td>
          <td>[{'double': [100, 1]}]</td>
          <td>20</td>
        </tr>
        <tr>
          <th>619</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>0</td>
          <td>131903</td>
          <td>X</td>
          <td>Re_Reshape_fence_after</td>
          <td>Reshape</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>620</th>
          <td>Session</td>
          <td>32082</td>
          <td>32082</td>
          <td>11483</td>
          <td>120426</td>
          <td>X</td>
          <td>SequentialExecutor::Execute</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>621</th>
          <td>Session</td>
          <td>32082</td>
          <td>32082</td>
          <td>11509</td>
          <td>120411</td>
          <td>X</td>
          <td>model_run</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
      </tbody>
    </table>
    <p>622 rows × 17 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 98-102

Graphs
++++++

First graph is by operator type.

.. GENERATED FROM PYTHON SOURCE LINES 102-116

.. code-block:: default


    gr_dur = df[['dur', "args_op_name"]].groupby(
        "args_op_name").sum().sort_values('dur')
    gr_n = df[['dur', "args_op_name"]].groupby(
        "args_op_name").count().sort_values('dur')
    gr_n = gr_n.loc[gr_dur.index, :]

    fig, ax = plt.subplots(1, 2, figsize=(8, 4))
    gr_dur.plot.barh(ax=ax[0])
    gr_n.plot.barh(ax=ax[1])
    ax[0].set_title("duration")
    ax[1].set_title("n occurences")
    fig.suptitle(model.__class__.__name__)




.. image-sg:: /gyexamples/images/sphx_glr_plot_profile_ort_002.png
   :alt: RadiusNeighborsRegressor, duration, n occurences
   :srcset: /gyexamples/images/sphx_glr_plot_profile_ort_002.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Text(0.5, 0.98, 'RadiusNeighborsRegressor')



.. GENERATED FROM PYTHON SOURCE LINES 117-118

Second graph is by operator name.

.. GENERATED FROM PYTHON SOURCE LINES 118-123

.. code-block:: default


    gr_dur = df[['dur', "args_op_name", "name"]].groupby(
        ["args_op_name", "name"]).sum().sort_values('dur')
    gr_dur.head(n=5)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th></th>
          <th>dur</th>
        </tr>
        <tr>
          <th>args_op_name</th>
          <th>name</th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>Flatten</th>
          <th>knny_Flatten_fence_after</th>
          <td>0</td>
        </tr>
        <tr>
          <th rowspan="2" valign="top">Less</th>
          <th>cond_Less_fence_after</th>
          <td>0</td>
        </tr>
        <tr>
          <th>cond_Less_fence_before</th>
          <td>0</td>
        </tr>
        <tr>
          <th rowspan="2" valign="top">Mul</th>
          <th>final_Mul_fence_after</th>
          <td>0</td>
        </tr>
        <tr>
          <th>final_Mul_fence_before</th>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 124-125

And the graph.

.. GENERATED FROM PYTHON SOURCE LINES 125-133

.. code-block:: default


    _, ax = plt.subplots(1, 1, figsize=(8, gr_dur.shape[0] // 2))
    gr_dur.plot.barh(ax=ax)
    ax.set_title("duration per node")
    for label in (ax.get_xticklabels() + ax.get_yticklabels()):
        label.set_fontsize(7)
    make_axes_area_auto_adjustable(ax)




.. image-sg:: /gyexamples/images/sphx_glr_plot_profile_ort_003.png
   :alt: duration per node
   :srcset: /gyexamples/images/sphx_glr_plot_profile_ort_003.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 134-136

The model spends most of its time in CumSum operator.
Operator Shape gets called the highest number of times.

.. GENERATED FROM PYTHON SOURCE LINES 136-140

.. code-block:: default



    # plt.show()








.. GENERATED FROM PYTHON SOURCE LINES 141-143

GPU or CPU
++++++++++

.. GENERATED FROM PYTHON SOURCE LINES 143-160

.. code-block:: default


    if get_device().upper() == 'GPU':
        ort_device = C_OrtDevice(
            C_OrtDevice.cuda(), C_OrtDevice.default_memory(), 0)
    else:
        ort_device = C_OrtDevice(
            C_OrtDevice.cpu(), C_OrtDevice.default_memory(), 0)

    # session
    sess = InferenceSession(onx.SerializeToString(), so,
                            providers=['CPUExecutionProvider',
                                       'CUDAExecutionProvider'])
    bind = SessionIOBinding(sess._sess)

    # moving the data on CPU or GPU
    ort_value = C_OrtValue.ortvalue_from_numpy(X, ort_device)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    somewhere/workspace/onnxcustom/onnxcustom_UT_39_std/_venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:54: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'
      warnings.warn(




.. GENERATED FROM PYTHON SOURCE LINES 161-162

A function which calls the API for any device.

.. GENERATED FROM PYTHON SOURCE LINES 162-172

.. code-block:: default



    def run_with_iobinding(sess, bind, ort_device, ort_value, dtype):
        bind.bind_input('X', ort_device, dtype, ort_value.shape(),
                        ort_value.data_ptr())
        bind.bind_output('variable', ort_device)
        sess._sess.run_with_iobinding(bind, None)
        ortvalues = bind.get_outputs()
        return ortvalues[0].numpy()








.. GENERATED FROM PYTHON SOURCE LINES 173-174

The profiling.

.. GENERATED FROM PYTHON SOURCE LINES 174-185

.. code-block:: default



    for i in tqdm(range(0, 10)):
        run_with_iobinding(sess, bind, ort_device, ort_value, X.dtype)

    prof = sess.end_profiling()
    with open(prof, "r") as f:
        js = json.load(f)
    df = pandas.DataFrame(OnnxWholeSession.process_profiling(js))
    df





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


      0%|          | 0/10 [00:00<?, ?it/s]
     10%|#         | 1/10 [00:00<00:00,  9.62it/s]
     30%|###       | 3/10 [00:00<00:00, 10.19it/s]
     50%|#####     | 5/10 [00:00<00:00, 10.26it/s]
     70%|#######   | 7/10 [00:00<00:00, 10.31it/s]
     90%|######### | 9/10 [00:00<00:00, 10.34it/s]
    100%|##########| 10/10 [00:00<00:00, 10.29it/s]


.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>cat</th>
          <th>pid</th>
          <th>tid</th>
          <th>dur</th>
          <th>ts</th>
          <th>ph</th>
          <th>name</th>
          <th>args_op_name</th>
          <th>args_thread_scheduling_stats</th>
          <th>args_input_type_shape</th>
          <th>args_activation_size</th>
          <th>args_parameter_size</th>
          <th>args_graph_index</th>
          <th>args_output_size</th>
          <th>args_provider</th>
          <th>args_output_type_shape</th>
          <th>args_exec_plan_index</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>Session</td>
          <td>32082</td>
          <td>32082</td>
          <td>812</td>
          <td>6</td>
          <td>X</td>
          <td>model_loading_array</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>1</th>
          <td>Session</td>
          <td>32082</td>
          <td>32082</td>
          <td>7226</td>
          <td>897</td>
          <td>X</td>
          <td>session_initialization</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>2</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>1</td>
          <td>14358</td>
          <td>X</td>
          <td>cond_CDist_fence_before</td>
          <td>CDist</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>3</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>19961</td>
          <td>14369</td>
          <td>X</td>
          <td>cond_CDist_kernel_time</td>
          <td>CDist</td>
          <td>{'main_thread': {'thread_pool_name': 'session-...</td>
          <td>[{'double': [1000, 10]}, {'double': [1000, 10]}]</td>
          <td>80000</td>
          <td>80000</td>
          <td>0</td>
          <td>8000000</td>
          <td>CPUExecutionProvider</td>
          <td>[{'double': [1000, 1000]}]</td>
          <td>0</td>
        </tr>
        <tr>
          <th>4</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>0</td>
          <td>34356</td>
          <td>X</td>
          <td>cond_CDist_fence_after</td>
          <td>CDist</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>617</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>0</td>
          <td>985706</td>
          <td>X</td>
          <td>Re_Reshape_fence_before</td>
          <td>Reshape</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>618</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>41</td>
          <td>985709</td>
          <td>X</td>
          <td>Re_Reshape_kernel_time</td>
          <td>Reshape</td>
          <td>{'main_thread': {'thread_pool_name': 'session-...</td>
          <td>[{'double': [1000]}, {'int64': [2]}]</td>
          <td>8000</td>
          <td>16</td>
          <td>20</td>
          <td>8000</td>
          <td>CPUExecutionProvider</td>
          <td>[{'double': [1000, 1]}]</td>
          <td>20</td>
        </tr>
        <tr>
          <th>619</th>
          <td>Node</td>
          <td>32082</td>
          <td>32082</td>
          <td>0</td>
          <td>985761</td>
          <td>X</td>
          <td>Re_Reshape_fence_after</td>
          <td>Reshape</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>620</th>
          <td>Session</td>
          <td>32082</td>
          <td>32082</td>
          <td>95520</td>
          <td>890247</td>
          <td>X</td>
          <td>SequentialExecutor::Execute</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>621</th>
          <td>Session</td>
          <td>32082</td>
          <td>32082</td>
          <td>95546</td>
          <td>890231</td>
          <td>X</td>
          <td>model_run</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
      </tbody>
    </table>
    <p>622 rows × 17 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 186-187

First graph is by operator type.

.. GENERATED FROM PYTHON SOURCE LINES 187-201

.. code-block:: default


    gr_dur = df[['dur', "args_op_name"]].groupby(
        "args_op_name").sum().sort_values('dur')
    gr_n = df[['dur', "args_op_name"]].groupby(
        "args_op_name").count().sort_values('dur')
    gr_n = gr_n.loc[gr_dur.index, :]

    fig, ax = plt.subplots(1, 2, figsize=(8, 4))
    gr_dur.plot.barh(ax=ax[0])
    gr_n.plot.barh(ax=ax[1])
    ax[0].set_title("duration")
    ax[1].set_title("n occurences")
    fig.suptitle(model.__class__.__name__)




.. image-sg:: /gyexamples/images/sphx_glr_plot_profile_ort_004.png
   :alt: RadiusNeighborsRegressor, duration, n occurences
   :srcset: /gyexamples/images/sphx_glr_plot_profile_ort_004.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Text(0.5, 0.98, 'RadiusNeighborsRegressor')



.. GENERATED FROM PYTHON SOURCE LINES 202-203

Second graph is by operator name.

.. GENERATED FROM PYTHON SOURCE LINES 203-208

.. code-block:: default


    gr_dur = df[['dur', "args_op_name", "name"]].groupby(
        ["args_op_name", "name"]).sum().sort_values('dur')
    gr_dur.head(n=5)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th></th>
          <th>dur</th>
        </tr>
        <tr>
          <th>args_op_name</th>
          <th>name</th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>Flatten</th>
          <th>knny_Flatten_fence_after</th>
          <td>0</td>
        </tr>
        <tr>
          <th rowspan="3" valign="top">Reshape</th>
          <th>knny_Reshape_fence_after</th>
          <td>0</td>
        </tr>
        <tr>
          <th>Re_Reshape_fence_before</th>
          <td>0</td>
        </tr>
        <tr>
          <th>Re_Reshape_fence_after</th>
          <td>0</td>
        </tr>
        <tr>
          <th>ReduceSum</th>
          <th>norm_ReduceSum_fence_before</th>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 209-210

And the graph.

.. GENERATED FROM PYTHON SOURCE LINES 210-218

.. code-block:: default


    _, ax = plt.subplots(1, 1, figsize=(8, gr_dur.shape[0] // 2))
    gr_dur.plot.barh(ax=ax)
    ax.set_title("duration per node")
    for label in (ax.get_xticklabels() + ax.get_yticklabels()):
        label.set_fontsize(7)
    make_axes_area_auto_adjustable(ax)




.. image-sg:: /gyexamples/images/sphx_glr_plot_profile_ort_005.png
   :alt: duration per node
   :srcset: /gyexamples/images/sphx_glr_plot_profile_ort_005.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 219-220

It shows the same results.

.. GENERATED FROM PYTHON SOURCE LINES 220-222

.. code-block:: default


    # plt.show()








.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  11.444 seconds)


.. _sphx_glr_download_gyexamples_plot_profile_ort.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_profile_ort.py <plot_profile_ort.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_profile_ort.ipynb <plot_profile_ort.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_