.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "gyexamples/plot_benchmark_op_leakyrelu.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_gyexamples_plot_benchmark_op_leakyrelu.py: .. _l-example-benckmark-ort-leaky-relu: Benchmark operator LeakyRelu ============================ The operator `LeakyRelu` is equivalent to the function: :math:`LeayRelu(x) = \begin{array}{l} x \text{ if } x > 0 \\ \alpha x \text{otherwise} \end{array}`. But it could be rewritten into the following decomposition :math:`LeayRelu(x) = x (\indicatrice{x} + \alpha (1 - \indicatrice{x})) = x ((1 - \alpha) \indicatrice{x} + \alpha)`. Let's compare the two implementation with onnx runtimes. .. contents:: :local: The ONNX graphs for both implementations of LeakyRely +++++++++++++++++++++++++++++++++++++++++++++++++++++ .. GENERATED FROM PYTHON SOURCE LINES 22-42 .. code-block:: default import numpy from numpy.testing import assert_almost_equal import matplotlib.pyplot as plt from pandas import DataFrame from onnx import TensorProto from onnxruntime import InferenceSession, get_device from skl2onnx.common.data_types import FloatTensorType from skl2onnx.algebra.onnx_ops import ( OnnxLeakyRelu, OnnxSign, OnnxMul, OnnxAdd, OnnxDiv, OnnxGreater, OnnxCast) from cpyquickhelper.numbers.speed_measure import measure_time from mlprodict.testing.experimental_c_impl.experimental_c import code_optimisation from mlprodict.plotting.plotting import onnx_simple_text_plot from onnxcustom.plotting.plotting_onnx import plot_onnxs from tqdm import tqdm print([code_optimisation(), get_device()]) .. rst-class:: sphx-glr-script-out .. code-block:: none ['AVX-omp=8', 'CPU'] .. GENERATED FROM PYTHON SOURCE LINES 43-44 First implementation: the operator LeayRelu. .. GENERATED FROM PYTHON SOURCE LINES 44-58 .. code-block:: default def build_leaky_relu(alpha=0.5, target_opset=15): x = OnnxLeakyRelu('X', alpha=alpha, op_version=target_opset, output_names=['Y']) return x.to_onnx({'X': FloatTensorType()}, outputs={'Y': FloatTensorType()}, target_opset=target_opset) onx_leaky = build_leaky_relu() print(onnx_simple_text_plot(onx_leaky)) .. rst-class:: sphx-glr-script-out .. code-block:: none opset: domain='' version=15 input: name='X' type=dtype('float32') shape=None LeakyRelu(X, alpha=0.50) -> Y output: name='Y' type=dtype('float32') shape=None .. GENERATED FROM PYTHON SOURCE LINES 59-61 Second option, the formula introduced above must adapted as ONNX operator Sign returns -1 if *x* is negative and not 0. .. GENERATED FROM PYTHON SOURCE LINES 61-83 .. code-block:: default def build_leaky_relu_decomposed(alpha=0.5, target_opset=15): signo = OnnxSign('X', op_version=target_opset) sign = OnnxDiv( OnnxAdd(signo, numpy.array([1], dtype=numpy.float32), op_version=target_opset), numpy.array([2], dtype=numpy.float32), op_version=target_opset) fact = OnnxAdd( OnnxMul(sign, numpy.array([1 - alpha], dtype=numpy.float32), op_version=target_opset), numpy.array([alpha], dtype=numpy.float32), op_version=target_opset) x = OnnxMul('X', fact, op_version=target_opset, output_names=['Y']) return x.to_onnx({'X': FloatTensorType()}, outputs={'Y': FloatTensorType()}, target_opset=target_opset) onx_leaky_dec = build_leaky_relu_decomposed() print(onnx_simple_text_plot(onx_leaky_dec)) .. rst-class:: sphx-glr-script-out .. code-block:: none opset: domain='' version=15 input: name='X' type=dtype('float32') shape=None init: name='Ad_Addcst' type=dtype('float32') shape=(1,) -- array([1.], dtype=float32) init: name='Di_Divcst' type=dtype('float32') shape=(1,) -- array([2.], dtype=float32) init: name='Mu_Mulcst' type=dtype('float32') shape=(1,) -- array([0.5], dtype=float32) Identity(Mu_Mulcst) -> Ad_Addcst1 Sign(X) -> Si_output0 Add(Si_output0, Ad_Addcst) -> Ad_C01 Div(Ad_C01, Di_Divcst) -> Di_C0 Mul(Di_C0, Mu_Mulcst) -> Mu_C0 Add(Mu_C0, Ad_Addcst1) -> Ad_C0 Mul(X, Ad_C0) -> Y output: name='Y' type=dtype('float32') shape=None .. GENERATED FROM PYTHON SOURCE LINES 84-85 Third option, use of operater Greater .. GENERATED FROM PYTHON SOURCE LINES 85-107 .. code-block:: default def build_leaky_relu_decomposed_greater(alpha=0.5, target_opset=15): signo = OnnxGreater('X', numpy.array([0], dtype=numpy.float32), op_version=target_opset) sign = OnnxCast(signo, to=TensorProto.FLOAT, op_version=target_opset) fact = OnnxAdd( OnnxMul(sign, numpy.array([1 - alpha], dtype=numpy.float32), op_version=target_opset), numpy.array([alpha], dtype=numpy.float32), op_version=target_opset) x = OnnxMul('X', fact, op_version=target_opset, output_names=['Y']) return x.to_onnx({'X': FloatTensorType()}, outputs={'Y': FloatTensorType()}, target_opset=target_opset) onx_leaky_dec_greater = build_leaky_relu_decomposed_greater() print(onnx_simple_text_plot(onx_leaky_dec_greater)) .. rst-class:: sphx-glr-script-out .. code-block:: none opset: domain='' version=15 input: name='X' type=dtype('float32') shape=None init: name='Gr_Greatercst' type=dtype('float32') shape=(1,) -- array([0.], dtype=float32) init: name='Mu_Mulcst' type=dtype('float32') shape=(1,) -- array([0.5], dtype=float32) Greater(X, Gr_Greatercst) -> Gr_C0 Cast(Gr_C0, to=1) -> Ca_output0 Mul(Ca_output0, Mu_Mulcst) -> Mu_C0 Identity(Mu_Mulcst) -> Ad_Addcst Add(Mu_C0, Ad_Addcst) -> Ad_C0 Mul(X, Ad_C0) -> Y output: name='Y' type=dtype('float32') shape=None .. GENERATED FROM PYTHON SOURCE LINES 108-109 Visually .. GENERATED FROM PYTHON SOURCE LINES 109-115 .. code-block:: default plot_onnxs(onx_leaky, onx_leaky_dec, onx_leaky_dec_greater, title=["One operator", "Decomposed\nLeakyRelu", "Decomposed\nLeakyRelu Greater"]) .. image-sg:: /gyexamples/images/sphx_glr_plot_benchmark_op_leakyrelu_001.png :alt: One operator, Decomposed LeakyRelu, Decomposed LeakyRelu Greater :srcset: /gyexamples/images/sphx_glr_plot_benchmark_op_leakyrelu_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none array([, , ], dtype=object) .. GENERATED FROM PYTHON SOURCE LINES 116-118 Check that both graph returns are equivalent ++++++++++++++++++++++++++++++++++++++++++++ .. GENERATED FROM PYTHON SOURCE LINES 118-134 .. code-block:: default sess1 = InferenceSession(onx_leaky.SerializeToString(), providers=['CPUExecutionProvider']) sess_dec = InferenceSession(onx_leaky_dec.SerializeToString(), providers=['CPUExecutionProvider']) sess_dec_greater = InferenceSession(onx_leaky_dec_greater.SerializeToString(), providers=['CPUExecutionProvider']) for shape in [(1, ), (10, ), (5, 5), (7, 2, 4)]: rnd = numpy.random.randn(*shape).astype(numpy.float32) res1 = sess1.run(None, {'X': rnd})[0] res_dec = sess_dec.run(None, {'X': rnd})[0] res_dec_greater = sess_dec_greater.run(None, {'X': rnd})[0] assert_almost_equal(res1, res_dec) assert_almost_equal(res1, res_dec_greater) .. GENERATED FROM PYTHON SOURCE LINES 135-137 Benchmark +++++++++ .. GENERATED FROM PYTHON SOURCE LINES 137-164 .. code-block:: default fcts = [('leakyrelu', sess1), ('dec', sess_dec), ('dec_greater', sess_dec_greater)] N = 100 data = [] for dim in tqdm([10, 128, 256, 512, 1000, 2000]): for shape in [(N, dim), (dim, N)]: rnd = numpy.random.randn(*shape).astype(numpy.float32) for name, sess in fcts: repeat = int(4001 / dim) obs = measure_time( lambda: sess.run(None, {'X': rnd}), context=dict(rnd=rnd, sess=sess), div_by_number=True, repeat=repeat, number=200) obs['name'] = name obs['N'] = N obs['dim'] = dim obs['orient'] = shape[0] == N obs['shape'] = "%dx%d" % shape data.append(obs) df = DataFrame(data) df[['name', 'N', 'dim', 'average', 'deviation']] print(df[['name', 'N', 'dim', 'average']]) .. rst-class:: sphx-glr-script-out .. code-block:: none 0%| | 0/6 [00:00
name dec dec_greater leakyrelu
shape
1000x100 0.358262 0.475129 1
100x10 0.689093 0.780442 1
100x1000 0.273181 0.346878 1
100x128 0.390485 0.649343 1
100x2000 0.336393 0.465464 1
100x256 0.272989 0.480707 1
100x512 0.183923 0.300987 1
10x100 0.692456 0.783562 1
128x100 0.384721 0.640376 1
2000x100 0.316868 0.453896 1
256x100 0.266726 0.477558 1
512x100 0.184956 0.302847 1


.. GENERATED FROM PYTHON SOURCE LINES 181-182 Graph. .. GENERATED FROM PYTHON SOURCE LINES 182-191 .. code-block:: default fig, ax = plt.subplots(1, 2, figsize=(12, 5)) speedup(df[df.orient].pivot('dim', 'name', 'average')).plot(ax=ax[0]) ax[0].set_title("LeakyRelu speedup, shape=(%d,dim)" "\nThe higher the better" % N) speedup(df[~df.orient].pivot('dim', 'name', 'average')).plot(ax=ax[1]) ax[1].set_title("LeakyRelu speedup, shape=(dim,%d)" "\nThe higher the better" % N) .. image-sg:: /gyexamples/images/sphx_glr_plot_benchmark_op_leakyrelu_002.png :alt: LeakyRelu speedup, shape=(100,dim) The higher the better, LeakyRelu speedup, shape=(dim,100) The higher the better :srcset: /gyexamples/images/sphx_glr_plot_benchmark_op_leakyrelu_002.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none Text(0.5, 1.0, 'LeakyRelu speedup, shape=(dim,100)\nThe higher the better') .. GENERATED FROM PYTHON SOURCE LINES 192-194 This kind of benchmark helps finding better implementation of operator runtime. .. GENERATED FROM PYTHON SOURCE LINES 194-196 .. code-block:: default # plt.show() .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 49.961 seconds) .. _sphx_glr_download_gyexamples_plot_benchmark_op_leakyrelu.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_benchmark_op_leakyrelu.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_benchmark_op_leakyrelu.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_