module tools.ort_wrapper#

Inheritance diagram of mlprodict.tools.ort_wrapper

Short summary#

module mlprodict.tools.ort_wrapper

Wrapper around onnxruntime.

Classes#

class

truncated documentation

InferenceSession

Wrappers around InferenceSession from onnxruntime.

Functions#

function

truncated documentation

prepare_c_profiling

Prepares model and data to be profiled with tool perftest …

Methods#

method

truncated documentation

__init__

end_profiling

Ends profiling.

get_inputs

Returns input types.

get_outputs

Returns output types.

run

Executes the ONNX graph.

Documentation#

Wrapper around onnxruntime.

New in version 0.6.

source on GitHub

class mlprodict.tools.ort_wrapper.InferenceSession(onnx_bytes, sess_options=None, log_severity_level=4, runtime='onnxruntime', providers=None)#

Bases: object

Wrappers around InferenceSession from onnxruntime.

Parameters:

  • onnx_bytes – onnx bytes

  • session_options – session options

  • log_severity_level – change the logging level

  • runtime – runtime to use, onnxruntime, onnxruntime-cuda, …

  • providers – providers

source on GitHub

__init__(onnx_bytes, sess_options=None, log_severity_level=4, runtime='onnxruntime', providers=None)#
end_profiling()#

Ends profiling.

get_inputs()#

Returns input types.

get_outputs()#

Returns output types.

run(output_names, input_feed, run_options=None)#

Executes the ONNX graph.

Parameters:

  • output_names – None for all, a name for a specific output

  • input_feed – dictionary of inputs

  • run_options – None or RunOptions

Returns:

array

source on GitHub

mlprodict.tools.ort_wrapper.prepare_c_profiling(model_onnx, inputs, dest=None)#

Prepares model and data to be profiled with tool perftest (onnxruntime) or onnx_test_runner. It saves the model in folder dest and dumps the inputs in a subfolder.

Parameters:

  • model_onnx – onnx model

  • inputs – inputs as a list of a dictionary

  • dest – destination folder, None means the current folder

Returns:

command line to use

source on GitHub