module `cli.asv_bench`#

Short summary#

module mlprodict.cli.asv_bench

Command line about validation of prediction runtime.

Functions#

function	truncated documentation
`asv_bench`	Creates an asv benchmark in a folder but does not run it.

Documentation#

Command line about validation of prediction runtime.

source on GitHub

mlprodict.cli.asv_bench.asv_bench(location='asvsklonnx', opset_min=-1, opset_max=None, runtime='scikit-learn, python_compiled', models=None, skip_models=None, extended_list=True, dims='1, 10, 100, 1000, 10000', n_features='4, 20', dtype=None, verbose=1, fLOG=<built-in function print>, clean=True, flat=False, conf_params=None, build=None, add_pyspy=False, env=None, matrix=None)#

Creates an asv benchmark in a folder but does not run it.

Parameters:

location – location of the benchmark
n_features – number of features to try
dims – number of observations to try
verbose – integer from 0 (None) to 2 (full verbose)
opset_min – tries every conversion from this minimum opset, -1 to get the current opset defined by module onnx
opset_max – tries every conversion up to maximum opset, -1 to get the current opset defined by module onnx
runtime – runtime to check, scikit-learn, python, python_compiled compiles the graph structure and is more efficient when the number of observations is small, onnxruntime1 to check onnxruntime, onnxruntime2 to check every ONNX node independently with onnxruntime, many runtime can be checked at the same time if the value is a comma separated list
models – list of models to test or empty string to test them all
skip_models – models to skip
extended_list – extends the list of scikit-learn converters with converters implemented in this module
dtype – ‘32’ or ‘64’ or None for both, limits the test to one specific number types
fLOG – logging function
clean – clean the folder first, otherwise overwrites the content
conf_params – to overwrite some of the configuration parameters, format name,value;name2,value2
flat – one folder for all files or subfolders
build – location of the outputs (env, html, results)
add_pyspy – add an extra folder with code to profile each configuration
env – default environment or same to use the current one
matrix – specifies versions for a module as a json string, example: {'onnxruntime': ['1.1.1', '1.1.2']}, if a package name starts with ‘~’, the package is removed

Returns:

created files

Automatically creates an asv benchmark

The command creates a benchmark based on asv module. It does not run it.

Example:

python -m mlprodict asv_bench --models LogisticRegression,LinearRegression

<<<

python -m mlprodict asv_bench --help

>>>

usage: asv_bench [-h] [-l LOCATION] [-o OPSET_MIN] [-op OPSET_MAX]
                 [-r RUNTIME] [-m MODELS] [-s SKIP_MODELS] [-e EXTENDED_LIST]
                 [-d DIMS] [-n N_FEATURES] [-dt DTYPE] [-v VERBOSE] [-c CLEAN]
                 [-f FLAT] [-co CONF_PARAMS] [-b BUILD] [-a ADD_PYSPY]
                 [--env ENV] [-ma MATRIX]

Creates an `asv` benchmark in a folder but does not run it.

optional arguments:
  -h, --help            show this help message and exit
  -l LOCATION, --location LOCATION
                        location of the benchmark (default: asvsklonnx)
  -o OPSET_MIN, --opset_min OPSET_MIN
                        tries every conversion from this minimum opset, `-1`
                        to get the current opset defined by module onnx
                        (default: -1)
  -op OPSET_MAX, --opset_max OPSET_MAX
                        tries every conversion up to maximum opset, `-1` to
                        get the current opset defined by module onnx (default:
                        )
  -r RUNTIME, --runtime RUNTIME
                        runtime to check, *scikit-learn*, *python*,
                        *python_compiled* compiles the graph structure and is
                        more efficient when the number of observations is
                        small, *onnxruntime1* to check `onnxruntime`,
                        *onnxruntime2* to check every ONNX node independently
                        with onnxruntime, many runtime can be checked at the
                        same time if the value is a comma separated list
                        (default: scikit-learn,python_compiled)
  -m MODELS, --models MODELS
                        list of models to test or empty string to test them
                        all (default: )
  -s SKIP_MODELS, --skip_models SKIP_MODELS
                        models to skip (default: )
  -e EXTENDED_LIST, --extended_list EXTENDED_LIST
                        extends the list of :epkg:`scikit-learn` converters
                        with converters implemented in this module (default:
                        True)
  -d DIMS, --dims DIMS  number of observations to try (default:
                        1,10,100,1000,10000)
  -n N_FEATURES, --n_features N_FEATURES
                        number of features to try (default: 4,20)
  -dt DTYPE, --dtype DTYPE
                        '32' or '64' or None for both, limits the test to one
                        specific number types (default: )
  -v VERBOSE, --verbose VERBOSE
                        integer from 0 (None) to 2 (full verbose) (default: 1)
  -c CLEAN, --clean CLEAN
                        clean the folder first, otherwise overwrites the
                        content (default: True)
  -f FLAT, --flat FLAT  one folder for all files or subfolders (default:
                        False)
  -co CONF_PARAMS, --conf_params CONF_PARAMS
                        to overwrite some of the configuration parameters,
                        format ``name,value;name2,value2`` (default: )
  -b BUILD, --build BUILD
                        location of the outputs (env, html, results) (default:
                        )
  -a ADD_PYSPY, --add_pyspy ADD_PYSPY
                        add an extra folder with code to profile each
                        configuration (default: False)
  --env ENV             default environment or ``same`` to use the current one
                        (default: )
  -ma MATRIX, --matrix MATRIX
                        specifies versions for a module as a json string,
                        example: ``{'onnxruntime': ['1.1.1', '1.1.2']}``, if a
                        package name starts with `'~'`, the package is removed
                        (default: )

source on GitHub

module cli.asv_bench#

Short summary#

Functions#

Documentation#

module `cli.asv_bench`#