Validation

Benchmark

mlprodict.onnxrt.validate.validate_difference.measure_relative_difference (skl_pred, ort_pred, batch = True, abs_diff = False)

Measures the relative difference between predictions between two ways of computing them. The functions returns nan if shapes are different.

mlprodict.onnxrt.validate.enumerate_benchmark_replay (folder, runtime = ‘python’, time_kwargs = None, skip_long_test = True, time_kwargs_fact = None, time_limit = 4, verbose = 1, fLOG = None)

Replays a benchmark stored with function enumerate_validated_operator_opsets or command line validate_runtime. Enumerates the results.

mlprodict.onnxrt.validate.enumerate_validated_operator_opsets (verbose = 0, opset_min = -1, opset_max = -1, check_runtime = True, debug = False, runtime = ‘python’, models = None, dump_folder = None, store_models = False, benchmark = False, skip_models = None, assume_finite = True, node_time = False, fLOG = <built-in function print>, filter_exp = None, versions = False, extended_list = False, time_kwargs = None, dump_all = False, n_features = None, skip_long_test = True, fail_bad_results = False, filter_scenario = None, time_kwargs_fact = None, time_limit = 4, n_jobs = None)

Tests all possible configurations for all possible operators and returns the results.

mlprodict.onnxrt.validate.validate_helper.measure_time (stmt, x, repeat = 10, number = 50, div_by_number = False, first_run = True, max_time = None)

Measures a statement and returns the results as a dictionary.

mlprodict.onnxrt.validate.sklearn_operators (subfolder = None, extended = False, experimental = True)

Builds the list of operators from scikit-learn. The function goes through the list of submodule and get the list of class which inherit from :epkg:`scikit-learn:base:BaseEstimator`.

mlprodict.onnxrt.validate.summary_report (df, add_cols = None, add_index = None)

Finalizes the results computed by function enumerate_validated_operator_opsets.

mlprodict.onnxrt.validate.validate.latency