mlprodict¶
Links: github, documentation, mlprodict, blog
mlprodict explores couple of ways to compute predictions faster than the library used to build the machine learned model, mostly scikit-learn which is optimized for training, which is equivalent to batch predictions. One way is to use ONNX. onnxruntime provides an efficient way to compute predictions. mlprodict implements a python/numpy runtime for ONNX which does not have any dependency on scikit-learn.
<<<
import numpy
from sklearn.linear_model import LinearRegression
from sklearn.datasets import load_iris
from mlprodict.onnxrt import OnnxInference
from mlprodict.onnxrt.validate.validate_difference import measure_relative_difference
from mlprodict.tools import get_ir_version_from_onnx
iris = load_iris()
X = iris.data[:, :2]
y = iris.target
lr = LinearRegression()
lr.fit(X, y)
# Predictions with scikit-learn.
expected = lr.predict(X[:5])
print(expected)
# Conversion into ONNX.
from mlprodict.onnx_conv import to_onnx
model_onnx = to_onnx(lr, X.astype(numpy.float32))
print("ONNX:", str(model_onnx)[:200] + "\n...")
# Predictions with onnxruntime
model_onnx.ir_version = get_ir_version_from_onnx()
oinf = OnnxInference(model_onnx, runtime='onnxruntime1')
ypred = oinf.run({'X': X[:5].astype(numpy.float32)})
print("ONNX output:", ypred)
# Measuring the maximum difference.
print("max abs diff:", measure_relative_difference(
expected, ypred['variable']))
>>>
[0.172 0.343 0.069 0.059 0.034]
ONNX: ir_version: 6
producer_name: "skl2onnx"
producer_version: "1.9.1101"
domain: "ai.onnx"
model_version: 0
doc_string: ""
graph {
node {
input: "X"
output: "variable"
name: "LinearRegressor
...
ONNX output: {'variable': array([[0.172],
[0.343],
[0.069],
[0.059],
[0.034]], dtype=float32)}
max abs diff: 6.303014714402957e-06
These predictions are obtained with the following ONNX graph.
Notebook ONNX visualization shows how to visualize an ONNX pipeline.
