Ubuntu-18.04
DEBUG=1 or 0
ONNX_USE_LITE_PROTO=OFF
ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
ONNX_BUILD_TESTS=1
ONNX_WERROR=ON
ONNX_ML=1 or 0

ONNX C++ tests
Style check (flake8, mypy, and clang-format)
Test doc generation
Test proto generation
Verify node test generation

| [Windows-CI](/.azure-pipelines/Windows-CI.yml) | Every PR |

windows-2019
ONNX_USE_LITE_PROTO=ON
ONNX_USE_PROTOBUF_SHARED_LIBS=ON
ONNX_BUILD_TESTS=1
ONNX_WERROR=ON
ONNX_ML=1 or 0

Test building ONNX in conda environment
Test doc generation
Test proto generation
Verify node test generation

| [Mac-CI](/.azure-pipelines/MacOS-CI.yml) | Every PR |

macOS-10.15
DEBUG=1
ONNX_USE_LITE_PROTO=ON or OFF
ONNX_ML=1 or 0
ONNX_BUILD_TESTS=1
ONNX_WERROR=ON

ONNX C++ tests
Test doc generation
Test proto generation
Verify node test generation

| [Windows_No_Exception CI](/.github/workflows/win_no_exception_ci.yml) | Every PR |

vs2019-winlatest
ONNX_DISABLE_EXCEPTIONS=ON
ONNX_USE_LITE_PROTO=ON
ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
ONNX_ML=1
ONNX_USE_MSVC_STATIC_RUNTIME=ON
ONNX_DISABLE_STATIC_REGISTRATION=ON or OFF

Only ONNX C++ tests
Test selective schema loading

| [Lint / Lint Python](/.github/workflows/lint.yaml) | Every PR |

ubuntu-latest|

Not required -- it shows lint warnings for suggestions in PR
flake8 (but required by `style.sh` in another CI)
pyflakes
misspell
shellcheck
pylint
mypy (but required by `style.sh` in another CI)

| [Lint / Enforce style](/.github/workflows/lint.yaml) | Every PR |

ubuntu-latest|

flake8
isort
black
mypy
clang-format
unix line endings
c++ namespace rules
Auto-generated files are up to date

| [WindowsRelease](/.github/workflows/release_win.yml) |

Main branch
Release branch
Weekly(1)

Latest Windows
x86 and x64
ONNX_USE_LITE_PROTO=ON
ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
ONNX_ML=1
ONNX_USE_MSVC_STATIC_RUNTIME=OFF

Release Windows wheel
Release onnx-weekly package
Verify with different dependency versions - latest and min supported numpy version, latest and min supported protobuf version(2)
Verify ONNX with the latest [ONNX Runtime PyPI package](https://pypi.org/project/onnxruntime/)(3).

| [LinuxRelease_aarch64](/.github/workflows/release_linux_aarch64.yml) |

Main branch
Release branch
Weekly

Latest manylinux2014_aarch64
ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
ONNX_ML=1
ONNX_USE_LITE_PROTO=ON

Release Linux aarch64 wheel
Release onnx-weekly package
Verify with different dependency versions - latest numpy version, latest and min supported protobuf version
Verify ONNX with the latest ONNX Runtime PyPI package

| [LinuxRelease_x86_64](/.github/workflows/release_linux_x86_64.yml) |

Main branch
Release branch
Weekly

Latest LinuxRelease_x86_64
ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
ONNX_ML=1
ONNX_USE_LITE_PROTO=ON

Release Linux x86_64 wheel
Release onnx-weekly package
Test TEST_HUB=1(4)
Verify with different dependency versions - latest numpy version, latest and min supported protobuf version
Verify ONNX with the latest ONNX Runtime PyPI package.

| [MacRelease](/.github/workflows/release_win.yml) |

Main branch
Release branch
Weekly

macos-11
MACOSX_DEPLOYMENT_TARGET=10.12(5)
ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
ONNX_ML=1
ONNX_USE_LITE_PROTO=ON

Release Mac wheel
Release onnx-weekly package
Verify with different dependency versions - latest numpy version, latest and min supported protobuf version
Verify ONNX with the latest ONNX Runtime PyPI package.
Test source distribution generation
Test build with source distribution
Release onnx-weekly source distribution

[Weekly CI with latest onnx.checker](/.github/workflows/weekly_mac_ci.yml) | weekly(6) |

macos-latest
MACOSX_DEPLOYMENT_TARGET=10.12
ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
ONNX_ML=1

Test latest ONNX checker
Test latest ONNX shape inference
With all models from [onnx/models](https://github.com/onnx/models)(7)

| * (1) When the release CIs will run: * After a PR has been merged into main/rel-* branch * Run weekly (Sunday midnight) and release Python wheel to [onnx-weekly](https://test.pypi.org/project/onnx-weekly/) package on TestPyPI. * Any PR targeting rel-* branch * To manually run them, add a PR label "run release CIs" (only maintainers have permission). * (2) Minimum supported versions are listed [here](/requirements.txt). * (3) [Test](/onnx/test/test_with_ort.py) ONNX Python wheel with `onnxruntime.InferenceSession` from latest ONNXRuntime. Please note that ONNX Runtime does not support Windows-x86 thus its verification is skipped. * (4) TEST_HUB=1 will test [onnx.hub](/onnx/test/hub_test.py) by using this API to download an ONNX model from onnx/models. This test is restricted to only 1 pipeline for saving quota usage. * (5) Although the build environment is macos-11, use MACOSX_DEPLOYMENT_TARGET=10.12 and -p [macosx_10_12_x86_64](https://github.com/onnx/onnx/blob/2e048660ffa8243596aaf3338e60c7c0575458f2/.github/workflows/release_mac.yml#L74) to force the wheel to support 10.12+. * (6): * The ONNX Model Zoo test will run weekly (Sunday midnight) * To manually trigger it, add a PR label "test ONNX Model Zoo" (only maintainers have permission). Please note that it will need a lot of download bandwidth from [onnx/models](https://github.com/onnx/models) so use it with caution. * (7) Some old deprecated models (opset-1) are [skipped](/workflow_scripts/config.py).