.. _l-onnx-doccom.microsoft-GridSample:

==========================
com.microsoft - GridSample
==========================

.. contents::
    :local:


.. _l-onnx-opcom-microsoft-gridsample-1:

GridSample - 1 (com.microsoft)
==============================

**Version**

* **name**: `GridSample (GitHub) <https://github.com/onnx/onnx/blob/main/docs/Operators.md#com.microsoft.GridSample>`_
* **domain**: **com.microsoft**
* **since_version**: **1**
* **function**:
* **support_level**:
* **shape inference**:

This version of the operator has been available
**since version 1 of domain com.microsoft**.

**Summary**

Given an `input` and a flow-field `grid`, computes the `output` using `input` values and pixel locations from `grid`.
Currently, only spatial (4-D) inputs are supported. For `input` with shape (N, C, H, W) and `grid` with shape (N, H_out, W_out, 2),
the `output` will have shape (N, C, H_out, W_out).
For each output location `output[n, :, h, w]`, the size-2 vector `grid[n, h, w]` specifies `input` pixel locations `x` and `y`,
which are used to interpolate the output value `output[n, :, h, w]`.
The GridSample operator is often used in doing grid generator and sampler in the [Spatial Transformer Networks](https://arxiv.org/abs/1506.02025).
See also in [torch.nn.functional.grid_sample](https://pytorch.org/docs/master/generated/torch.nn.functional.grid_sample.html#torch-nn-functional-grid-sample).

**Attributes**

* **align_corners**:
  If align_corners=1, the extrema (-1 and 1) are considered as
  referring to the center points of the input's corner pixels. If
  align_corners=0, they are instead considered as referring to the
  corner points of the input's corner pixels, making the sampling more
  resolution agnostic. Default value is ``?``.
* **mode**:
  Three interpolation modes: bilinear (default), nearest and bicubic. Default value is ``?``.
* **padding_mode**:
  Support padding modes for outside grid values: `zeros`(default),
  `border`, `reflection`. zeros: use 0 for out-of-bound grid
  locations, border: use border values for out-of-bound grid
  locations, reflection: use values at locations reflected by the
  border for out-of-bound grid locations. Default value is ``?``.

**Inputs**

* **X** (heterogeneous) - **T1**:
  4-D tensor of shape (N, C, H, W), where N is the batch size, C is
  the numbers of channels, H and W are the height and width of the
  input data.
* **Grid** (heterogeneous) - **T1**:
  Input offset, 4-D tensor of shape (N, H_out, W_out, 2), where H_out
  and W_out are the height and width of grid and output, Grid
  specifies the sampling pixel locations normalized by the input
  spatial dimensions. Therefore, it should have most values in the
  range of [-1, 1]. If grid has values outside the range of [-1, 1],
  the corresponding outputs will be handled as defined by
  padding_mode.

**Outputs**

* **Y** (heterogeneous) - **T2**:
  4-D tensor of shape (N, C, H_out, W_out).

**Examples**

**_gridsample**

::

    node = onnx.helper.make_node(
        "GridSample",
        inputs=["X", "Grid"],
        outputs=["Y"],
        mode="bilinear",
        padding_mode="zeros",
        align_corners=0,
    )
    # X shape, [N, C, H, W] - [1, 1, 4, 4]
    X = np.array(
        [
            [
                [
                    [0.0, 1.0, 2.0, 3.0],
                    [4.0, 5.0, 6.0, 7.0],
                    [8.0, 9.0, 10.0, 11.0],
                    [12.0, 13.0, 14.0, 15.0],
                ]
            ]
        ],
        dtype=np.float32,
    )
    # Grid shape, [N, H_out, W_out, 2] - [1, 6, 6, 2]
    Grid = np.array(
        [
            [
                [
                    [-1.0000, -1.0000],
                    [-0.6000, -1.0000],
                    [-0.2000, -1.0000],
                    [0.2000, -1.0000],
                    [0.6000, -1.0000],
                    [1.0000, -1.0000],
                ],
                [
                    [-1.0000, -0.6000],
                    [-0.6000, -0.6000],
                    [-0.2000, -0.6000],
                    [0.2000, -0.6000],
                    [0.6000, -0.6000],
                    [1.0000, -0.6000],
                ],
                [
                    [-1.0000, -0.2000],
                    [-0.6000, -0.2000],
                    [-0.2000, -0.2000],
                    [0.2000, -0.2000],
                    [0.6000, -0.2000],
                    [1.0000, -0.2000],
                ],
                [
                    [-1.0000, 0.2000],
                    [-0.6000, 0.2000],
                    [-0.2000, 0.2000],
                    [0.2000, 0.2000],
                    [0.6000, 0.2000],
                    [1.0000, 0.2000],
                ],
                [
                    [-1.0000, 0.6000],
                    [-0.6000, 0.6000],
                    [-0.2000, 0.6000],
                    [0.2000, 0.6000],
                    [0.6000, 0.6000],
                    [1.0000, 0.6000],
                ],
                [
                    [-1.0000, 1.0000],
                    [-0.6000, 1.0000],
                    [-0.2000, 1.0000],
                    [0.2000, 1.0000],
                    [0.6000, 1.0000],
                    [1.0000, 1.0000],
                ],
            ]
        ],
        dtype=np.float32,
    )
    # Y shape, [N, C, H_out, W_out] - [1, 1, 6, 6]
    Y = np.array(
        [
            [
                [
                    [0.0000, 0.1500, 0.5500, 0.9500, 1.3500, 0.7500],
                    [0.6000, 1.5000, 2.3000, 3.1000, 3.9000, 2.1000],
                    [2.2000, 4.7000, 5.5000, 6.3000, 7.1000, 3.7000],
                    [3.8000, 7.9000, 8.7000, 9.5000, 10.3000, 5.3000],
                    [5.4000, 11.1000, 11.9000, 12.7000, 13.5000, 6.9000],
                    [3.0000, 6.1500, 6.5500, 6.9500, 7.3500, 3.7500],
                ]
            ]
        ],
        dtype=np.float32,
    )
    expect(node, inputs=[X, Grid], outputs=[Y], name="test_gridsample")

**_gridsample_paddingmode**

::

    # X shape, [N, C, H, W] - [1, 1, 3, 2]
    X = np.array(
        [[[[0.0, 1.0], [2.0, 3.0], [4.0, 5.0]]]],
        dtype=np.float32,
    )
    # Grid shape, [N, H_out, W_out, 2] - [1, 2, 4, 2]
    Grid = np.array(
        [
            [
                [
                    [-10.0000, -10.0000],
                    [-5.0000, -5.0000],
                    [-0.2000, -0.2000],
                    [10.0000, 10.0000],
                ],
                [
                    [10.0000, 10.0000],
                    [-0.2000, -0.2000],
                    [5.0000, 5.0000],
                    [10.0000, 10.0000],
                ],
            ]
        ],
        dtype=np.float32,
    )

    # setting padding_mode = 'zeros'
    node = onnx.helper.make_node(
        "GridSample",
        inputs=["X", "Grid"],
        outputs=["Y"],
        padding_mode="zeros",
    )
    # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4]
    Y_zeros = np.array(
        [[[[0.0000, 0.0000, 1.7000, 0.0000], [0.0000, 1.7000, 0.0000, 0.0000]]]],
        dtype=np.float32,
    )

    expect(
        node,
        inputs=[X, Grid],
        outputs=[Y_zeros],
        name="test_gridsample_zeros_padding",
    )

    # setting padding_mode = 'border'
    node = onnx.helper.make_node(
        "GridSample",
        inputs=["X", "Grid"],
        outputs=["Y"],
        padding_mode="border",
    )
    # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4]
    Y_border = np.array(
        [[[[0.0000, 0.0000, 1.7000, 5.0000], [5.0000, 1.7000, 5.0000, 5.0000]]]],
        dtype=np.float32,
    )

    expect(
        node,
        inputs=[X, Grid],
        outputs=[Y_border],
        name="test_gridsample_border_padding",
    )

    # setting padding_mode = 'reflection'
    node = onnx.helper.make_node(
        "GridSample",
        inputs=["X", "Grid"],
        outputs=["Y"],
        padding_mode="reflection",
    )
    # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4]
    Y_reflection = np.array(
        [[[[2.5000, 0.0000, 1.7000, 2.5000], [2.5000, 1.7000, 5.0000, 2.5000]]]],
        dtype=np.float32,
    )

    expect(
        node,
        inputs=[X, Grid],
        outputs=[Y_reflection],
        name="test_gridsample_reflection_padding",
    )

**_gridsample_mode_aligncorners**

::

        # X shape, [N, C, H, W] - [1, 1, 3, 2]
        X = np.array(
            [[[[0.0, 1.0], [2.0, 3.0], [4.0, 5.0]]]],
            dtype=np.float32,
        )
        # Grid shape, [N, H_out, W_out, 2] - [1, 2, 4, 2]
        Grid = np.array(
            [
                [
                    [
                        [-1.0000, -1.0000],
                        [-0.5000, -0.5000],
                        [-0.2000, -0.2000],
                        [0.0000, 0.0000],
                    ],
                    [
                        [0.0000, 0.0000],
                        [-0.2000, -0.2000],
                        [0.5000, 0.5000],
                        [1.0000, 1.0000],
                    ],
                ]
            ],
            dtype=np.float32,
        )

        # setting mode = 'bilinear', default align_corners = 0
        node = onnx.helper.make_node(
            "GridSample",
            inputs=["X", "Grid"],
            outputs=["Y"],
            mode="bilinear",
        )
        # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4]
        Y_bilinear = np.array(
            [[[[0.0000, 0.5000, 1.7000, 2.5000], [2.5000, 1.7000, 4.5000, 1.2500]]]],
            dtype=np.float32,
        )

        expect(
            node,
            inputs=[X, Grid],
            outputs=[Y_bilinear],
            name="test_gridsample_bilinear",
        )

        # setting mode = 'bilinear', align_corners = 1
        node = onnx.helper.make_node(
            "GridSample",
            inputs=["X", "Grid"],
            outputs=["Y"],
            mode="bilinear",
            align_corners=1,
        )
        # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4]
        Y_align_corners = np.array(
            [[[[0.0000, 1.2500, 2.0000, 2.5000], [2.5000, 2.0000, 3.7500, 5.0000]]]],
            dtype=np.float32,
        )

        expect(
            node,
            inputs=[X, Grid],
            outputs=[Y_align_corners],
            name="test_gridsample_aligncorners_true",
        )

        # setting mode = 'nearest'
        node = onnx.helper.make_node(
            "GridSample",
            inputs=["X", "Grid"],
            outputs=["Y"],
            mode="nearest",
        )
        # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4]
        Y_nearest = np.array(
            [[[[0.0, 0.0, 2.0, 2.0], [2.0, 2.0, 5.0, 0.0]]]],
            dtype=np.float32,
        )

        expect(
            node, inputs=[X, Grid], outputs=[Y_nearest], name="test_gridsample_nearest"
        )

        # setting mode = 'bicubic'
        node = onnx.helper.make_node(
            "GridSample",
            inputs=["X", "Grid"],
            outputs=["Y"],
            mode="bicubic",
        )
        # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4]
        Y_bicubic = np.array(
            [[[[-0.1406, 0.3828, 1.7556, 2.9688], [2.9688, 1.7556, 5.1445, 1.3906]]]],
            dtype=np.float32,
        )

        expect(
            node, inputs=[X, Grid], outputs=[Y_bicubic], name="test_gridsample_bicubic"
        )

    """
    For someone who want to test by script. Comment it cause github ONNX CI
    do not have the torch python package.