.. _l-onnx-doc-GridSample: ========== GridSample ========== .. contents:: :local: .. _l-onnx-op-gridsample-16: GridSample - 16 =============== **Version** * **name**: `GridSample (GitHub) `_ * **domain**: **main** * **since_version**: **16** * **function**: False * **support_level**: SupportType.COMMON * **shape inference**: True This version of the operator has been available **since version 16**. **Summary** Given an input `X` and a flow-field `grid`, computes the output `Y` using `X` values and pixel locations from `grid`. Currently, only spatial (4-D) inputs are supported. For input `X` with shape (N, C, H, W) and `grid` with shape (N, H_out, W_out, 2), the output `Y` will have shape (N, C, H_out, W_out). The tensor `X` contains values at centers of square pixels in a H by W 2-dimensional image. The tensor `grid` describes normalized positions where the output `Y` is to be computed using a specified interpolation method (the mode) and a padding mode (for grid positions falling outside the 2-dimensional image). Elements in `grid[N, H_out, W_out]` are size-2 vectors specifying positions in the 2-dimensional space of `X`. They are used to interpolate output values of `Y[N, C, H_out, W_out]`. The GridSample operator is often used in doing grid generator and sampler in the [Spatial Transformer Networks](https://arxiv.org/abs/1506.02025). See also in [torch.nn.functional.grid_sample](https://pytorch.org/docs/master/generated/torch.nn.functional.grid_sample.html#torch-nn-functional-grid-sample). **Attributes** * **align_corners**: If align_corners=1, the extrema (-1 and 1) are considered as referring to the center points of the input's corner pixels. If align_corners=0, they are instead considered as referring to the corner points of the input's corner pixels, making the sampling more resolution agnostic. Default value is ``0``. * **mode**: Three interpolation modes: bilinear (default), nearest and bicubic. Default value is ``'bilinear'``. * **padding_mode**: Support padding modes for outside grid values: `zeros`(default), `border`, `reflection`. zeros: use 0 for out-of-bound grid locations, border: use border values for out-of-bound grid locations, reflection: use values at locations reflected by the border for out-of-bound grid locations. If index 0 represents the margin pixel, the reflected value at index -1 will be the same as the value at index 1. For location far away from the border, it will keep being reflected until becoming in bound. If pixel location x = -3.5 reflects by border -1 and becomes x' = 1.5, then reflects by border 1 and becomes x'' = 0.5. Default value is ``'zeros'``. **Inputs** * **X** (heterogeneous) - **T1**: 4-D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the input data. * **grid** (heterogeneous) - **T2**: Input offset, 4-D tensor of shape (N, H_out, W_out, 2), where H_out and W_out are the height and width of grid and output, Grid specifies the sampling pixel locations normalized by the input spatial dimensions. Therefore, it should have most values in the range of [-1, 1]. If grid has values outside the range of [-1, 1], the corresponding outputs will be handled as defined by padding_mode. **Outputs** * **Y** (heterogeneous) - **T1**: 4-D tensor of shape (N, C, H_out, W_out) of sampled values. For integer input types, intermediate values are computed as floating point and cast to integer at the end. **Type Constraints** * **T1** in ( tensor(bool), tensor(complex128), tensor(complex64), tensor(double), tensor(float), tensor(float16), tensor(int16), tensor(int32), tensor(int64), tensor(int8), tensor(string), tensor(uint16), tensor(uint32), tensor(uint64), tensor(uint8) ): Constrain input `X` and output `Y` types to all tensor types. * **T2** in ( tensor(double), tensor(float), tensor(float16) ): Constrain grid types to float tensors. **Examples** **_gridsample** :: node = onnx.helper.make_node( "GridSample", inputs=["X", "Grid"], outputs=["Y"], mode="bilinear", padding_mode="zeros", align_corners=0, ) # X shape, [N, C, H, W] - [1, 1, 4, 4] X = np.array( [ [ [ [0.0, 1.0, 2.0, 3.0], [4.0, 5.0, 6.0, 7.0], [8.0, 9.0, 10.0, 11.0], [12.0, 13.0, 14.0, 15.0], ] ] ], dtype=np.float32, ) # Grid shape, [N, H_out, W_out, 2] - [1, 6, 6, 2] Grid = np.array( [ [ [ [-1.0000, -1.0000], [-0.6000, -1.0000], [-0.2000, -1.0000], [0.2000, -1.0000], [0.6000, -1.0000], [1.0000, -1.0000], ], [ [-1.0000, -0.6000], [-0.6000, -0.6000], [-0.2000, -0.6000], [0.2000, -0.6000], [0.6000, -0.6000], [1.0000, -0.6000], ], [ [-1.0000, -0.2000], [-0.6000, -0.2000], [-0.2000, -0.2000], [0.2000, -0.2000], [0.6000, -0.2000], [1.0000, -0.2000], ], [ [-1.0000, 0.2000], [-0.6000, 0.2000], [-0.2000, 0.2000], [0.2000, 0.2000], [0.6000, 0.2000], [1.0000, 0.2000], ], [ [-1.0000, 0.6000], [-0.6000, 0.6000], [-0.2000, 0.6000], [0.2000, 0.6000], [0.6000, 0.6000], [1.0000, 0.6000], ], [ [-1.0000, 1.0000], [-0.6000, 1.0000], [-0.2000, 1.0000], [0.2000, 1.0000], [0.6000, 1.0000], [1.0000, 1.0000], ], ] ], dtype=np.float32, ) # Y shape, [N, C, H_out, W_out] - [1, 1, 6, 6] Y = np.array( [ [ [ [0.0000, 0.1500, 0.5500, 0.9500, 1.3500, 0.7500], [0.6000, 1.5000, 2.3000, 3.1000, 3.9000, 2.1000], [2.2000, 4.7000, 5.5000, 6.3000, 7.1000, 3.7000], [3.8000, 7.9000, 8.7000, 9.5000, 10.3000, 5.3000], [5.4000, 11.1000, 11.9000, 12.7000, 13.5000, 6.9000], [3.0000, 6.1500, 6.5500, 6.9500, 7.3500, 3.7500], ] ] ], dtype=np.float32, ) expect(node, inputs=[X, Grid], outputs=[Y], name="test_gridsample") **_gridsample_paddingmode** :: # X shape, [N, C, H, W] - [1, 1, 3, 2] X = np.array( [[[[0.0, 1.0], [2.0, 3.0], [4.0, 5.0]]]], dtype=np.float32, ) # Grid shape, [N, H_out, W_out, 2] - [1, 2, 4, 2] Grid = np.array( [ [ [ [-10.0000, -10.0000], [-5.0000, -5.0000], [-0.2000, -0.2000], [10.0000, 10.0000], ], [ [10.0000, 10.0000], [-0.2000, -0.2000], [5.0000, 5.0000], [10.0000, 10.0000], ], ] ], dtype=np.float32, ) # setting padding_mode = 'zeros' node = onnx.helper.make_node( "GridSample", inputs=["X", "Grid"], outputs=["Y"], padding_mode="zeros", ) # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4] Y_zeros = np.array( [[[[0.0000, 0.0000, 1.7000, 0.0000], [0.0000, 1.7000, 0.0000, 0.0000]]]], dtype=np.float32, ) expect( node, inputs=[X, Grid], outputs=[Y_zeros], name="test_gridsample_zeros_padding", ) # setting padding_mode = 'border' node = onnx.helper.make_node( "GridSample", inputs=["X", "Grid"], outputs=["Y"], padding_mode="border", ) # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4] Y_border = np.array( [[[[0.0000, 0.0000, 1.7000, 5.0000], [5.0000, 1.7000, 5.0000, 5.0000]]]], dtype=np.float32, ) expect( node, inputs=[X, Grid], outputs=[Y_border], name="test_gridsample_border_padding", ) # setting padding_mode = 'reflection' node = onnx.helper.make_node( "GridSample", inputs=["X", "Grid"], outputs=["Y"], padding_mode="reflection", ) # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4] Y_reflection = np.array( [[[[2.5000, 0.0000, 1.7000, 2.5000], [2.5000, 1.7000, 5.0000, 2.5000]]]], dtype=np.float32, ) expect( node, inputs=[X, Grid], outputs=[Y_reflection], name="test_gridsample_reflection_padding", ) **_gridsample_mode_aligncorners** :: # X shape, [N, C, H, W] - [1, 1, 3, 2] X = np.array( [[[[0.0, 1.0], [2.0, 3.0], [4.0, 5.0]]]], dtype=np.float32, ) # Grid shape, [N, H_out, W_out, 2] - [1, 2, 4, 2] Grid = np.array( [ [ [ [-1.0000, -1.0000], [-0.5000, -0.5000], [-0.2000, -0.2000], [0.0000, 0.0000], ], [ [0.0000, 0.0000], [-0.2000, -0.2000], [0.5000, 0.5000], [1.0000, 1.0000], ], ] ], dtype=np.float32, ) # setting mode = 'bilinear', default align_corners = 0 node = onnx.helper.make_node( "GridSample", inputs=["X", "Grid"], outputs=["Y"], mode="bilinear", ) # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4] Y_bilinear = np.array( [[[[0.0000, 0.5000, 1.7000, 2.5000], [2.5000, 1.7000, 4.5000, 1.2500]]]], dtype=np.float32, ) expect( node, inputs=[X, Grid], outputs=[Y_bilinear], name="test_gridsample_bilinear", ) # setting mode = 'bilinear', align_corners = 1 node = onnx.helper.make_node( "GridSample", inputs=["X", "Grid"], outputs=["Y"], mode="bilinear", align_corners=1, ) # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4] Y_align_corners = np.array( [[[[0.0000, 1.2500, 2.0000, 2.5000], [2.5000, 2.0000, 3.7500, 5.0000]]]], dtype=np.float32, ) expect( node, inputs=[X, Grid], outputs=[Y_align_corners], name="test_gridsample_aligncorners_true", ) # setting mode = 'nearest' node = onnx.helper.make_node( "GridSample", inputs=["X", "Grid"], outputs=["Y"], mode="nearest", ) # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4] Y_nearest = np.array( [[[[0.0, 0.0, 2.0, 2.0], [2.0, 2.0, 5.0, 0.0]]]], dtype=np.float32, ) expect( node, inputs=[X, Grid], outputs=[Y_nearest], name="test_gridsample_nearest" ) # setting mode = 'bicubic' node = onnx.helper.make_node( "GridSample", inputs=["X", "Grid"], outputs=["Y"], mode="bicubic", ) # Y shape, [N, C, H_out, W_out] - [1, 1, 2, 4] Y_bicubic = np.array( [[[[-0.1406, 0.3828, 1.7556, 2.9688], [2.9688, 1.7556, 5.1445, 1.3906]]]], dtype=np.float32, ) expect( node, inputs=[X, Grid], outputs=[Y_bicubic], name="test_gridsample_bicubic" ) """ For someone who want to test by script. Comment it cause github ONNX CI do not have the torch python package.