.. _l-onnx-doc-NegativeLogLikelihoodLoss: ========================= NegativeLogLikelihoodLoss ========================= .. contents:: :local: .. _l-onnx-op-negativeloglikelihoodloss-13: NegativeLogLikelihoodLoss - 13 ============================== **Version** * **name**: `NegativeLogLikelihoodLoss (GitHub) `_ * **domain**: **main** * **since_version**: **13** * **function**: False * **support_level**: SupportType.COMMON * **shape inference**: True This version of the operator has been available **since version 13**. **Summary** A NegativeLogLikelihoodLoss operator computes (weighted) negative log likelihood loss. Its "input" tensor has the shape of (N, C, d1, d2, ..., dk) where k >= 0. The "input" tensor contains log-probabilities for input[n, :, d_1, d_2,..., d_k] being in a class of [0, C). The operator's "target" input tensor has the shape of (N, d1, d2, ..., dk). It encodes class labels (one of C classes) or it may contain a special value (indicated by an attribute ignore_index) for N x d1 x d2 x ... x dk samples. The loss value for input[n, :, d_1, d_2,...d_k] being classified as class c = target[n][d_1][d_2]...[d_k] is computed as: loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k]. When an optional "weight" is provided, the sample loss is calculated as: loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k] * weight[c]. loss is zero for the case when target-value equals ignore_index. loss[n][d_1][d_2]...[d_k] = 0, when target[n][d_1][d_2]...[d_k] = ignore_index If "reduction" attribute is set to "none", the operator's output will be the above loss with shape (N, d1, d2, ..., dk). If "reduction" attribute is set to "mean" (the default attribute value), the output loss is (weight) averaged: mean(loss), if "weight" is not provided, or if weight is provided, sum(loss) / sum(weight[target[n][d_1][d_2]...[d_k]]]), for all samples. If "reduction" attribute is set to "sum", the output is a scalar: sum(loss). See also https://pytorch.org/docs/stable/nn.html#torch.nn.NLLLoss. Example 1: // negative log likelihood loss, "none" reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] target = [[2, 1], [0, 2]] loss = np.zeros((N, d1)) for n in range(N): for d_1 in range(d1): c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1] // print(loss) // [[-3. -2.] // [-0. -2.]] Example 2: // weighted negative log likelihood loss, sum reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] target = [[2, 1], [0, 2]] weight = [0.2, 0.3, 0.1] loss = np.zeros((N, d1)) for n in range(N): for d_1 in range(d1): c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1] * weight[c] loss = np.sum(loss) // print(loss) // -1.1 Example 3: // weighted negative log likelihood loss, mean reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] target = [[2, 1], [0, 2]] weight = [0.2, 0.3, 0.1] loss = np.zeros((N, d1)) weight_total = 0 for n in range(N): for d_1 in range(d1): c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1] * weight[c] weight_total = weight_total + weight[c] loss = np.sum(loss) / weight_total // print(loss) // -1.57 **Attributes** * **ignore_index**: Specifies a target value that is ignored and does not contribute to the input gradient. It's an optional value. * **reduction**: Type of reduction to apply to loss: none, sum, mean (default). 'none': the output is the loss for each sample. 'sum': the output will be summed. 'mean': the sum of the output will be divided by the sum of applied weights. Default value is ``'mean'``. **Inputs** Between 2 and 3 inputs. * **input** (heterogeneous) - **T**: Input tensor of shape (N, C) or (N, C, d1, d2, ..., dk). * **target** (heterogeneous) - **Tind**: Target tensor of shape (N) or (N, d1, d2, ..., dk). Target element value shall be in range of [0, C). If ignore_index is specified, it may have a value outside [0, C) and the target values should either be in the range [0, C) or have the value ignore_index. * **weight** (optional, heterogeneous) - **T**: Optional rescaling weight tensor. If given, it has to be a tensor of size C. Otherwise, it is treated as if having all ones. **Outputs** * **loss** (heterogeneous) - **T**: The negative log likelihood loss **Type Constraints** * **T** in ( tensor(double), tensor(float), tensor(float16) ): Constrain input, weight, and output types to floating-point tensors. * **Tind** in ( tensor(int32), tensor(int64) ): Constrain target to integer types **Examples** **_input_shape_is_NC** :: reduction = 'none' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target'], outputs=['loss'], reduction=reduction ) N, C = 3, 5 np.random.seed(0) input = np.random.rand(N, C).astype(np.float32) target = np.random.randint(0, high=C, size=(N, )).astype(np.int64) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction) expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss], name='test_nllloss_NC') **_input_shape_is_NCd1d2** :: reduction = 'none' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target'], outputs=['loss'], reduction=reduction ) N, C, dim1, dim2 = 3, 5, 6, 6 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction) expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2') **_input_shape_is_NCd1d2_reduction_mean** :: reduction = 'mean' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target'], outputs=['loss'], reduction=reduction ) N, C, dim1, dim2 = 3, 5, 6, 6 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction) expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2_reduction_mean') **_input_shape_is_NCd1d2_reduction_sum** :: reduction = 'sum' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target'], outputs=['loss'], reduction=reduction ) N, C, dim1, dim2 = 3, 5, 6, 6 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction) expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2_reduction_sum') **_input_shape_is_NCd1d2_with_weight** :: reduction = 'none' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target', 'weight'], outputs=['loss'], reduction=reduction ) N, C, dim1, dim2 = 3, 5, 6, 6 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64) weight = np.random.rand(C).astype(np.float32) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction) expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2_with_weight') **_input_shape_is_NCd1d2_with_weight_reduction_mean** :: reduction = 'mean' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target', 'weight'], outputs=['loss'], reduction=reduction ) N, C, dim1, dim2 = 3, 5, 6, 6 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64) weight = np.random.rand(C).astype(np.float32) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction) expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2_with_weight_reduction_mean') **_input_shape_is_NCd1d2_with_weight_reduction_sum** :: reduction = 'sum' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target', 'weight'], outputs=['loss'], reduction=reduction ) N, C, dim1, dim2 = 3, 5, 6, 6 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64) weight = np.random.rand(C).astype(np.float32) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction) expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2_with_weight_reduction_sum') **_input_shape_is_NCd1d2_with_weight_reduction_sum_ii** :: reduction = 'sum' ignore_index = np.int64(0) node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target', 'weight'], outputs=['loss'], reduction=reduction, ignore_index=ignore_index ) N, C, dim1, dim2 = 3, 5, 6, 6 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64) target[0][0][0] = np.int64(0) weight = np.random.rand(C).astype(np.float32) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction, ignore_index=ignore_index) expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2_with_weight_reduction_sum_ii') **_input_shape_is_NCd1d2_no_weight_reduction_mean_ii** :: reduction = 'mean' ignore_index = np.int64(1) node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target'], outputs=['loss'], reduction=reduction, ignore_index=ignore_index ) N, C, dim1, dim2 = 3, 5, 6, 6 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2)).astype(np.int64) target[0][0][0] = np.int64(1) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, reduction=reduction, ignore_index=ignore_index) expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2_no_weight_reduction_mean_ii') **_input_shape_is_NCd1** :: reduction = 'mean' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target'], outputs=['loss'], reduction=reduction ) N, C, d1 = 3, 5, 2 np.random.seed(0) input = np.random.rand(N, C, d1).astype(np.float32) target = np.random.randint(0, high=C, size=(N, d1)).astype(np.int64) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction) expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1') **_input_shape_is_NCd1_weight** :: reduction = 'mean' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target', 'weight'], outputs=['loss'], reduction=reduction ) N, C, d1 = 3, 5, 2 np.random.seed(0) input = np.random.rand(N, C, d1).astype(np.float32) target = np.random.randint(0, high=C, size=(N, d1)).astype(np.int64) weight = np.random.rand(C).astype(np.float32) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction) expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1_weight') **_input_shape_is_NCd1_ii** :: reduction = 'mean' ignore_index = np.int64(1) node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target'], outputs=['loss'], reduction=reduction, ignore_index=ignore_index ) N, C, d1 = 3, 5, 2 np.random.seed(0) input = np.random.rand(N, C, d1).astype(np.float32) target = np.random.randint(0, high=C, size=(N, d1)).astype(np.int64) target[0][0] = np.int64(1) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=None, reduction=reduction, ignore_index=ignore_index) expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1_ii') **_input_shape_is_NCd1_weight_ii** :: reduction = 'mean' ignore_index = np.int64(1) node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target', 'weight'], outputs=['loss'], reduction=reduction, ignore_index=ignore_index ) N, C, d1 = 3, 5, 2 np.random.seed(0) input = np.random.rand(N, C, d1).astype(np.float32) target = np.random.randint(0, high=C, size=(N, d1)).astype(np.int64) target[0][0] = np.int64(1) weight = np.random.rand(C).astype(np.float32) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction, ignore_index=ignore_index) expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1_weight_ii') **_input_shape_is_NCd1d2d3d4d5_mean_weight** :: reduction = 'mean' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target', 'weight'], outputs=['loss'], reduction=reduction) N, C, dim1, dim2, dim3, dim4, dim5 = 3, 5, 6, 6, 5, 3, 4 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2, dim3, dim4, dim5).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2, dim3, dim4, dim5)).astype(np.int64) weight = np.random.rand(C).astype(np.float32) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction) expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2d3d4d5_mean_weight') **_input_shape_is_NCd1d2d3d4d5_none_no_weight** :: reduction = 'none' node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target'], outputs=['loss'], reduction=reduction) N, C, dim1, dim2, dim3, dim4, dim5 = 3, 5, 6, 6, 5, 3, 4 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2, dim3, dim4, dim5).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2, dim3, dim4, dim5)).astype(np.int64) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, reduction=reduction) expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2d3d4d5_none_no_weight') **_input_shape_is_NCd1_mean_weight_negative_ii** :: reduction = 'mean' ignore_index = np.int64(-1) node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target', 'weight'], outputs=['loss'], reduction=reduction, ignore_index=ignore_index) N, C, dim1 = 3, 5, 6 np.random.seed(0) input = np.random.rand(N, C, dim1).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1)).astype(np.int64) target[0][0] = -1 weight = np.random.rand(C).astype(np.float32) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction, ignore_index=ignore_index) expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1_mean_weight_negative_ii') **_input_shape_is_NCd1d2d3_none_no_weight_negative_ii** :: reduction = 'none' ignore_index = np.int64(-5) node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target'], outputs=['loss'], reduction=reduction, ignore_index=ignore_index) N, C, dim1, dim2, dim3 = 3, 5, 6, 6, 5 np.random.seed(0) input = np.random.rand(N, C, dim1, dim2, dim3).astype(np.float32) target = np.random.randint(0, high=C, size=(N, dim1, dim2, dim3)).astype(np.int64) target[0][0][0][0] = -5 negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, reduction=reduction, ignore_index=ignore_index) expect(node, inputs=[input, target], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2d3_none_no_weight_negative_ii') **_input_shape_is_NCd1d2d3_sum_weight_high_ii** :: reduction = 'sum' ignore_index = np.int64(10) node = onnx.helper.make_node( 'NegativeLogLikelihoodLoss', inputs=['input', 'target', 'weight'], outputs=['loss'], reduction=reduction, ignore_index=ignore_index) N, C = 3, 5 np.random.seed(0) input = np.random.rand(N, C).astype(np.float32) target = np.random.randint(0, high=C, size=(N)).astype(np.int64) target[0] = 10 weight = np.random.rand(C).astype(np.float32) negative_log_likelihood_loss = compute_negative_log_likelihood_loss(input, target, weight=weight, reduction=reduction, ignore_index=ignore_index) expect(node, inputs=[input, target, weight], outputs=[negative_log_likelihood_loss], name='test_nllloss_NCd1d2d3_sum_weight_high_ii') **Differences** .. raw:: html
 `0` `0` `A NegativeLogLikelihoodLoss operator computes (weighted) negative log likelihood loss.` `A NegativeLogLikelihoodLoss operator computes (weighted) negative log likelihood loss.` `1` `1` `Its "input" tensor has the shape of (N, C, d1, d2, ..., dk) where k >= 0.` `Its "input" tensor has the shape of (N, C, d1, d2, ..., dk) where k >= 0.` `2` `2` `The "input" tensor contains log-probabilities for input[n, :, d_1, d_2,..., d_k] being in a class of [0, C).` `The "input" tensor contains log-probabilities for input[n, :, d_1, d_2,..., d_k] being in a class of [0, C).` `3` `3` `The operator's "target" input tensor has the shape of (N, d1, d2, ..., dk). It encodes class labels (one of C classes)` `The operator's "target" input tensor has the shape of (N, d1, d2, ..., dk). It encodes class labels (one of C classes)` `4` `4` `or it may contain a special value (indicated by an attribute ignore_index) for N x d1 x d2 x ... x dk samples.` `or it may contain a special value (indicated by an attribute ignore_index) for N x d1 x d2 x ... x dk samples.` `5` `5` `The loss value for input[n, :, d_1, d_2,...d_k] being classified as class c = target[n][d_1][d_2]...[d_k] is computed as:` `The loss value for input[n, :, d_1, d_2,...d_k] being classified as class c = target[n][d_1][d_2]...[d_k] is computed as:` `6` `6` `7` ` loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k].` ` loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k].` `8` `7` `9` `When an optional "weight" is provided, the sample loss is calculated as:` `When an optional "weight" is provided, the sample loss is calculated as:` `10` `8` `11` ` loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k] * weight[c].` ` loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k] * weight[c].` `12` `9` `13` `loss is zero for the case when target-value equals ignore_index.` `loss is zero for the case when target-value equals ignore_index.` `10` `14` `11` `15` ` loss[n][d_1][d_2]...[d_k] = 0, when target[n][d_1][d_2]...[d_k] = ignore_index` ` loss[n][d_1][d_2]...[d_k] = 0, when target[n][d_1][d_2]...[d_k] = ignore_index` `16` `12` `17` `If "reduction" attribute is set to "none", the operator's output will be the above loss with shape (N, d1, d2, ..., dk).` `If "reduction" attribute is set to "none", the operator's output will be the above loss with shape (N, d1, d2, ..., dk).` `13` `18` `If "reduction" attribute is set to "mean" (the default attribute value), the output loss is (weight) averaged:` `If "reduction" attribute is set to "mean" (the default attribute value), the output loss is (weight) averaged:` `19` `14` `20` ` mean(loss), if "weight" is not provided,` ` mean(loss), if "weight" is not provided,` `21` `15` `22` `or if weight is provided,` `or if weight is provided,` `23` `16` `24` ` sum(loss) / sum(weight[target[n][d_1][d_2]...[d_k]]]), for all samples.` ` sum(loss) / sum(weight[target[n][d_1][d_2]...[d_k]]]), for all samples.` `25` `17` `26` `If "reduction" attribute is set to "sum", the output is a scalar:` `If "reduction" attribute is set to "sum", the output is a scalar:` `18` `27` ` sum(loss).` ` sum(loss).` `28` `19` `29` `See also https://pytorch.org/docs/stable/nn.html#torch.nn.NLLLoss.` `See also https://pytorch.org/docs/stable/nn.html#torch.nn.NLLLoss.` `30` `20` `31` `Example 1:` `Example 1:` `32` `21` `33` ` // negative log likelihood loss, "none" reduction` ` // negative log likelihood loss, "none" reduction` `22` `34` ` N, C, d1 = 2, 3, 2` ` N, C, d1 = 2, 3, 2` `23` `35` ` input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],` ` input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],` `24` `36` ` [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]` ` [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]` `25` `37` ` target = [[2, 1], [0, 2]]` ` target = [[2, 1], [0, 2]]` `38` `26` `39` ` loss = np.zeros((N, d1))` ` loss = np.zeros((N, d1))` `27` `40` ` for n in range(N):` ` for n in range(N):` `28` `41` ` for d_1 in range(d1):` ` for d_1 in range(d1):` `29` `42` ` c = target[n][d_1]` ` c = target[n][d_1]` `30` `43` ` loss[n][d_1] = -input[n][c][d_1]` ` loss[n][d_1] = -input[n][c][d_1]` `44` `31` `45` ` // print(loss)` ` // print(loss)` `32` `46` ` // [[-3. -2.]` ` // [[-3. -2.]` `33` `47` ` // [-0. -2.]]` ` // [-0. -2.]]` `48` `34` `49` `Example 2:` `Example 2:` `50` `35` `51` ` // weighted negative log likelihood loss, sum reduction` ` // weighted negative log likelihood loss, sum reduction` `36` `52` ` N, C, d1 = 2, 3, 2` ` N, C, d1 = 2, 3, 2` `37` `53` ` input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],` ` input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],` `38` `54` ` [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]` ` [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]` `39` `55` ` target = [[2, 1], [0, 2]]` ` target = [[2, 1], [0, 2]]` `40` `56` ` weight = [0.2, 0.3, 0.1]` ` weight = [0.2, 0.3, 0.1]` `41` `57` ` loss = np.zeros((N, d1))` ` loss = np.zeros((N, d1))` `42` `58` ` for n in range(N):` ` for n in range(N):` `43` `59` ` for d_1 in range(d1):` ` for d_1 in range(d1):` `44` `60` ` c = target[n][d_1]` ` c = target[n][d_1]` `45` `61` ` loss[n][d_1] = -input[n][c][d_1] * weight[c]` ` loss[n][d_1] = -input[n][c][d_1] * weight[c]` `62` `46` `63` ` loss = np.sum(loss)` ` loss = np.sum(loss)` `47` `64` ` // print(loss)` ` // print(loss)` `48` `65` ` // -1.1` ` // -1.1` `66` `49` `67` `Example 3:` `Example 3:` `68` `50` `69` ` // weighted negative log likelihood loss, mean reduction` ` // weighted negative log likelihood loss, mean reduction` `51` `70` ` N, C, d1 = 2, 3, 2` ` N, C, d1 = 2, 3, 2` `52` `71` ` input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],` ` input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]],` `53` `72` ` [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]` ` [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]]` `54` `73` ` target = [[2, 1], [0, 2]]` ` target = [[2, 1], [0, 2]]` `55` `74` ` weight = [0.2, 0.3, 0.1]` ` weight = [0.2, 0.3, 0.1]` `56` `75` ` loss = np.zeros((N, d1))` ` loss = np.zeros((N, d1))` `57` `76` ` weight_total = 0` ` weight_total = 0` `58` `77` ` for n in range(N):` ` for n in range(N):` `59` `78` ` for d_1 in range(d1):` ` for d_1 in range(d1):` `60` `79` ` c = target[n][d_1]` ` c = target[n][d_1]` `61` `80` ` loss[n][d_1] = -input[n][c][d_1] * weight[c]` ` loss[n][d_1] = -input[n][c][d_1] * weight[c]` `62` `81` ` weight_total = weight_total + weight[c]` ` weight_total = weight_total + weight[c]` `82` `63` `83` ` loss = np.sum(loss) / weight_total` ` loss = np.sum(loss) / weight_total` `64` `84` ` // print(loss)` ` // print(loss)` `65` `85` ` // -1.57` ` // -1.57` `66` `86` `67` `87` `**Attributes**` `**Attributes**` `68` `88` `69` `89` `* **ignore_index**:` `* **ignore_index**:` `70` `90` ` Specifies a target value that is ignored and does not contribute to` ` Specifies a target value that is ignored and does not contribute to` `71` `91` ` the input gradient. It's an optional value.` ` the input gradient. It's an optional value.` `72` `92` `* **reduction**:` `* **reduction**:` `73` `93` ` Type of reduction to apply to loss: none, sum, mean (default).` ` Type of reduction to apply to loss: none, sum, mean (default).` `74` `94` ` 'none': the output is the loss for each sample. 'sum': the output` ` 'none': the output is the loss for each sample. 'sum': the output` `75` `95` ` will be summed. 'mean': the sum of the output will be divided by the` ` will be summed. 'mean': the sum of the output will be divided by the` `76` `96` ` sum of applied weights. Default value is 'mean'.` ` sum of applied weights. Default value is 'mean'.` `77` `97` `78` `98` `**Inputs**` `**Inputs**` `79` `99` `80` `100` `Between 2 and 3 inputs.` `Between 2 and 3 inputs.` `81` `101` `82` `102` `* **input** (heterogeneous) - **T**:` `* **input** (heterogeneous) - **T**:` `83` `103` ` Input tensor of shape (N, C) or (N, C, d1, d2, ..., dk).` ` Input tensor of shape (N, C) or (N, C, d1, d2, ..., dk).` `84` `104` `* **target** (heterogeneous) - **Tind**:` `* **target** (heterogeneous) - **Tind**:` `85` `105` ` Target tensor of shape (N) or (N, d1, d2, ..., dk). Target element` ` Target tensor of shape (N) or (N, d1, d2, ..., dk). Target element` `86` `106` ` value shall be in range of [0, C). If ignore_index is specified, it` ` value shall be in range of [0, C). If ignore_index is specified, it` `87` `107` ` may have a value outside [0, C) and the target values should either` ` may have a value outside [0, C) and the target values should either` `88` `108` ` be in the range [0, C) or have the value ignore_index.` ` be in the range [0, C) or have the value ignore_index.` `89` `109` `* **weight** (optional, heterogeneous) - **T**:` `* **weight** (optional, heterogeneous) - **T**:` `90` `110` ` Optional rescaling weight tensor. If given, it has to be a tensor of` ` Optional rescaling weight tensor. If given, it has to be a tensor of` `91` `111` ` size C. Otherwise, it is treated as if having all ones.` ` size C. Otherwise, it is treated as if having all ones.` `92` `112` `93` `113` `**Outputs**` `**Outputs**` `94` `114` `95` `115` `* **loss** (heterogeneous) - **T**:` `* **loss** (heterogeneous) - **T**:` `96` `116` ` The negative log likelihood loss` ` The negative log likelihood loss` `97` `117` `98` `118` `**Type Constraints**` `**Type Constraints**` `99` `119` `100` `120` `* **T** in (` `* **T** in (` `101` `121` ` tensor(double),` ` tensor(double),` `102` `122` ` tensor(float),` ` tensor(float),` `103` `123` ` tensor(float16)` ` tensor(float16)` `104` `124` ` ):` ` ):` `105` `125` ` Constrain input, weight, and output types to floating-point tensors.` ` Constrain input, weight, and output types to floating-point tensors.` `106` `126` `* **Tind** in (` `* **Tind** in (` `107` `127` ` tensor(int32),` ` tensor(int32),` `108` `128` ` tensor(int64)` ` tensor(int64)` `109` `129` ` ):` ` ):` `110` `130` ` Constrain target to integer types` ` Constrain target to integer types`
.. _l-onnx-op-negativeloglikelihoodloss-12: NegativeLogLikelihoodLoss - 12 ============================== **Version** * **name**: `NegativeLogLikelihoodLoss (GitHub) `_ * **domain**: **main** * **since_version**: **12** * **function**: False * **support_level**: SupportType.COMMON * **shape inference**: True This version of the operator has been available **since version 12**. **Summary** A NegativeLogLikelihoodLoss operator computes (weighted) negative log likelihood loss. Its "input" tensor has the shape of (N, C, d1, d2, ..., dk) where k >= 0. The "input" tensor contains log-probabilities for input[n, :, d_1, d_2,..., d_k] being in a class of [0, C). The operator's "target" input tensor has the shape of (N, d1, d2, ..., dk). It encodes class labels (one of C classes) or it may contain a special value (indicated by an attribute ignore_index) for N x d1 x d2 x ... x dk samples. The loss value for input[n, :, d_1, d_2,...d_k] being classified as class c = target[n][d_1][d_2]...[d_k] is computed as: loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k]. When an optional "weight" is provided, the sample loss is calculated as: loss[n][d_1][d_2]...[d_k] = -input[n][c][d_1][d_2]...[d_k] * weight[c]. loss is zero for the case when target-value equals ignore_index. loss[n][d_1][d_2]...[d_k] = 0, when target[n][d_1][d_2]...[d_k] = ignore_index If "reduction" attribute is set to "none", the operator's output will be the above loss with shape (N, d1, d2, ..., dk). If "reduction" attribute is set to "mean" (the default attribute value), the output loss is (weight) averaged: mean(loss), if "weight" is not provided, or if weight is provided, sum(loss) / sum(weight[target[n][d_1][d_2]...[d_k]]]), for all samples. If "reduction" attribute is set to "sum", the output is a scalar: sum(loss). See also https://pytorch.org/docs/stable/nn.html#torch.nn.NLLLoss. Example 1: // negative log likelihood loss, "none" reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] target = [[2, 1], [0, 2]] loss = np.zeros((N, d1)) for n in range(N): for d_1 in range(d1): c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1] // print(loss) // [[-3. -2.] // [-0. -2.]] Example 2: // weighted negative log likelihood loss, sum reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] target = [[2, 1], [0, 2]] weight = [0.2, 0.3, 0.1] loss = np.zeros((N, d1)) for n in range(N): for d_1 in range(d1): c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1] * weight[c] loss = np.sum(loss) // print(loss) // -1.1 Example 3: // weighted negative log likelihood loss, mean reduction N, C, d1 = 2, 3, 2 input = [[[1.0, 2.0], [2.0, 2.0], [3.0, 2.0]], [[0.0, 1.0], [2.0, 2.0], [1.0, 2]]] target = [[2, 1], [0, 2]] weight = [0.2, 0.3, 0.1] loss = np.zeros((N, d1)) weight_total = 0 for n in range(N): for d_1 in range(d1): c = target[n][d_1] loss[n][d_1] = -input[n][c][d_1] * weight[c] weight_total = weight_total + weight[c] loss = np.sum(loss) / weight_total // print(loss) // -1.57 **Attributes** * **ignore_index**: Specifies a target value that is ignored and does not contribute to the input gradient. It's an optional value. * **reduction**: Type of reduction to apply to loss: none, sum, mean (default). 'none': the output is the loss for each sample. 'sum': the output will be summed. 'mean': the sum of the output will be divided by the sum of applied weights. Default value is ``'mean'``. **Inputs** Between 2 and 3 inputs. * **input** (heterogeneous) - **T**: Input tensor of shape (N, C) or (N, C, d1, d2, ..., dk). * **target** (heterogeneous) - **Tind**: Target tensor of shape (N) or (N, d1, d2, ..., dk). Target element value shall be in range of [0, C). If ignore_index is specified, it may have a value outside [0, C) and the target values should either be in the range [0, C) or have the value ignore_index. * **weight** (optional, heterogeneous) - **T**: Optional rescaling weight tensor. If given, it has to be a tensor of size C. Otherwise, it is treated as if having all ones. **Outputs** * **loss** (heterogeneous) - **T**: The negative log likelihood loss **Type Constraints** * **T** in ( tensor(double), tensor(float), tensor(float16) ): Constrain input, weight, and output types to floating-point tensors. * **Tind** in ( tensor(int32), tensor(int64) ): Constrain target to integer types